JPH11234671A - Signal processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent occurrence of overflow and underflow of an image buffer verification device as to a buffer for smoothing in the case of transmission of coded moving image data. SOLUTION: A discrete cosine transform DCT coefficient quantized by a quantization device 2 and processed by a scan order converter 3 is given to a variable length coder 4, where the coefficient is coded in variable length and the result is smoothed by a buffer 5 and then transmitted. A rate controller 8 determines an objective code amount to each picture and compares it with a code amount for each picture by the variable-length coder 4. When the code amount is deficient, code amount supplement devices 10, 11 use a specific code denoting a deficiency and stuffing process is conducted. Furthermore, the rate controller 8 determines an objective code amount for each macro block and a code amount adjustment device 9 reduces the excess codes with respect to the objective code amount by reducing the quantization DCT coefficient or supplements the deficiency by fixed-length coding.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing apparatus for encoding digital video signals with high efficiency, and more particularly to a code amount control method.

[0002]

2. Description of the Related Art As a typical method for compressing a moving image,
"ITU-T White Book, Audiovisual / Multimedia Related (H Series) Recommendations" ITU Japan
Association Published February 18, 1995 pp.375-595 (hereinafter referred to as Reference 1)
) And "1.5 Mbit / s encoding of moving picture signal and accompanying audio signal for digital recording medium-Part 2: Moving picture" (hereinafter referred to as "MPEG2 method"). Japanese Industrial Standard X4322-1996)
Moving image compression standard ISO / IE specified in Reference 2)
C 11172-2 (commonly known as the MPEG1 system) is known.

The general procedure of moving picture compression by the MPEG1 system and the MPEG2 system is as follows.

[0004] A macroblock of a field or a frame as a compression unit is processed and compressed so as to be a signal specified in the MPEG1 system or the MPEG2 system. In the MPEG1 system and the MPEG2 system, digital moving image information is compressed by using redundancy of moving image information and human visual characteristics and deleting redundant information and information that is not important to human visual characteristics. The code amount of digital information (bit stream) obtained as a result of compression greatly changes depending on the properties of an input image, for example, the correlation between frames or fields and the amount of high spatial frequency components, even under the same compression conditions. Therefore, the code amount obtained as a result of compression changes every moment during one frame period or one field period which is a compression unit. In general, a target of the code amount is determined for each frame or field that is a compression unit, and the compression parameter is controlled so that the code amount matches the target.

In order to transmit such a stream via a transmission line having a constant data transfer rate, during a period in which the code amount is large, codes exceeding the transmission capacity are stored in a buffer to limit the transmission amount. It is necessary to take out the code from the buffer and transmit it at a predetermined data transfer rate during the period when the number is small. In order to perform such a smoothing process, a certain amount of information is stored in advance on the transmission side, transmission can be performed at a constant speed even if the generated code amount changes, and a buffer is also used on the reception side. It is necessary to prepare such that even if the code amount required for decoding changes, decoding can be continued.

FIG. 6 is a block diagram showing an example of such a conventional signal processing apparatus for moving image compression coding with code amount control, wherein 1 is an input terminal, 2 is a quantization device, and 3 is scan order conversion. The device, 4 is a variable length coding device, 5 is a buffer, 6 is an output terminal, 7 is a code amount totalizing device, and 8 is a rate control device.

In FIG. 1, a moving image signal input from an input terminal 1 is quantized by a quantization device 2 and irreversibly compressed. In the case of MPEG1 and MPEG2, the quantization device 2
, Discrete cosine transform (DCT) for each block (8 × 8 pixels) which is the minimum unit of a frame or a field
Processing has been done. In the scan order conversion device 3, the signal quantized by the quantization device 2 is stored in a built-in buffer, and its quantization unit (MPEG1 system) is processed so that the processing in the subsequent variable length coding device 4 becomes advantageous. And MPE
In the case of the G2 system, it is a quantized DCT coefficient.
Hereinafter, it is assumed that the encoding is performed by the MPEG1 system or the MPEG2 system, and the order of the quantization unit is referred to as a quantized DCT coefficient. As a method of the replacement, a zigzag scan method is used in the MPEG1 and 2 systems, but an alternate scan is further used in the MPEG2 system.

[0008] In the variable length coding device 4, the quantized DCT coefficients are coded by using the variable length code and lossless compression of the information amount is performed. The output signal of the variable length coding device 4 composed of such a code is temporarily stored in the buffer 5, read out, and output from the output terminal 6 as a bit stream. As described above, the buffer 5 absorbs the difference between the transmission rate due to the transmission capacity of the transmission path from the output terminal 6 and the actual code amount output from the variable length coding device 4, and this code amount changes. Even so, the transmission path can be transmitted at a constant transmission rate.

The code amount totalizing device 7 includes a variable length coding device 4
, And the rate control device 8 controls the quantization device 2 to change the size of the quantization scale for dividing the DCT coefficient in accordance with the result of the calculation. The code amount is controlled.

The control of the code amount is performed by the rate control device 8.
Sets the target value of the generated code amount for the next picture in accordance with the result of counting by the code amount counting device 7, and sets the quantization scale in the quantization device 2 so that the code amount actually generated becomes the target value. The control is performed so that the average of the code amount output from the variable length coding device 4 becomes the transmission rate of the transmission path. Therefore, if the generated code amount of the coded picture is smaller than the target value, a value obtained by subtracting the difference between the target value and the generated code amount from the target value is set as the target value of the next picture, and If the generated code amount of the picture obtained is larger than the target value, a value obtained by subtracting the difference between the generated code amount and the target value from the target value is set as the target value of the next picture. Then, the buffer 5 smoothes the code amount output from the variable length coding device 4 and sends the code amount to the transmission path so as to have a constant transmission rate.

[0011]

In the conventional code amount control method, when a difference occurs between the target value of the generated code amount and the actual code generation amount, the difference is fed back to the target value of the next picture to compensate. I do. However, if the generated code amount and the target value are far apart, the buffer may not be able to absorb the difference.

In general, since the code amount control is control by manipulating the compression parameter, if pictures having such characteristics are successively encoded by the MPEG1 system, the encoding apparatus fails in smoothing. At this time, p
The "Movie Buffer Verifier" described on pages 52-53 overflows and violates the MPEG1 system.

In the case of the MPEG2 system, such a problem can be avoided by using a variable bit rate. However, encoding at a fixed bit rate is often required for communication and broadcasting. Even in the case of the MPEG2 system, if pictures having such characteristics are continuously encoded at a fixed bit rate, smoothing fails, and p.
The "image buffer verifier" described on pages 531-535 overflows and violates the MPEG2 system.

The above-mentioned "moving image buffer verifier" and "image buffer verifier" are the same, and will be hereinafter referred to as "image buffer verifier". The “image buffer verifier” is a virtual verifier that assumes the capacity of the buffer on the receiving side. When the receiving side receives the encoded moving image data from the transmitting side, the encoded moving image data has a larger or smaller code amount per picture. In order to transmit such coded moving image data to the receiving side through a transmission path having a constant transfer rate and continuously decode the data,
It is necessary to have buffers on both the transmitting and receiving sides. The image buffer verifier simulates the capacity of the buffer on the receiving side on the transmitting side and verifies whether the receiving side can perform the decoding process without overflowing (overflowing) the buffer of this virtual capacity. is there.

Although the average code amount of a picture is the same as the transfer rate of the transmission path, if the deviation of the code amount between the pictures is large, it takes more time to transmit an image with a large code amount, so that the receiving side It is necessary to set the remaining amount of the buffer higher. For this reason, there is a concern that the buffer on the receiving side may overflow in a section in which images with a small code amount continue. Immediately after decoding an image having a large code amount, the buffer on the receiving side may underflow because the remaining amount of the buffer on the receiving side is reduced. Such overflows and underflows are against the standards of the MPEG1 and MPEG2 systems.

As a method for solving such a problem,
A method using stuffing has been proposed in JP-A-8-65683 and the like. The stuffing is to insert a codeword that can be inserted into a bit stream and discarded in decoding processing. Since the stuffing bits are discarded in the decoding process, the shortage of the actual code amount with respect to the target value can be filled without affecting the image.

In the MPEG2 system, GOPs, pictures,
It is possible to perform stuffing in units of slices, and in the MPEG1 system, GOP, picture,
Stuffing can be performed in units of slices and macroblocks. It is more advantageous in terms of image quality to perform stuffing in large image units such as pictures than in small image units such as macroblocks or slices. This is because, when stuffing is performed on a picture basis, the code amount can be more freely assigned to each macroblock according to the complexity of the image of each macroblock.

However, the stuffing process for each picture has the following problems.

The stuffing on a picture basis cannot be executed until the encoding of one picture is completed and the total of the code amount for one picture is calculated. In the case of encoding a moving image in real time, the period from the end of encoding for one picture to the start of encoding processing of the next picture is an extremely short period, for example, a vertical blanking period. . On the other hand, when stuffing is performed on a picture basis, the stuffing amount may be considerably large. Therefore, it is necessary to add a large amount of stuffing bits in a very short time. That is, it is necessary to write a bit stream (stuffing bits) at a very high transfer rate into the buffer (the buffer 5 in FIG. 4) of the encoding device. However, the buffer is RA
M, the transfer rate of input data is physically limited.

For the above reasons, it has been difficult to realize a stuffing processing apparatus for each picture in moving picture coding which performs coding in real time. This is the first problem.

In some cases, it may be desired that the amount of code generation in macroblock units be strictly set to a target value. However, in the case of the MPEG2 system, the standard specifies that
Stuffing in macro block units is not defined. Therefore, realizing filling of the code amount in macroblock units in the MPEG2 system is a second problem.

Further, even when a code amount extremely larger than the target value occurs, the smoothing may fail.
In a conventional encoding device, a quantization scale has been used as a compression parameter to be operated for code amount control. The quantization scale is information that must be added to the MPEG2 stream, and is specified in the above-mentioned documents 1 and 2. There is also an upper limit for the size of the quantization scale,
Values beyond that cannot be used for coding control.
If the code amount before quantization is extremely large, the code amount may not be compressed to the target value even if the quantization scale is set to the maximum value. If a frame or a field having such characteristics is continuously encoded, the encoding device fails in smoothing. At this time, the "image buffer verifier" underflows, violating the MPEG1 system and the MPEG2 system.

As a method for solving such a problem,
A method for forcibly cutting quantized DCT coefficients is disclosed in
It has been proposed in JP-A-8-46964 and JP-A-8-65683.

The method described in Japanese Patent Application Laid-Open No. 8-46964
After encoding of all macroblocks included in the picture, set a target value for each macroblock,
Again, the quantized DCT coefficients are forcibly cut so as to be smaller than the target value of each macroblock. Since this method performs compression processing for one picture in two stages, it is difficult to use it when encoding a moving image in real time.

In the method described in Japanese Patent Application Laid-Open No. 8-65683, a target value is determined for each macroblock, and the number of quantized DCT coefficients to be cut is determined according to the target value. It is something that advances. The problem with this method is that the number of quantized DCT coefficients to be cut is determined in advance before encoding, so that the actual code amount does not always match the target value. That is, even if this method is used, the underflow of the image buffer verifier cannot always be completely prevented.

As described above, it is difficult to completely prevent the underflow of the image buffer verifier in the moving picture coding in which the coding is performed in real time even by using the method conventionally proposed. This is the third problem.

An object of the present invention is to provide a signal processing apparatus which solves each of the above-mentioned problems, prevents overflow and underflow of an image buffer verifier, and enables video encoding in real time. It is in.

[0028]

In order to solve the above-mentioned first problem, according to the present invention, a code amount filling means is arranged at a subsequent stage of a buffer for smoothing a code amount.

According to the present invention, the encoding means does not insert a stuffing code when writing a bit stream into the buffer, but outputs the bit stream from the buffer to an image output terminal by using the code amount filling means. Output the stuffing code. This enables stuffing irrespective of the restriction on the transfer rate to the buffer.

However, the positions where the stuffing code can be inserted into the bit stream are MPEG1, MPEG
It is defined by two standards. At the encoding stage, it is easy to determine the position where the stuffing code is inserted, but in the state of being stored in the buffer in the form of a bit stream, it is easy to determine the position where the stuffing code is inserted. is not.

Therefore, in the present invention, at the stage of encoding by the encoding means, the code indicating the position where the stuffing code is to be inserted can be distinguished from other codes, and the codeword which is not permitted in the bit stream. (Hereinafter, referred to as a special code) is inserted into the bit stream from the encoding means, and information indicating the stuffing amount (hereinafter, referred to as stuffing amount information) follows the special code.
Is also inserted into this bit stream and stored in the buffer. As described above, since the codeword that is not permitted in the bitstream is used as the code related to the stuffing, it is easy to distinguish the codeword from the codeword constituting the bitstream, and the position to be stuffed can be easily detected.

During the normal operation, the code amount filling device reads the bit stream from the buffer and detects the special code, and when the special code is not detected, transmits the read bit stream as it is from the output terminal. The normal operation of outputting to the channel is continued, but when the special code is detected, the reading of the bit stream from the buffer is suspended, and the amount of stuffing code indicated by the special code is output from the output terminal to the transmission line. Then, when the output of the stuffing code of the amount indicated by the special code is completed, the code amount filling apparatus resumes reading the bit stream from the buffer and returns to the normal operation.

In order to solve the second problem, the present invention is to fill the code amount in macroblock units by using a fixed length code.

In the MPEG1 system and the MPEG2 system,
In order to encode the quantized DCT coefficients, a variable length code is defined. The variable length code is a short code length for data with a high frequency of occurrence and a long code length for data with a low frequency of occurrence. This is a compression method aimed at reducing the total code amount by assigning, but MPEG2
In the scheme, the quantized DCT coefficient is assigned a variable length code of 2 to 17 bits and a fixed length code of 24 bits.

Originally, this fixed-length code is used to handle data that is not covered by the variable-length code table. However, data to which the variable-length code is assigned is encoded with the fixed-length code. There is no problem. Accordingly, in the present invention, the coding amount is filled in macroblock units by coding the quantized DCT coefficient data that can be subjected to variable length coding with a fixed length code having a bit length larger than that of the variable length code. It is.

In order to solve the third problem, according to the present invention, the target value determining means determines a next encoding value based on a target value for each picture and a total code amount of encoded macroblocks in the picture. The target value of the code amount after coding of the macroblock to be encoded is set, and the coding means performs coding so that the code amount is smaller than the target value for each macroblock.

More specifically, in a macroblock, for example, each quantized DCT coefficient is encoded in a zigzag scan order or an alternate scan order, and
Every time a coefficient is encoded, the total code amount up to that point is monitored, and when this total code amount exceeds the target value, encoding of the quantized DCT coefficients thereafter is stopped, and the end code (EO
B) is output to stop encoding the quantized DCT coefficients in this macroblock.

In this case, since the respective quantized DCT coefficients are encoded in the zigzag scan order or the alternate scan order, when the encoding is stopped halfway, the high-frequency components are deleted. In many cases, the visual characteristics are not important, so that the image quality is not particularly deteriorated. This method is sufficiently possible even when encoding a moving image in real time, and can completely prevent underflow of the image buffer verifier.

[0039]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a signal processing device according to the present invention, wherein 9 is a code amount adjusting device, 10 and 11 are code amount filling devices, and portions corresponding to FIG. A duplicate description will be omitted.

In the figure, a code amount adjusting device 9 is provided between the scan order conversion device 3 and the variable length coding device 4, and a code amount filling device 10 is provided between the variable length coding device 4 and the buffer 5.
Is the code amount filling device 1 between the buffer 5 and the output terminal 6.
1 are provided, and the rate control device 8 is provided with a quantization device 2, a code amount adjustment device 9, and a code amount filling device in accordance with the output of the variable length coding device 4 and the result of aggregation of the code amount aggregation device 7. The device 10 is controlled. Other configurations are the same as those of the conventional signal processing device shown in FIG.

With such a configuration, the rate control device 8
In addition to controlling the quantization device 2 to adjust the size of the quantization scale, the code amount adjusting device 9 and the code amount filling device 1 are controlled.
By controlling 0 and 11, the code amount per picture is directly adjusted.

That is, the code amount adjustment device 9 receives the sequence of the quantized DCT coefficients from the scan order conversion device 3 and adjusts the code amount in macroblock units. As the code amount adjustment, the scan order conversion device 3 and the variable length coding device 4 are controlled using the control information supplied from the rate control device 8 to reduce and fill the code amount. .

The code amount filling device 10 performs filling (stuffing) of the code amount in picture, slice, and macroblock units in the MPEG1 system, and in the picture and slice units in the MPEG2 system. By controlling the variable length coding device 4 using the control information obtained from the device 8, the code amount is filled. Further, when the code amount filling apparatus 10 cannot fill the stuffing code of the entire target code amount due to the limit of its own stuffing ability, the code amount filling apparatus 10 cannot fully fill the stuffing code with the special code which is not defined by the MPEG1 and MPEG2 systems. The stuffing amount information indicating the code amount is inserted into the bit stream from the variable length coding device 4. Alternatively, instead of filling the stuffing code, the code amount filling apparatus 10 may insert only the special code and the stuffing amount information into the bit stream. In this case, the stuffing amount information indicates the code amount of the stuffing code in picture units.

The code amount filling device 11 is used for filling the code amount in picture units. Code amount filling device 11
Analyzes the bit stream, detects the special code inserted by the code amount filling device 10, obtains the amount of stuffing code from the stuffing amount information, and fills the bit stream with this amount of stuffing code.
In this case, the special code and the stuffing amount information are discarded, and the stuffing code is inserted from the position where the special code was inserted. Thereby, the output terminal 6
Output from MPEG1, MPEG2
Since special codes not specified in the system have been removed,
It satisfies the MPEG1 and MPEG2 systems.

Here, the stuffing is to insert a code word (stuffing code) that can be inserted into a bit stream and is discarded in decoding processing. The stuffing process in the MPEG2 system is described in next_start_p.
Defined as code () function. next_start_code ()
The function is defined as a function that advances the current position under decoding to the first bit of a byte boundary and removes all zero bytes after the start code. Therefore, ne
The stuffing by the xt_start_code () function can be performed in 1-byte units.

In the MPEG2 system, pp. 4
As defined in 04-418, the next_start_code () function has the following layers: sequence header, extension data and user data, user data, sequence extension, sequence display extension, sequence scalable extension, group of picture header, Picture header, picture coding extension, quantization matrix extension, picture display extension, temporal scalable image extension, spatial scalable image extension, copyright extension, picture data,
Included in the slice. Therefore, stuffing can be performed in these data units. The macro_block, macroblock mode, motion vector, and coded block pattern do not include a next_start_code () function, and therefore cannot be stuffed in these data units.

In the MPEG1 system, p.
As defined in −24, the video sequence layer, sequence header, image group layer, image layer,
An xt_start_code () function is included, and stuffing can be performed on these data units. Next_start for MPEG1 macroblock layer and block layer
Although _code () is not included, another stuffing is defined in the macroblock layer. In the first part of the definition of the macroblock layer, stuffing by repeating an 11-bit stuffing code is defined. Therefore, the stuffing in MPEG1 macroblock units is in 11-bit units.

Since the stuffing code is discarded in the decoding process, the image quality does not change at all by the stuffing code. This is an advantage of code amount filling using stuffing.

FIG. 2 shows the code amount filling apparatus 10 in FIG.
11 is a block diagram showing a specific example of 11, where 10a is a stuffing unit, 10b is a code amount comparing unit, 11a is a buffer unit, 11b is a stuffing unit, 11c is a special code detecting unit, and a part corresponding to FIG. Have the same reference numerals.

In the figure, the code amount filling device 10 is composed of a stuffing unit 10a and a code amount comparison unit 10b. In the case of the MPEG2 system and the MPEG1 system, stuffing is performed in 1-byte units for pictures and slices. In the case of the MPEG1 system,
Stuffing is performed in 11-bit units for each macro block.

The rate control device 8 determines the lower limit target value of the code amount for each picture, slice or macroblock. The code amount comparing unit 10b calculates the lower limit target value and the actual code amount (total value) obtained from the code amount totaling device 7.
When the actual code amount is smaller than the lower limit target value, the variable length coding device 4 and the stuffing unit 1
0a is controlled to perform the stuffing process.

More specifically, when the actual code amount is smaller than the lower limit target value, the code amount comparison unit 10b transmits a stop signal to the variable length coding device 4 and information indicating the stuffing amount to the stuffing unit 10a. Supply each. This stuffing amount is a difference between the lower limit target value and the code amount, and is in byte units. Upon receiving the stop signal, the variable-length encoding device 4 stops the process at the stage of outputting the next_start_code () function, and instructs the stuffing unit 10a to perform the stuffing process. The stuffing unit 10a normally outputs the bit stream output from the variable length coding device 4 as it is, but when receiving a stuffing processing instruction from the variable length coding device 4, the code amount comparing unit 10b
Output the stuffing code of the amount specified by.
The stuffing is realized by the above operation.

FIG. 3A schematically shows a bit stream output from the stuffing unit 10a, and shows a vertical stream between a coded picture A and the next coded picture B. During the ranking period, an amount of stuffing data corresponding to the difference between the code amount of the picture A and the lower limit target value is added.

However, the code amount filling apparatus 10 has the following problems. Since the moving image compression encoding apparatus of the present invention assumes encoding in real time, the time that can be spent for encoding is limited. Stuffing is next_s
Since it can be performed only at the output timing of the tart_code () function, the period during which the stuffing process can be performed is, for example,
The vertical blanking period is limited. On the other hand, since the buffer 5 is constituted by a RAM, the transfer rate of the input data is limited. Therefore, the code amount that can be filled by the code amount filling device 10 (the code amount of the stuffing code) has an upper limit, and stuffing exceeding the upper limit cannot be performed.

In order to solve such a problem, in this embodiment, when the stuffing capability of the code amount filling device 10 is exceeded or when the stuffing capability of the code amount filling device 10 is exceeded, the code amount filling device 10 is used. 11 performs code amount filling (stuffing).

For this reason, when the stuffing amount specified by the code amount comparing section 10b exceeds the stuffing ability, the code amount filling device 10 provides information necessary for the code amount filling device 11 to perform the stuffing process. Can be distinguished from other codes in the bitstream,
In addition, information indicating a stuffing amount to be inserted following the special code (ie, stuffing amount information) is inserted into the bit stream as a code not permitted in the bit stream (ie, special code). Insert As the special code, for example, a “reserved” code defined in the MPEG1 and MPEG2 system is used. The “reservation” indicates that the value can be extended and used in the future. This special code is used temporarily between the code amount filling device 10 and the code amount filling device 11, and is discarded by the code amount filling device 11.

FIG. 3B shows the stuffing unit 10a when the stuffing amount specified by the code amount comparing unit 10b exceeds the stuffing capability of the buffer 5.
1 schematically shows a specific example of a bit stream output from a picture A. In a vertical blanking period between an encoded picture A and a next encoded picture B, This figure shows a case where a part of the stuffing code corresponding to the difference between the code amount and the lower limit target value, special data indicating the remaining amount of the difference, and stuffing amount information are added.

FIG. 3C shows another specific example of the bit stream output from the stuffing unit 10a when the stuffing amount specified by the code amount comparing unit 10b exceeds the stuffing capability of the buffer 5. FIG. 4 schematically shows an example, in a vertical blanking period between a coded picture A and a next coded picture B, special data and stuffing quantity information for all designated stuffing quantities. And the case where "." Is added.

The code amount filling device 10 is a variable length coding device 4
, The timing of the stuffing process can be directly instructed by the variable-length coding device 4. On the other hand, since the code amount filling device 11 is connected to the variable length coding device 4 via the buffer 5,
The stuffing processing timing is changed by the variable length coding device 4
Cannot be obtained directly from The special code is inserted at the position where the stuffing code is to be inserted in the bit stream output from the buffer 5 for the purpose of facilitating the timing control of the stuffing process in the code amount filling device 11.

Following the special code, the stuffing amount information is added to the bit stream as a fixed length code and output from the stuffing unit 10a. The code amount filling device 11 reads the bit stream from the buffer 5,
The special code is detected from this bit stream.

Since the buffer 5 requires a large storage capacity, it is constituted by a DRAM or an SDRAM. D
The RAM and the SDRAM can process an operation of continuously reading a large amount of data (ie, a burst read) at a high speed. Since a high-speed operation is required, the buffer 5 employs a burst read operation using a DRAM or an SDRAM.

The code amount filling device 11 must read out a bit stream in a unit of data. In order to facilitate the special code detecting operation and the stuffing process in the code amount filling device 11, the code amount filling device 10 Adjusts the insertion position of the special code in the bit stream. For example, the insertion position of the special code is set at the head of a group of data to be subjected to the burst read processing (for example, a special code detection unit 1 described later of the code amount filling apparatus 11).
When 1c processes data as a unit of data in units of 4 bytes, the special code detection unit 11c positions the special code at the head of the unit of 4 bytes where detection is easy. By doing so, there is also obtained an effect that the special code detecting unit 11c is easily made. The insertion position of the special code (that is, the insertion start position of the stuffing code in the code amount filling device 11) is determined by the code amount filling device 10 in FIG.
As shown in (b), when a part of the stuffing code is added, the adjustment is performed so that the special code is located next to the stuffing code.

Next, the code amount filling device 11 shown in FIG. 2 will be described.

The code amount filling device 11 includes a buffer unit 11a,
The stuffing unit 11b and the special code detecting unit 11c perform stuffing of m bytes (where m is an integer of 1 or more) in picture units.

Buffer section 11a and stuffing section 11
b is controlled by the special code detecting unit 11c.
The bit stream output from the buffer 5 is temporarily stored in the buffer unit 11a, and is stored in the special code detection unit 1a.
The special code and the stuffing amount information supplied to 1c and inserted by the code amount filling device 10 are detected. Since it is promised that the special code exists at the head position of a continuous data group read by the burst read operation from the buffer 5, the detection of the special code only needs to be checked.

When no special code is detected from the supplied bit stream, the special code detecting unit 11c
The output of the bit stream is permitted to the buffer unit 11a, and the stuffing unit 11b is instructed to select and output the bit stream. When the special code is detected, the stuffing amount information following the special code is detected, and the buffer is detected. The stuffing unit 11a instructs the unit 11a to stop the output operation of the bit stream, and instructs the stuffing unit 11b to select the stuffing code, and indicates the stuffing amount indicated by the stuffing amount information detected following the special code. The staffing section 11b
To supply.

When the buffer unit 11a receives a bit stream output instruction from the special code detection unit 11c, the buffer unit 11a outputs the bit stream held by itself, and outputs the bit stream from the special code detection unit 11c. When the instruction to stop the output operation is received, the output of the bit stream is stopped. Upon receiving the bit stream selection instruction from the special code detection unit 11c, the stuffing unit 11b outputs the bit stream output from the buffer unit 11a to the output terminal 6 as it is, but issues a stuffing code selection instruction from the special code detection unit 11c. Upon receiving the stuffing code, the stuffing code of the amount indicated by the information indicating the stuffing amount supplied from the special code detecting unit 11c is output to the output terminal 6.

FIG. 3D schematically shows a portion of the bit stream output from the stuffing section 11b into which the stuffing code is inserted, and FIG. 3B or FIG. From the indicated bit stream.

With the above operation, when the code amount from the variable length coding device 4 is smaller than the lower limit target value, even if a stuffing code having a stuffing amount exceeding the stuffing ability of the code amount filling device 10 is required, the image buffer verification is performed. The stuffing can be performed without overflowing the vessel.

FIG. 4 is a block diagram showing a specific example of the code amount adjusting device 9 shown in FIG.
9b is a non-zero component counting section, 9c is a code amount comparing section, and portions corresponding to FIG. 1 are denoted by the same reference numerals.

In the figure, a code amount adjusting device 9 reduces or fills a code amount in macroblock units, and comprises a coefficient adding unit 9a, a non-zero component counting unit 9b, and a code amount comparing unit 9c. ing. The coefficient adding unit 9a is a DCT coefficient (hereinafter referred to as a quantized DCT coefficient) in a block (8 × 8 pixels) quantized by the quantization device 2.
Is added to the non-zero component of. The non-zero component counting section 9b counts the number of non-zero components of the quantized DCT coefficient in one block. The code amount comparing section 9c compares the lower limit target value and the upper limit target value in macroblock units obtained from the rate control device 8 with the code amount obtained from the variable length coding device 4, and is obtained from the non-zero component counting section 9b. On the basis of the number of non-zero components, the scan order converter 3, the coefficient adding unit 9a, and the variable-length encoder 4
Is to be controlled.

Next, the operation of the code amount adjusting device 9 will be described.

Quantized DCT from scan order converter 3
When coding the coefficients, the code amount comparison unit 9c receives the upper target value and the lower target value for each macroblock from the rate control device 8, and allocates these to each block in the macroblock. In the macro block of the image of the 4: 2: 0 format, there are a total of six blocks of four luminance signal (Y) blocks, one color difference signal (Cb) block and one color difference signal (Cr) block. An upper target value and a lower target value are assigned to these six blocks, respectively, according to the upper target value from the rate controller 8. There are various possible allocation methods, and a specific example will be described with reference to FIG. 5 using an upper limit target value as an example.

In this example, in this specific example, four Y
The blocks are represented by Y1, Y2, Y3, Y
If there are four blocks, Y1, Y2, Y3, Y4, Cb, Cr
And assigns the upper limit target value while encoding in the order of.

[0075] Now, when the upper limit target value assigned to the macroblock to be encoded from the rate control unit 8 and M _1, first, as shown in FIG. 5 (a), 1 of the upper limit target value M ₁ / assign a value of 4 M _1/4 as the upper limit target value of the Y1 block. The variable-length encoding device 4 receives Y from the scan order conversion device 3 via the coefficient adding unit 9a.
One block of quantized DCT coefficients is sequentially encoded. Each time one quantized DCT coefficient is encoded (when a zero quantized DCT coefficient continues, a code indicating the number of successive zero DCT coefficients is added). Every time it is obtained), the accumulated code amount of the quantized DCT coefficient encoded so far in the Y1 block is detected and supplied to the code amount comparison unit 9c, and the accumulated code amount and the upper limit assigned to the Y1 block are calculated. It is compared with the target value M _1/4.

[0076] encoding is completed for all the quantized DCT coefficients of the Y1 block, the (actual code amount of the Y1 block) accumulated code amount at this time as N _1, when it is N ₁ <M _1/4, the As shown in FIG. 5B, the value of M ₂ = M ₁ −N ₁ is 1 /
4, i.e., the upper limit target value for Y2 block which then encode the M _2/4. Then, the encoding process in this Y2 block is performed, and the accumulated code amount at this time and the upper limit target value M
It is compared with the _2/4.

[0077] In the same manner, the actual code amount of the Y2 block When N _2, as shown in FIG. 5 (c), M ₃ =
As M ₂ -N _2, assigned to M _3/2 as the upper limit of Y3 block, when the actual code amount of the Y3 block and N _3, then, as shown in FIG. _{5 (d), M 4 =} M as ₃ -N _3, assigned to M _4/2 as the upper limit of Y4 block,
If the actual code amount of Y4 block and N _4, then, as shown in FIG. 5 (e), as _{M 5 = M 4 -N 4,} M 5/2
Assignment as the upper limit of Cb blocks and the actual code amount of the Cb block and N _5, then, as shown in FIG. 5 (f), assigning the M ₅ -N ₅ as the upper limit of Cr blocks.

Although the above description is for the upper target value, the same applies to the assignment of the lower target value.

In this way, 1/4 of the upper limit target value M ₁ given from the rate control device 8 is assigned to the luminance signal block to be encoded first, and the upper limit value is assigned to the remaining three luminance signal blocks. 1 / of the value obtained by sequentially subtracting the actual code amount in the previously encoded luminance signal block from M ₁
4 or と し て is assigned as an upper limit target value, and the remaining value obtained by subtracting the code amount of the four luminance signal blocks from the upper limit target value M _{1 is 1} for the color difference signal block.
Therefore, the upper limit value to be assigned to each block can be obtained by subtraction and multiplication by 1/2, 1/4, and such multiplication can be performed by bit shift. The circuit configuration of the circuit can be reduced in scale.

Further, according to this allocation method, many target values are allocated to the luminance signal. This means that a larger amount of code is allocated to a luminance signal having a larger amount of information, which is advantageous in human visual characteristics.

Next, as the operation of the code amount adjusting device 9, the operation of reducing the code amount will be described first. In this case, the code amount adjusting device 9 uses the quantization D as the code amount reducing means.
The CT coefficient is deleted.

Now, when the quantized DCT coefficients of a certain block are converted from the scan order transforming device 3 and are read out in order of the coefficients having the lower frequencies, the quantized DCT coefficients are changed in variable length via the coefficient adding section 9a. It is supplied to the encoding device 4 and encoded with a variable length code (however, a zero quantized DCT coefficient is encoded into a variable length code representing the continuous number thereof). The variable length coding device 4
Further, every time the quantized DCT coefficient is encoded, the accumulated code amount of the already encoded quantized DCT coefficient of the block is obtained. The code amount comparing unit 9c constantly compares the upper limit target value assigned to this block as described above with the cumulative code amount in this block obtained by the variable length coding device 4, and this cumulative code amount Does not reach the upper limit target value, the variable-length encoder 4 continues the processing operation of reading and coding the quantized DCT coefficients from the scan order converter 3, and the accumulated code amount becomes equal to the upper limit target value. If it exceeds, the operation of the scan order conversion device 3 is stopped, the reading of the quantized DCT coefficient after this point in this block is prohibited, 0 is output, and the operation of the variable length coding device 4 is also stopped. In this case, the variable length coding device 4
The code of the quantized DCT coefficient when the accumulated code amount exceeds the upper limit target value is not output.

As described above, when the code amount of the block from the scan order conversion device 3 exceeds the upper limit target value assigned to this block, the code amount is limited to the upper limit target value or less. The encoding is performed, and an excessive amount of code is reduced.

When such code amount reduction processing is performed, since the reduced quantized DCT coefficient is a high-frequency component that is inconspicuous visually, it is possible to avoid deterioration of the image quality due to this, and it is possible to prevent human vision. This is advantageous in characteristics.

Next, a description will be given of a code amount filling operation as an operation of the code amount adjusting device 9. In this case, as a code amount filling method, switching between fixed length coding / variable length coding and quantization are performed. Forced addition of DCT coefficients is used.

The variable-length code assigns a short code length to data with a high frequency of occurrence and a long code length to data with a low frequency of occurrence, thereby reducing the overall code amount. This is a compression method for the purpose. In the MPEG2 system, 2 for the quantized DCT coefficient
A variable length code from bits to 17 bits and a fixed length code of 24 bits are allocated. Originally, the fixed-length code is used to handle data that is not covered by the variable-length code table, but there is no problem in encoding the data to which the variable-length code is assigned with the fixed-length code. . In the present invention, the code amount is filled by performing fixed-length coding on quantized DCT coefficient data that can be subjected to variable-length coding.

However, in the MPEG2 system, level 0
The fixed-length code of the quantized DCT coefficient is prohibited. If the quantized DCT coefficients of the non-zero components (that is, non-zero) are small, the code amount may not reach the lower limit target value even if all non-zero components are fixed-length coded. In such a case, fixed-length encoding is performed by sequentially adding a value 1 as a weak signal to quantized DCT coefficients of zero components (that is, zero). Even if such a weak signal is added, the image quality deterioration in human visual characteristics is small. The means for adding the value 1 is the coefficient adding unit 9a in the code amount adjusting device 9.

As in the case of the code amount reduction operation, the variable length coding device 4 codes the quantized DCT coefficients to obtain the accumulated code amount in the block at that time, and the code amount comparison unit 9
c compares the lower limit target value assigned to this block with this accumulated code amount. At this time, the non-zero component counting unit 9b counts the non-zero components of the quantized DCT coefficients output from the quantization device 2. The code amount comparing unit 9c estimates the number of quantized DCT coefficients to be fixed-length coded and the number to be added with quantized DCT coefficients from the count number of the non-zero component and the accumulated code amount, and from the result, A determination is made as to whether to perform fixed-length coding or variable-length coding, and to determine whether to add a quantized DCT coefficient. Specifically, the determination is made according to the following algorithm.

That is, T is the lower limit target value assigned to the block to be encoded, n is the count value of the non-zero component by the non-zero component counting unit 9b, and f is the fixed-length code length (MPEG2
In the case of f = 24), the quantization length D
The accumulated code amount of the block output every time the CT coefficient is encoded is s, and the number of the current non-zero component of the quantized DCT coefficient output from the quantization device 2 is i (i = 1 to 1).
n), the number of the current zero component of the quantized DCT coefficient is k (k =
1-64-n), the determination as to whether to use a variable-length code or a fixed-length code and the addition of a quantized DCT count are made according to the following determination formula. s + f × (ni)> T
When f × (n + k) <T, add a quantized DCT coefficient when f × (n + k) <T This operation includes multiplication, but since the fixed code length f is a fixed value, It can be realized by a table drawing circuit.

The code amount comparing section 9c supplies the variable length coding device 4 with the result of the determination whether to perform the variable length coding or the fixed length coding, and the variable length coding device 4 according to the determination result. When it is determined that the variable-length encoding is performed, the supplied quantized DCT coefficient is encoded into a variable-length code of 2 to 17 bits. To a fixed-length code of Thus, the code amount in this block increases.

When the code amount comparing section 9c determines that a quantized DCT coefficient is added, the result of the determination is supplied to the coefficient adding section 9a. The coefficient adding unit 9a supplies the quantized DCT coefficient from the scan order conversion device 3 to the variable length coding device 4 as it is when the instruction to add the quantized DCT coefficient is not received from the code amount comparing unit 9c. Upon receiving an instruction to add a quantized DCT coefficient from the code amount comparing unit 9c, a weak signal having a value of 1 is replaced with a variable length as a quantized DCT coefficient instead of the zero component of the quantized DCT coefficient from the scan order converter 3. It is supplied to the encoding device 4. The variable-length encoding device 4 performs variable-length or fixed-length encoding on the weak signal in accordance with the instruction from the code amount comparing unit 9c. In this way, the zero components are individually coded as inconspicuous non-zero components, and therefore the code amount in this block increases.

[0092]

As described above, according to the present invention,
By directly filling and deleting codes, overflow and underflow of the image buffer verifier can be completely prevented, and moving image coding in real time becomes possible.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a signal processing device according to the present invention.

FIG. 2 is a block diagram showing a specific example of two code amount filling devices in FIG.

FIG. 3 is a diagram schematically illustrating a form of a bit stream in which stuffing is performed between two code amount filling devices in FIG.

FIG. 4 is a block diagram showing a specific example of a code amount adjusting device in FIG. 1;

FIG. 5 is an explanatory diagram showing a specific example of a method of assigning a target code amount for each block in the specific example shown in FIG. 4;

FIG. 6 is a block diagram illustrating an example of a conventional signal processing device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 input terminal 2 quantization device 3 scan order conversion device 4 variable length coding device 5 buffer 6 image output terminal 7 code amount totaling device 8 rate control device 9 code amount adjusting device 9a coefficient adding portion 9b non-zero component counting portion 9c code Amount comparison unit 10 Code amount filling unit 10a Stuffing unit 10b Code amount comparison unit 11 Code amount filling unit 11a Buffer unit 11b Stuffing unit 11c Special code detection unit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yukio Isobe 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture Inside the multimedia system development headquarters of Hitachi, Ltd. (72) Inventor Hideo Arai Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa 292 Hitachi Multimedia Systems Development Division

Claims (5)

[Claims] 1. A signal processing apparatus having encoding means for encoding a moving image signal, comprising: for each picture of the moving image signal, a target value of a code amount of a code sequence obtained by encoding the moving image signal; First means for determining, a first code indicating a deficiency of a code amount with respect to the target value for each picture, and a second code indicating an insertion position of the first code in the code string. Second means for inserting into the code sequence, detecting the first code from the code sequence from the second means based on the second code, and detecting the first and second codes in the code sequence. And a third means for replacing the code with a code word discarded in the decoding process of the code amount corresponding to the shortage of the code amount indicated by the first code. 2. A signal processing apparatus comprising an encoding unit for dividing each picture of a moving image into blocks and encoding the blocks to obtain a code sequence, wherein a target value of a code amount of the code sequence in the picture is provided. From the target value of the code amount of the code sequence in the picture and the total code amount of the already-coded blocks in the picture, A target value determining means for determining a target value of the code amount of the column; and controlling the encoding means such that the block has a code amount within a target value determined for the block by the target value determining means. A signal processing device comprising: a control unit. 3. The target according to claim 2, wherein a video signal is orthogonally transformed for each block to derive a spatial frequency component, and a code amount of a code sequence in the block is determined for the block. A signal processing device comprising means for cutting a high-frequency component of the spatial frequency component so as to be smaller than the value. 4. A signal processing apparatus having an encoding means for compressing a moving image signal using variable length encoding, wherein one of a plurality of codeword candidates having different word lengths is assigned to the same encoding unit. A signal processing device comprising means for selecting and outputting as a codeword. 5. The method according to claim 4, wherein a target code amount is determined for each macroblock defined by the moving picture compression standard H.262, and the shortage of the actual code amount with respect to the target code amount is determined by using a code having a long word length. A signal processing device comprising means for filling by selecting a word candidate.