JPH0998431A - Device and method for transmitting image signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce an encoded data amount to be transmitted by adaptively significance-judging motion compensation residual signals and to control the encoded data amount to be transmitted to be fixed at all times by performing feedback buffering. SOLUTION: As for input image data, motion vector is detected by a block unit by block matching for instance with the locally decoded images of a frames in the past. By the motion vector of the block unit, motion compensation is performed to a previous frame, predicted images are prepared and a difference with the input image data is calculated. For the predicted residual, significance judgement is performed for respective blocks and the residual signals of the block judged as a significant block are quantized, then variable length encoded and supplied to a buffer circuit 11. A hierarchical branch to perform transmission is appropriately selected from the residual amount of the buffer circuit 11.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal transmitting apparatus and method capable of further reducing the data transmission amount when compressing a blocked image signal.

[0002]

2. Description of the Related Art Conventionally, in image compression, a general method is to reduce the amount of information by quantizing a prediction residual obtained by some preprocessing with a predetermined number of bits. As an example, as shown in FIG. 7, the prediction residual is supplied to the motion compensation predictive coding circuit 42 via the input terminal 41. In the motion compensation predictive coding circuit 42, the motion compensated predictive image and A residual signal is generated for each sample value with the input image. This prediction residual is supplied to the quantization circuit 43, and is quantized in the quantization circuit 43 with a bit number smaller than the original bit number.

The quantized value from the quantization circuit 43 is supplied to the variable length coding circuit 44, variable length coding is performed, and it is supplied to the buffering circuit 45 and the buffering circuit 45.
Controls the quantizing step width of the quantizing circuit 43 according to the buffer remaining amount, so that the rate of the output data to be transmitted is controlled to be constant. As one of effective methods for reducing the data amount (hereinafter, referred to as the encoded data amount) generated by quantizing the prediction residual, the difference between the supplied sample value and its predicted value is calculated. By doing so, the obtained prediction error is quantized with a predetermined number of bits, but since all prediction residuals are quantized, the amount of encoded data becomes considerable.

As one method of suppressing the amount of coded data, the prediction residual is divided into blocks and a significant judgment is made to judge whether or not it is significant to transmit each block,
In the significance judgment, there is a method in which a block judged to be insignificant is not transmitted.

[0005]

However, in this case, the probability of appearance of a block that becomes significant when the block size is small is reduced, but since the total number of blocks is increased, overhead such as flags is increased, resulting in encoding. There was a problem that the amount of data would increase.
In addition, if the block size is small, the number of significant blocks changes significantly depending on the content of the image, which makes it difficult to control the amount of information to be constant.

Therefore, an object of the present invention is to reduce the amount of encoded data by adopting a hierarchical structure and to make the amount of encoded data constant by performing feedback buffering. And to provide a method.

[0007]

According to a first aspect of the present invention, there is provided a video signal transmission apparatus for encoding a prediction residual between an input pixel value and its prediction value, wherein 1 block, and the block size of the 1st block is subdivided into a 1st block significance determining means for determining whether or not the 1st block is significant based on the activity of the 1st block. To generate a second block, and for each second block, based on the activity of the second block, a significance judgment means for the second block that performs significance judgment, and a block size of the second block are subdivided. To generate a third block, and based on the activity of the third block for each third block, a significant judgment means for the third block, and a significant judgment for the first block. When it is determined to be significant by the stage, the processing is branched to the second block or the third block in a predetermined manner so that the encoded data amount becomes small, and A prediction residual included in a block determined to be significant by the significance determining unit for blocks, the significance determining unit for second blocks, or the significance determining unit for third blocks, and a flag for identifying significance / insignificance Is an image signal transmission device.

According to a second aspect of the present invention, in the image signal transmission device for encoding the prediction residual between the input pixel value and its prediction value, the prediction residual is transferred to the first block. The first block is divided, and the block size of the first block is subdivided, and the second block is divided into the second block and the second block. Second block significant judgment means for making a significant judgment for each second block, a first block significant judgment means, and a second block significant judgment means.
Quantizing means for quantizing a prediction residual included in a block determined to be significant by the significance determining means for the block, variable length coding means for variable length coding the output of the quantizing means, and variable length coding A feedback type information amount control configured to make the amount of information to be transmitted constant by changing the threshold to which significance is judged based on the buffer to which the output of the digitizing means is supplied and the remaining amount of the buffer. The image signal transmitting device is characterized by comprising means and means for transmitting a flag for identifying significant / insignificant and an output of the buffer.

Further, in the image signal transmission method according to the fifth aspect of the present invention, in which the prediction residual of the input pixel value and the prediction value thereof is encoded, the prediction residual is transferred to the first block. Dividing and determining whether or not each first block is significant based on the activity of the first block, and subdividing the block size of the first block,
Generating a second block, performing a significance judgment for each second block based on the activity of the second block, and subdividing the block size of the second block to generate a third block, Third for every third block
By the step of making a significance judgment based on the activity of the block, and the significance judgment of the first block,
When it is determined to be significant, the second block or the third block is set in a predetermined manner so that the encoded data amount becomes small.
A prediction residual included in a block which is determined to be significant by the significance judgment of the first block, the significance judgment of the second block or the significance judgment of the third block, and The image signal transmission method is characterized in that a flag for distinguishing significance / insignificance is transmitted.

According to a sixth aspect of the present invention, in the image signal transmission method for encoding the prediction residual between the input pixel value and its prediction value, the prediction residual is transferred to the first block. Dividing and determining whether or not each first block is significant based on the activity of the first block, and subdividing the block size of the first block,
A step of generating a second block and performing a significance judgment for each second block; and a prediction residual included in the block determined to be significant by the significance judgment of the first block and the significance judgment of the second block. Quantizing, variable-length coding the output of the quantization stage, a buffer to which the variable-length coding output is supplied, and changing the significance judgment threshold value based on the remaining capacity of the buffer. And a feedback type information amount control configured to make the amount of information to be transmitted constant, and transmitting a flag for identifying significant / insignificant and an output of the buffer. Image signal transmission method.

The amount of encoded data to be transmitted can be reduced by hierarchically determining the prediction residual error, and the amount of encoded data can be controlled to be constant by performing feedback buffering.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. A block diagram of an embodiment of the image signal transmission device is shown in FIG. The image data of the current frame is input from the input terminal indicated by 1, and the image data is
It is supplied to the motion detection circuit 2 and the subtractor 3. Subtractor 3
Then, the prediction image data from the prediction circuit 8 is subtracted from the input image data, and the prediction residual is supplied to the significance determination circuit 9 for layering. In the significance judgment circuit 9, as will be described later, a threshold for significance judgment is supplied from the ROM table 12, significance judgment is performed based on the threshold, and a significance flag and a significance block for identifying significance / insignificance are provided. Data is output.

The significance flag from the significance judgment circuit 9 is supplied to the buffer circuit 11, and the data of the significance block is supplied to the quantization circuit 4. In the quantization circuit 4, the supplied significant block data is quantized based on the quantization step width supplied from the ROM table 12. That is, when the remaining amount of the buffer circuit 11 is small, feedback control is performed so that the quantization step width becomes large, and when the remaining amount is large, feedback control is performed so that the quantization step width becomes small.

The quantized value from the quantizing circuit 4 is supplied to the inverse quantizing circuit 5 and the variable length coding circuit 10. The variable-length coding circuit 10 further reduces the amount of information for the supplied quantized value of the significant block by using entropy coding such as Huffman coding, and supplies the information to the buffer circuit 11 together with the significant flag. It The inverse quantization circuit 5 performs inverse quantization of the quantized value, that is, decoding based on the same quantization step width as the quantization step width supplied to the quantization circuit 4, and the original prediction residual is Restored.

The restored prediction residual is supplied to the adder 6 and is added to the predicted image data from the prediction circuit 8 in the adder 6. From the output of the adder 6 to the frame memory 7
Supplied to The locally decoded past frame image is stored in the frame memory 7, the data from the adder 6 is written to the past frame image, and the written image data is stored in the frame memory 7. Is supplied to the motion detection circuit 2 and the prediction circuit 8. The motion detection circuit 2 uses the input image data and the image data supplied from the frame memory 7 to detect a motion vector for each block by block matching, for example. The detected motion vector is supplied from the motion detection circuit 2 to the buffer circuit 11 and the prediction circuit 8.

In the prediction circuit 8, motion compensation is performed on the image data from the frame memory 7 by the supplied motion vector to generate predicted image data. The generated prediction image data is supplied from the prediction circuit 8 to the subtractor 3 and the adder 6.

The buffer circuit 11 is supplied with a significance flag, a variable length coded quantized value, a motion vector, and a threshold number (address) shown in FIG. The quantized value input to the buffer circuit 11 changes in information amount depending on the content of the image, but the output of the buffer circuit 11 is output at a constant rate according to the capacity of the transmission path. Therefore, the difference becomes the remaining amount of the buffer circuit 11, and when the input quantized value increases, the remaining buffer amount decreases, and when the input quantized value decreases, the remaining buffer amount increases. The buffer remaining amount is supplied from the buffer circuit 11 to the ROM table 12.

In the ROM table 12, the supplied buffer remaining amount is determined by a certain number of steps (for example, 16 steps).
It is input to the ROM table 12 which outputs the threshold value for significance judgment and the quantization step width using this as an address. The contents of the ROM table 12 are determined in advance based on measurement, simulation, experience, etc. so that the threshold value for significance determination and the quantization step width may be increased or decreased collectively according to the remaining buffer capacity. Moreover, a value is selected so that the buffer circuit 11 does not overflow or underflow. And the ROM table 12
Is supplied to the significance judgment circuit 9, and R
The quantization step width from the OM table 12 is supplied to each of the quantization circuit 4 and the inverse quantization circuit 5, and the buffer circuit 11 is supplied with a threshold number.

Here, an example of a hierarchical structure of blocks is shown in FIG.
Shown in The prediction residual is divided into, for example, a first block having a size of 16 × 16 pixels (first layer: L1), and first, the maximum absolute value (maximum error) of the prediction residuals in the target block is detected. Then, this maximum value is compared with a predetermined threshold value. As a result, if the maximum value is smaller than the threshold value, it is determined to be an insignificant block, the significance flag is set to "0", and the prediction residual is not transmitted.

On the other hand, when the maximum residual is larger than the threshold value, it is determined that the block is a significant block, a significant flag is set (set to "1"), and control is moved to the significant determination of the lower hierarchy. . The block determined to be significant in the first layer is further subdivided into a second block having a size of 8 × 8 pixels (second layer: L2), and similar significance determination is performed,
The block determined to be significant in the hierarchy is further divided into a third block having a size of 4 × 4 pixels (third hierarchy: L
3), significance judgment is performed.

Although the hierarchical branching of blocks has been described above as being performed in the order of L1-L2-L3, when hierarchical branching is performed in the order of L1-L2, the hierarchical branching of L1-L3 is performed. In some cases, hierarchical branching may be performed in order, or hierarchical branching may be performed in L2-L3 order. In order to determine how to perform hierarchical branching, a simulation using a large number of image data is performed in advance and a mode of hierarchical branching that reduces the amount of encoded data is obtained.

FIG. 3 shows an example of this simulation calculation, and shows the relationship between the threshold value for significance judgment and the amount of encoded data when a 2-bit quantizer is assumed, for example. FIG. 3 also shows that the average number of bits exceeds 2 bits, but this is because not only the prediction residual but also data such as a motion vector, a significant flag, and a threshold number are transmitted. The number of bits per prediction error represents the ratio of significant blocks.

According to FIG. 3, for each threshold,
The mode of hierarchical branching in which the average number of bits is minimum is not constant. That is, the mode of the hierarchical branch in which the average number of bits is the minimum with respect to the threshold value is L1-L2-L3, or
It may be 1-L3. Therefore, it is possible to increase the compression efficiency by controlling the mode of hierarchical branching according to the threshold value based on the simulation result of FIG.

Next, an example of the contents of the ROM table 12 is shown in FIG. In FIG. 4, when the buffer remaining amount is large, the one with a small address is assigned, and when the buffer remaining amount is small, the one with a large address is assigned, and the threshold value for the significance judgment and the quantization step width are assigned to each address. It has been set against. This quantization step width is shown for the luminance value Y and the color difference signals U and V because the input digital component signal is processed.

Regarding the threshold value for significance judgment, the threshold value is small when the buffer remaining amount is large (the address is small),
The threshold is high when the buffer is low (the address is large). Regarding the quantization step width, when the buffer remaining amount is large (the address is small), the quantization step width is small and the buffer remaining amount is small (the address is large).
Sometimes the quantization step size is large.

ROM table 12 according to the remaining buffer capacity
The significance judgment threshold value read out from is supplied to the significance judgment circuit 9, and by comparing the maximum residual of the block with the threshold value, whether or not it is significant is hierarchically judged. The mode of hierarchical branching is selected according to the threshold value.

Similarly, the quantization step width read from the ROM table 12 according to the remaining buffer amount is supplied to the quantization circuit 4 and the inverse quantization circuit 5, and the quantization circuit 4 determines that it is a significant block. The residual signal is quantized, and the inverse quantization circuit 5 performs local decoding in order to perform local decoding.
The prediction residual is decoded.

Although the quantization circuit 4 in this example performs linear quantization, nonlinear quantization having different quantization step widths depending on the level, for example, ADRC (Dynamic Range Adaptive Coding) may be used. The ADRC is a method for adaptively removing redundancy in the level direction by utilizing local correlation of images. That is, within the dynamic range of 0 to 255 of the 8-bit original data, the intra-block dynamic range required for requantization for each block is significantly reduced. The required number of bits can be significantly reduced from the original 8 bits.

The ROM table 12 for the fixed length ADRC is shown in FIG. In this FIG. 5, as in the case of the ROM table 12 in FIG. 4 described above, the case where the buffer remaining amount is large is assigned to the one having a smaller address, and the case where the buffer remaining amount is smaller is assigned to the one having a larger address. A threshold value and a quantization step size are set for each address. Here, Th1 is set as the quantization step width corresponding to the threshold number. Note that variable-length ADRC, which increases the number of quantization bits as the dynamic range increases, can also be used as a quantization method.

Here, FIG. 6 shows a block diagram of an example of the hierarchical significance determination processing circuit 4. The prediction residual input from the input terminal 21 is supplied to the first layer blocking circuit 22, the second layer blocking circuit 23, and the third layer blocking circuit 24. The first layer blocking circuit 22 divides it into 16 × 16 blocks and supplies them to the maximum residual detection circuit 25 and the selection determination circuit 33.

In the maximum residual detection circuit 25, the supplied 1
The maximum absolute value of the residual is detected for each 6 × 16 block. The detected maximum residual is supplied to the comparison circuit 30,
It is compared with the threshold value supplied from the terminal 28. If it is larger than the threshold value, flag1 is set to 1, and if it is smaller than the threshold value, flag1 is set to 0 and supplied to the selection judgment circuit 33. It That is, in the comparison circuit 30, the significance determination is performed, and if it is determined to be significant, flag1 is set to 1, and if it is determined to be insignificant, flag1 is set to 0.

In the second layer blocking circuit 23, 8 × 8
It is divided into blocks and supplied to the maximum residual detection circuit 26 and the selection judgment circuit 33. The maximum residual detection circuit 26 detects the maximum absolute value of the residual for each supplied 8 × 8 block. The maximum residual detected is calculated by the comparison circuit 31.
Is supplied to the terminal 28 and is compared with the threshold value supplied from the terminal 28. If it is larger than the threshold value, flag2 is set to 1. If it is smaller than the threshold value, flag2 is set to 0 and the selection judgment circuit 33. That is, in the comparison circuit 31, a significant judgment is made, flag1 is set to 1 when it is judged to be significant, and fl when it is judged to be insignificant.
0 is set in ag2.

Similarly, in the third layer blocking circuit 24, the maximum residual difference detecting circuit 27 is divided into 4 × 4 blocks.
And the selection judgment circuit 33. The maximum residual detection circuit 27 detects the maximum absolute value of the residual for each supplied 4 × 4 block. The detected maximum residual is supplied to the comparison circuit 32 and compared with the threshold value supplied from the terminal 28. If it is larger than the threshold value, flag3 is set to 1 and if it is smaller than the threshold value. , Flag3 are set to 0 and supplied to the selection judgment circuit 33. That is, in the comparison circuit 32, the significance determination is performed, and if it is determined to be significant, flag3 is set to 1, and if it is determined to be insignificant, flag3 is set to 0.

The threshold value supplied from the terminal 28 is also supplied to the ROM 29, and a selection signal indicating in advance which hierarchical branch should be selected according to the threshold value is written therein. . Therefore, the selection signal corresponding to the supplied threshold value is read from the ROM 29 and supplied to the selection judgment circuit 33. In the selection judgment circuit 33, flag1 and prediction residual from the first layer, flag2 from the second layer
The prediction residual and the flag3 and the prediction residual from the third layer are supplied, and the flag transmitted from the output terminal 34 and the significant block data transmitted from the output terminal 35 are selected according to the selection signal.

In general, as can be judged from the data of FIG. 3, when the threshold value is small, the L1-L3 hierarchical branch is performed, and when the threshold value is slightly increased, the L2-L3 hierarchical branch is performed. If the threshold value increases further, L1-
The hierarchical branch of L2-L3 is selected.

The significant flags are grouped together in a tree structure without compression according to the hierarchical branch of the block, or run length Huffman coding is performed on the flags for each layer in frame units. ,
Further, the amount of information may be reduced. However, in this case, a frame memory is required.

In this embodiment, the first, second and third blocks are generated in parallel. However, after the first block is generated, the second block is generated and then the third block is generated.
It is also possible to appropriately select the hierarchical branch by generating the block of.

Further, in this embodiment, the significance value is determined by comparing the absolute value of the maximum value of the prediction residual with the threshold value, but the sum of squares or the sum of products of the prediction residual and the threshold value are compared. It is also possible to make a significant judgment by making a comparison.

[0039]

According to the present invention, the compression efficiency can be improved by hierarchically determining the significance of prediction residual blocks and changing the hierarchical branching according to the threshold value. By performing feedback buffering, it is possible to control the amount of information to be constant.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of an image signal transmission device of the present invention.

FIG. 2 is a schematic diagram showing an example of a hierarchical block according to the present invention.

FIG. 3 is a diagram showing an example of a bit rate of the hierarchical branching method according to the present invention.

FIG. 4 is a diagram showing an example of a ROM table for rate control based on linear quantization according to the present invention.

FIG. 5 is a diagram showing an example of a ROM table for rate control based on nonlinear quantization according to the present invention.

FIG. 6 is a block diagram showing an example of a significance judgment circuit of the present invention.

FIG. 7 is a block diagram of compression encoding using conventional feedback buffering.

[Explanation of symbols]

 2 Motion Detection Circuit 3 Subtractor 4 Quantization Circuit 5 Inverse Quantization Circuit 6 Adder 7 Frame Memory 8 Prediction Circuit 9 Significance Judgment Circuit 10 Variable Length Coding Circuit 11 Buffer Circuit 12 ROM Table

Claims (8)

[Claims] 1. An image signal transmission apparatus for encoding a prediction residual between an input pixel value and its prediction value, wherein the prediction residual is divided into a first block and the first block. Based on the activity of the first block, the first block significance determining means for determining whether or not the first block is significant, and the block size of the first block is subdivided,
Block is generated, and the second block is generated for each second block.
And a block size of the second block is further subdivided, and a third block significance determination means for performing a significance determination based on the activity of the block
Block is generated, and the third block is generated for each of the third blocks.
When the significance is determined to be significant by the significance determining means for the third block and the significance determining means for the first block based on the activity of the block, the encoded data amount is small. So that
The processing is branched to the processing to the second block or the third block in a predetermined mode, and the significance determining means for the first block, the significance determining means for the second block or the An image signal transmitting apparatus, characterized in that the prediction residual included in the block determined to be significant by the significance determining means for the third block and the flag for identifying the significance / insignificance are transmitted. . 2. An image signal transmission apparatus for encoding a prediction residual between an input pixel value and its prediction value, wherein the prediction residual is divided into a first block and the first block. Based on the activity of the first block, the first block significance determining means for determining whether or not the first block is significant, and the block size of the first block is subdivided,
Second block, and the second block significant judgment means for performing the significance judgment for each of the second blocks, the first block significance judgment means, and the second block significance judgment means. Quantizing means for quantizing the prediction residual included in the block determined to be significant, variable length coding means for variable length coding the output of the quantizing means, and output of the variable length coding means Is supplied, and a feedback type information amount control means configured to keep the amount of information transmitted by changing the threshold value for significance determination based on the remaining amount of the buffer. The image signal transmission device is characterized in that the flag for identifying the significance / insignificance and the output of the buffer are transmitted. 3. The image signal transmission device according to claim 2, wherein the quantization means adaptively changes a quantization step width of linear quantization or non-linear quantization. . 4. The image signal transmitting apparatus according to claim 1 or 2, wherein the significance determining means compares the maximum absolute value of the prediction residual with a predetermined threshold value, and determines the maximum value. An image signal transmitting apparatus, wherein when the value is larger than the threshold value, it is determined to be a significant block, and when the maximum value is smaller than the threshold value, it is determined to be an insignificant block. 5. An image signal transmission method for encoding a prediction residual between an input pixel value and its prediction value, wherein the prediction residual is divided into a first block and the first block. Determining whether each of the first blocks is significant based on the activity of the first block, and dividing the block size of the first block into
Block is generated, and the second block is generated for each second block.
Determining the significance based on the activity of the block, and subdividing the block size of the second block into the third block.
Block is generated, and the third block is generated for each of the third blocks.
The step of performing a significance judgment based on the activity of the block and the significance judgment of the first block described above are performed in a predetermined manner so that the encoded data amount becomes small when the significance is judged. The processing is branched to the processing to the second block or the third block, and is significant by the significance judgment of the first block, the significance judgment of the second block or the significance judgment of the third block. An image signal transmission method, wherein the prediction residual included in the determined block and a flag for identifying the significance / insignificance are transmitted. 6. An image signal transmission method for encoding a prediction residual between an input pixel value and its prediction value, wherein the prediction residual is divided into a first block and the first block. Determining whether each of the first blocks is significant based on the activity of the first block, and dividing the block size of the first block into
Generating the block and performing the significance judgment for each of the second blocks, and the prediction included in the blocks determined to be significant by the significance judgment of the first block and the significance judgment of the second block. The step of quantizing the residual, the step of variable-length coding the output of the quantization stage, the buffer to which the output of the variable-length coding is supplied, and the significant judgment based on the remaining amount of the buffer. A feedback type information amount control configured to make the transmitted information amount constant by changing the threshold value, and transmits the flag for identifying the significant / insignificant and the output of the buffer. A method for transmitting an image signal, characterized in that. 7. The image signal transmission method according to claim 6, wherein the quantization adaptively changes a quantization step width of linear quantization or non-linear quantization. 8. The image signal transmission method according to claim 5 or 6, wherein in the significance determination, the maximum value of the absolute values of the prediction residuals is compared with a predetermined threshold value, and the maximum value is determined. Is larger than the threshold, it is determined to be a significant block, and if the maximum value is smaller than the threshold, it is determined to be an insignificant block.