JPH04227185A - Encoding output data quantity control system - Google Patents

Abstract

PURPOSE:To control the quantity of data in encoding at a constant level, to eliminate inconvenience on picture quality, to realize the real time processing of a moving image by a microprocessor, etc., to reduce the quantity of data, and to apply this system to an accumulation system medium in an encoding system with high efficiency which efficiently performs the encoding of an image signal with the small quantity of codes. CONSTITUTION:Vertual quantization step S2 is directly found from activity An at a feedforward control part 10, and it is corrected afterwards, and also, control for the suppression or emphasis of a high frequency component at a variable LPF 5, and that for quantization step at a quantizer 7 in several tens block unit are performed by a feedback control part 17.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to a high-efficiency encoding method for efficiently encoding an image signal with a smaller amount of code in a device for recording, transmitting, and displaying an image signal. This invention relates to a method for controlling the amount of encoded output data to keep the amount constant.

0002

2. Description of the Related Art Prediction residuals in predictive coding and orthogonal transform coefficients in transform coding have a considerably biased level distribution of generated signals. Therefore, by assigning codes of different lengths according to the frequency of occurrence of signal levels, the average value of the data amount can be made shorter than that of fixed-length codes. The variable length code is determined by a Huffman code or the like, and the more biased the frequency of data occurrence, the lower the total amount of data. By using variable length codes in this way, the average amount of data can be made smaller than with fixed length codes while maintaining the same quality of the reproduced signal. However, since the amount of data generated is not constant, in an actual device, it is necessary to control the amount of data generated so that it becomes uniform.

The most common method of controlling the amount of data is feedback control using a data output buffer. This is to reduce the amount of data by coarsening the quantization steps as the amount of encoded data increases and the amount of data stored in the buffer increases. Such a control method is used in systems such as video conferencing. On the other hand, when considering the application of encoding to recording media,
In addition to simply recording and reproducing, functions such as special reproduction are desired. Furthermore, in editing encoded image data, the data length needs to be fixed in units of several frames (fields) in order to replace the image data.

In such a case, although the average value of the amount of data is kept at a constant value in the feedback control, it is not possible to reliably keep the amount of data within a certain value in the desired unit. Therefore, as a data amount control method that keeps the data amount within a certain value,
Prior application 1, Japanese Patent Application No. 2-62887 ``Encoded output data amount control method and its decoding device'' and Prior Application 2, Japanese Patent Application No. 2-223981 ``Encoded output data amount control method'' by the same inventor There is. This method uses variable-length coding and performs both feedforward and feedback control to ensure that the amount of data is kept within a certain value without reducing coding efficiency or deteriorating image quality. be.

[0005]

[Problem to be solved by the invention] In conventional feedback control, high coding efficiency can be obtained by variable length coding, but it is not possible to reliably keep the amount of data within a certain value in a certain unit such as a frame unit. However, there was a problem in that it was difficult to apply to storage media. On the other hand, in the prior application 1, it is necessary to transmit information on the quantization step of each block determined by feedback control to the decoding device, and as the blocks are made smaller, the information increases.
On the other hand, when the blocks are made larger, there is a problem in that control accuracy decreases and data loss occurs.

[0006] Prior application 2 does not require additional information, but changes the image by suppressing and emphasizing high frequency components.
There was a problem with image quality. Furthermore, in either case, the amount of calculation required for feedforward processing is large, and real-time processing of moving images is difficult to implement using a microprocessor or the like. Due to the accuracy of the quantization step that is set, even with the optimal value set in feedforward processing, there was a problem that an error of several percent occurred in the amount of data, resulting in loss. The present invention was made with attention to the above points, and minimizes the loss of data amount by controlling both the quantization step and the suppression and emphasis of high frequency components, and allocating the deviation in data amount. By calculating the appropriate quantization value directly from the overall activity, feedforward processing is speeded up, so there is no problem with image quality, and real-time video processing can be realized with a microprocessor, etc., and the amount of data is reduced. The purpose of this invention is to provide a method for controlling the amount of encoded output data that has little loss and can be applied to storage media.

[0007]

[Means for Solving the Problems] In order to solve the above-mentioned problems, the present invention provides: (1) When determining the degree of activity of an image for each block and controlling the encoded output data amount by feedback type control. a means for summing the activities of each block and calculating a temporary quantization step from that value; and a means for calculating the predicted data amount of each block from the temporary quantization step and each activity, and calculating the predicted data amount for each block and the total and the desired total. A method for controlling the amount of encoded output data is provided, comprising: a correction means for correcting the provisional quantization step based on the amount of data;

(2) The amount of encoded output data for each block of an image is predicted, and feedback control is performed based on the difference between the predicted value and the actual value to bring the data amount closer to the target value. The method includes a predicted data amount correction means for correcting the predicted data amount of each block by multiplying the predicted data amount of each block by the value of the ratio obtained by dividing the target total data amount by the total predicted data amount. Provides an encoded output data amount control method characterized by:

000
9] (3) High frequency component changing means for changing the degree of suppression or emphasis of high frequency components on a block-by-block basis in order to control the output data amount of image encoding; A method for controlling the amount of encoded output data is provided, comprising a quantization step width changing means for adding the quantization steps and changing the quantization step width in units of a plurality of blocks based on the result. .

0010

[Operation] The present invention uses feedforward processing to determine a provisional quantization value, and collects the activities obtained for each block in units (for example, one frame) in which the amount of data is desired to be constant, and combines that value with the target data. A temporary quantization step is obtained by directly determining an appropriate quantization step from the amount, determining the predicted data amount in that case from each block activity, and correcting the quantization step from the difference from the target value. Multiply the data amount prediction value of each block by the ratio of the data amount predicted value of one frame obtained from the temporary quantization step and the target value, so that the frame total value of the corrected predicted value is the same as the target value. Make it. The difference between the corrected data amount predicted value and the actual data amount for each block is accumulated, and then the degree of suppression or emphasis of high frequency components is controlled for each block, and the control information is further accumulated. Control the quantization step on a block-by-block basis.

First, regarding feedforward control,
By first directly determining the quantization step from the total activity, the process of determining the predicted data amount for each block and summing them to determine the predicted frame data amount can be performed only about two times. By correcting the predicted block data amount, loss in data amount caused by a shift in frame data amount caused by the precision of the quantization step is eliminated. By controlling the prefilter on a block-by-block basis and the quantization step on a tens-of-block basis, little information is generated, and the control range for suppressing or emphasizing high-frequency components is reduced, so there are fewer problems with image quality. Become.

0012

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing a first embodiment of an encoding apparatus using an encoded output data amount control method according to the present invention. In FIG. 1, an image signal input from an image input terminal 1 is supplied to a high-efficiency encoding processing section 2, subjected to high-efficiency encoding processing, and image data output is taken out from a data output terminal 3. The high-efficiency encoding processing unit 2 further includes a frame memory 4, a variable LPF 5, an orthogonal transformer 6, a quantizer 7, a variable length encoder 8, and a buffer 9.

[0013] This high-efficiency encoding processing unit 2 uses the video conference system (px64 kbps codec) standardized by the International Telegraph and Telephone Consultative Committee CCITT, SG15, and the I/O codec.
Still images standardized by SO-IEC SC2/WG8/JPEG and ISO-IEC SC2/WG11/MP
It is the same as the encoding processing unit used for storage media videos standardized by EG. The operation of the high-efficiency encoding processing section 2 will be explained below. The image input signal is input from the image input terminal 1 and supplied to the frame memory 4. The frame memory 4 is for compensating for delays caused by feedforward control, which will be described later, and is a memory for an interval in which the amount of data is desired to be constant (here, for example, one frame), and is used to store image signals for one frame. , variable LPF5
is supplied to.

The variable LPF 5 is a variable pass band filter, and operates to change the degree of spatial filtering according to a Pn signal supplied from an FB control unit 17, which will be described later, and supplies its output signal to the orthogonal transformer 6. . The orthogonal transformer 6 first transforms the input signal into coefficients for each block of 8×8 pixels or the like. Discrete cosine transform (DCT) is used as an orthogonal transform method, as is well known.
is a DC coefficient (average) and all other conversion coefficients are AC coefficients (alternating current). Each coefficient, which is an output signal of the orthogonal transformer 6, is supplied to a quantizer 7. The quantizer 7 uniformly quantizes the input signal at intervals of quantization steps S, which will be described later. The output signal of the quantizer 7 is supplied to a variable length encoder 8. Here, since most of the AC components are concentrated around 0, they are converted into a preset Huffman code or the like.

In this Huffman code, the word length of 0 is usually the shortest, and the word length increases as the absolute value of the level increases. Furthermore, regarding 0, instead of encoding each coefficient, the number of successive 0s is encoded. The amount of output data obtained in this way differs from block to block. The output signal of the variable length encoder 8 is supplied to a buffer 9. Buffer 9 is FIFO (First In
The input speed changes according to the amount of data output from the variable length encoder 8, but once it is stored and converted to a constant data rate, it is sent via the data output terminal 3. High-efficiency encoded image encoded data is extracted.

Next, among the data amount control system, which is a characteristic part of the present invention, first, F
The F control section 10 will be explained. The FF control unit 10 further includes an activity detector 11, an activity memory 12, a histogram detector 13, a temporary quantization step generator 14, a temporary quantization step corrector 15, and a frame data amount predictor 16. The FF control section 10 outputs a temporary quantization step S1 determined for each frame based on the activity of the image signal, and its operation will be explained below.

First, an image signal input from the image input terminal 1 is supplied to the activity detector 11 . The activity detector 11 detects activity (
The activity) An is calculated for each block. This block has a size of about 16×16 pixels, and does not necessarily have to be the same as an orthogonal transformation block, and several orthogonal transformation blocks may be grouped together. Here, if this block is made larger, the data amount of feedback control information to be described later will be reduced, but the fluctuation of the data amount will also increase, resulting in a loss, so it is desirable that the block be large enough to balance the two. Since the activity is used to predict the amount of data for variable length encoding, the higher the correlation with the activity, the better, and it is given by the average absolute value of the BPF output.

[0018] Since this value is not directly related to the decoding device side, if a better activity detection method is developed later, it can be replaced without causing any compatibility problems. The obtained activity An is stored in the activity memory 12.
And it is supplied to the histogram detector 13. The activity memory 12 operates to delay one frame to compensate for the delay in feedforward control. In order to quickly obtain the amount of data per frame, the histogram detector 13 counts the number Hi of blocks of the activity value for each activity value, and uses the temporary quantization step generator 14 and the temporary quantization step corrector. 15 and a frame data amount predictor 16.

[0019] Here, assuming that one block is 16 x 16 pixels, the number of blocks in an image of 720 x 480 pixels is 1.
There are 350 activity values, but it is sufficient to prepare about 32 levels of activity values exponentially, so rather than calculating the predicted data amount for each block and summing the frames, multiply the predicted data amount of each activity by the number of blocks, The amount of frame data can be determined more quickly by summing them. Let Ai be the activity of each activity level i, and let D(Ai, S) be the amount of data when the block of activity Ai is quantized by S.

[0020]

[Math 1]

The histogram detector 13 has a memory (11 bits x 32) corresponding to the number of blocks for each activity value, and simply detects a histogram by incrementing the memory content of the input activity value by 1. The temporary quantization step generator 14 operates to obtain the activity A of the entire frame, and multiplies the number of blocks of each activity by its activity value and adds it to obtain the activity A as shown in the following equation (1). demand.

[0022]

[Math 2]

[0023] Then, the temporary quantization step is determined from the activity A and the target data amount D of one frame.If the quantization step S and activity A are determined in advance, how much data amount D will be? is calculated statistically, and a table is created in which S for A and D can be calculated from the average value. Here, D
Assuming that does not change from frame to frame, this becomes a table for finding S from A, which is extremely simple. If D is variable, it will be a two-dimensional table, but this process is only an approximation, and it will be corrected later, so a low precision is fine. I can't get it.

[0024] Therefore, A and D are 64 levels, 6 bits.
It can be realized with a 12-bit input ROM. The value obtained in this way is used in the preliminary quantization step S before correction.
2 and is supplied to the temporary quantization step corrector 15. The temporary quantization step corrector 15 first calculates the predicted frame data amount D2 at the preliminary quantization step S2 before correction from the output Hi of the histogram detector 13 using the following (2).
Find it like the formula.

[0025]

[Math 3]

Since D2 normally deviates from the target value D, S2 is corrected from the average relationship between S and D obtained for the processing of the temporary quantization step generator 14. A temporary quantization step S1 is obtained and supplied to a frame data amount predictor 16, a block data amount predictor 18, and an adder 19 of an FB control section 17, which will be described later. The easiest way to obtain the temporary quantization step S1 is to calculate S and D.
Assuming that there is an inversely proportional relationship, S1 = S2 × D2
/D. This process can be repeated several times by regarding S1 as S2. At that time, if the correction is excessive, S
will diverge, so it is desirable to keep the amount of correction to a small value. On the other hand, if D2 can already be sufficiently close to D, this correction process may not be necessary. In that case, S1
=S2. The frame data amount predictor 16 calculates the frame data amount D1 for S1 using the following equation (3), and supplies (D/D1) to the multiplier 20 of the FB control unit 17.

[0027]

[Math 4]

D1 is closer to D than D2, but includes some errors. This means that the type of value of S is 1.06
When set to the m power (m=1, 2, 3,...32), etc.,
Even if the optimal S is selected, there will be an error of several percent. In the previous application, this was ignored, but this can be corrected by multiplying the predicted data amount Dn1 of each block by D/D1, and the following (4)
As shown in the formula, it is possible to set the final encoded output data amount to D instead of D1.

[0029]

[Math 5]

Based on this idea, it can be said that even if S1 is not a very accurate value, it will become an appropriate S in accordance with D1 corrected by feedback control, which will be described later. Here, Dn is an integer value, but Dn1 or (D/D1)
Dn1 does not need to be an integer; rather, unless it is a real value, Σ(D/D1)Dn1=D will not hold. Next, the operation of the FB control section 17 will be explained. The FB control unit 17 includes a block data amount predictor 18 , a multiplier 20 , a subtracter 21 , a cumulative adder 22 , an LPF controller 23 , a cumulative adder 24 , a quantization step controller 25 , and an adder 19
It is made up of.

Feedback control is performed by suppressing or emphasizing high frequency parts and controlling both the quantization step. First, the block data amount predictor 18 calculates the activity An supplied from the activity memory 12.
From this, Dn1=D(An, S) is determined and supplied to the multiplier 20. The multiplier 20 multiplies (D/D1) supplied from the frame data amount predictor 16 by Dn1,
Dn1 (D/D1) is determined and supplied to the subtracter 21. The subtracter 21 calculates a predicted value Dn1(
D/D1) is subtracted to obtain the deviation amount, and the cumulative adder 22
is supplied to. The cumulative adder 22 cumulatively adds the amount of deviation between the actual data amount Dn and the predicted value, updates the value for each block, and supplies the updated value to the LPF controller 23.

The LPF controller 23 outputs a control signal Pn and supplies it to the variable LPF 5 and the cumulative adder 24. The control in this variable LPF 5 is the same as in the prior application 2, and when the actual amount of data becomes larger than the predicted value, that is, D
If n-Dn1 (D/D1) is positive, the LPF (low pass filter) is strengthened to suppress high frequency components. In the opposite case, the suppression of high frequency components is reduced or rather emphasized. With this kind of control, it is possible to fix the amount of data, as shown in Prior Application 2, but if S1 deviates significantly from its original value, the control will cause a state in which the amount of data is strongly suppressed or emphasized. This will cause problems in image quality.

Therefore, the cumulative adder 24 receives the control signal Pn
are cumulatively added and supplied to the quantization step controller 25. This is to ultimately control the quantization step. This control does not compensate for variations in the predicted data amount of each block, but rather compensates for shifts in the entire S1 frame, so there is no need to perform it in units of blocks, but in units of about several dozen blocks (block slice).
You can do it with The cumulative addition of the control signal Pn is performed for each block, and is output to the quantization step controller 25 at the break of block slices, and is reset at the same time.

FIG. 3 is a diagram for explaining block slicing. In FIG. 3, one block has 16×16 pixels, and all the pixels in the horizontal direction are one block slice. One block slice consists of 45 blocks, 72
Contains 0x16 pixels. There are 30 block slices in one frame. The control signal Pn is
If the appropriate state is set to 0, positive when suppressing high-frequency components, and negative when emphasizing high-frequency components, then ΣPn will be almost 0 if only block variations exist, and quantization is too fine and high-frequency components It is positive when the amount of data is suppressed by suppressing the quantization, and negative when the amount of data is increased by emphasizing high frequency components due to too coarse quantization.

Therefore, if ΣPn is positive, it is necessary to reduce the amount of data by coarsening the quantization step, so the quantization step controller 25 generates a positive ΔSm, and conversely Σ
If Pn is negative, a negative △Sm is generated and the adder 19
is supplied to. The adder 19 adds the temporary quantization step S1 supplied from the temporary quantization step corrector 15 and ΔSm supplied from the quantization step controller 25 to obtain the quantization step S. 7 and is outputted via the S output terminal 26. in short,
These quantization step controls operate such that if ΣPn is positive, the quantization step is made coarser to suppress the amount of data, and if ΣPn is negative, the quantization step is made finer to increase the amount of data.

FIG. 4 is a diagram showing quantization step control characteristics. This characteristic is determined from the suppression of high frequency components and the change in data amount due to quantization step shift. In FIG. 4, the horizontal axis represents ΣPn, and the vertical axis represents ΔSm - ΔSm-1. ΔSm is generated in the form of a percentage of S1 and in the form of correcting the value of the previous block slice. This can be expressed as the following equation (5).

[0037]

[Math 6]

In reality, ΣPn is almost never close to 45Pmax or -45Pmax, and ΔS
The correction of m is about 0.1S. The characteristics in FIG. 4 are Pn
Unlike the generation curve of , a simple straight line is sufficient, and ΣPn
The reason why is an accumulation for each slice is that this control is strictly for preventing excessive filtering. Now, with this method of controlling both the suppression or emphasis of high frequency components and the quantization step, it is possible to achieve an appropriate balance between resolution and quantization error in encoding by changing the characteristics shown in Figure 4. can.

Specifically, when the quantization step becomes very coarse, high frequency components are suppressed to some extent,
Even if the resolution is lowered, it may be better to make the quantization step finer. In Fig. 4, if the characteristic is shifted to the lower right as shown by the broken line, △Sm - △Sm-1 is controlled to become 0, so ΣPn settles down to a certain degree of positive value, and high frequency components are suppressed. state. In this way, by slightly changing this characteristic using S1, the balance can be easily adjusted. With the suppression of high frequency components and control using quantization step shift as explained above, the suppression of high frequency components does not generate necessary information on the decoding device side, and the quantization step shift is performed in block slice units, so the control information is is small.

FIG. 2 is a block diagram showing a second embodiment of an encoding apparatus using the encoded output data amount control method according to the present invention. The main difference from FIG. 1 is that the activity detection method and high frequency component control are based on weighting after orthogonal transformation. 1 and 2 are 2 in the prior application 2.
This corresponds to various embodiments. The high-efficiency encoding processing unit 27 first inputs the image input signal to the orthogonal transformer 6,
After the orthogonal transformation, the FB controller 30 is configured to reach the quantizer 7 via the coefficient memory 28 and the variable weighter 29. Correspondingly, the FB controller 30 performs weighting instead of the LPF controller 23 in FIG. A controller 31 is provided. In FIG. 2, the weighting of the variable weighter 29 is controlled to control both the suppression or emphasis of high frequency components and the quantization step shift.
The detailed explanation is omitted since it is described in Prior Application 2, but it has the same effect as FIG. 1.

As explained in detail above, according to the present invention, the temporary quantization step is directly determined from the activity using feedforward control, is corrected later, and the prefilter (such as variable LPF 5 in FIG. 1) is calculated using feedback control.
By suppressing high frequency components on a block-by-block basis and controlling the quantization step on a block-by-several-dozen-block basis, feedforward control can be performed at high speed, resulting in loss of output data, image quality deterioration, and generation of large amounts of control information. The amount of data can be adjusted to the desired value with extremely high accuracy. Furthermore, it is also possible to easily balance resolution and quantization error to an appropriate level.

[0042]

[Effects of the Invention] The encoded output data amount control method of the present invention minimizes loss in data amount by controlling both the quantization step and the suppression and emphasis of high-frequency components, and allocating deviations in data amount. By calculating the appropriate quantization value directly from the overall activity, feedforward processing is speeded up, so there is no problem with image quality, and real-time video processing can be realized with a microprocessor, etc., and the amount of data is reduced. It has extremely excellent practical effects, such as low loss and applicability to storage media.

[0043]

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing a first embodiment of an encoding apparatus using an encoded output data amount control method according to the present invention.

FIG. 2 is a block diagram showing a second embodiment of an encoding apparatus using an encoded output data amount control method according to the present invention.

FIG. 3 is a diagram for explaining block slices.

FIG. 4 is a diagram showing quantization step control characteristics.

[0044]

[Explanation of symbols]

2, 27 High efficiency encoding processing unit 4 Frame memory 5 Variable LPF 7 Quantizer 8 Variable length encoder 10 FF control section 11 Activity detector 12 Activity memory 13 Histogram detector 14 Temporary quantization step generator 15 Temporary quantization step corrector 16 Frame data amount predictor 17, 30 FB control section 18 Block data amount predictor 19 Adder 20 Multiplier 21 Subtractor 22, 24 Accumulative adder 23 LPF controller 25 Quantization step controller 29 Variable weighter 31 Weighting controller

Claims (3)

[Claims] Claim 1: When the activity of an image is determined for each block and the amount of encoded output data is controlled in a feedback type, the activity of each block is summed and a provisional quantization step is performed from that value. and a correction means for calculating the predicted data amount of each block from the temporary quantization step and each activity, and correcting the temporary quantization step based on the sum and the target total data amount. What is claimed is: 1. An encoded output data amount control method comprising: [Claim 2] Predicting the amount of encoded output data for each block of an image and performing feedback control based on the deviation between the predicted value and the actual value to bring the data amount closer to the target value. It is configured with a predicted data amount correction means that corrects the predicted data amount of each block by multiplying the predicted data amount of each block by the value of the ratio obtained by dividing the target total data amount by the total data amount. An encoded output data amount control method characterized by: 3. High-frequency component changing means for changing the degree of suppression or emphasis of high-frequency components on a block-by-block basis in order to control the amount of output data of image encoding; 1. A method for controlling the amount of encoded output data, comprising a quantization step width changing means for adding the sum and changing the quantization step width in units of a plurality of blocks based on the result.