JPH07107482A - Encoding control system - Google Patents

Abstract

PURPOSE:To provide a highly efficient encoding control system capable of properly keeping the reproduction of the movement, the space resolution and the noise balance in the visual characteristic point of view with a simple control. CONSTITUTION:When an encoding rate 105 as the result of encoding of the 1st quantization accuracy 104A is higher than the upper limit value 102 of the optimal encoding rate in the visual characteristic corresponding to the 1st quantization accuracy, the next encoding is performed by a 2nd quantization accuracy 104B which is more accurate than the 1st one. When it is lower than a lower limit value 103, the encoding is performed by a 3rd quantization accuracy 104C which is less accurate than the 1st one and when it is in the area, the encoding is performed by the 1st quantization accuracy. The quantization accuracy of the next encoding is decided according to high or low between the encoding rate of the encoding result which is encoded by the quantization accuracy and the area of the encoding rate most suitable for the visual characteristic according to the quantization accuracy.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding / decoding device used in video conferences, video telephones and the like.

[0002]

2. Description of the Related Art Conventionally, CCIT has been used as an image compression encoding system for performing moving image communication at a low rate of 64 Kbps.
Recommendation H.T. There is 261. In FIG. 261 is a block diagram of an image encoding method recommended by H.261. In FIG. 5, 504 is a pre-processing unit that separates the NTSC signal into YC and A
A / D conversion unit 512 for performing D / D conversion and a CIF signal (Common I
NT to convert to
It is composed of an SC / CIF conversion unit 513 and a pre-processing filter 514. The intermediate format is a common format that resolves differences in television systems between regions and allows all codecs to communicate without being aware of the other party. Reference numeral 505 denotes an encoding unit, which includes an encoding unit 503 and an encoding control unit 501 that controls the encoding unit.
The encoding unit 503 is capable of motion compensation in the range of 16 × 16 pixels and calculates the error between frames. The motion-compensated inter-frame prediction unit 506 and the prediction error signal are orthogonally transformed in 8 × 8 block units to generate spatial coordinate data. To a frequency coordinate data, a quantization unit 508 to linearly quantize the orthogonally transformed transform coefficient, a first variable length coding unit 509 to Huffman-encode the quantized transform coefficient, and motion compensation. A second variable length coding unit 510 for Huffman coding the used motion vector, main information coded by the first variable length coding unit, and side information coded by the second variable length coding unit. 511 that multiplexes the data to form a transmission frame
Composed of. Reference numeral 502 is a smoothing buffer and 515 is a transmission path.

In order to obtain visually good image quality, not only high coding efficiency but also the coding control characteristic of the coding control unit 501 which determines the coding parameter for the input image is adapted to the human visual characteristic. Need to be In order to compress and code a moving image signal up to several tens of Kbps, some spatial and temporal distortion must be tolerated. Since the amount of information that a moving image signal has varies greatly with time, it is necessary to encode the moving image so that it fits a transmission line with a constant speed, and to maintain good reproduced image quality, The coding control algorithm that controls the parameters is an important element. That is, it is desirable to use a coding control method that can keep the balance of motion reproducibility, spatial resolution, and noise optimal in terms of visual characteristics within a limited amount of information. Here, the spatial resolution is determined by the quantization precision of the quantization unit 508, and the higher the quantization precision, the more the generated code amount R (t) when the input image is encoded at time t. The generated information is stored in the smoothing buffer 502. Since an L-bit code per unit time is sent to the transmission line 615 from the smoothing buffer, assuming that the staying amount of the smoothing buffer is B (t), when the image at time t is coded, B (t + 1). ) =
B (t) + R (t) -L, and when the input image at time t is dropped, B (t + 1) = B (t) -L. Here, whether the input image of t + 1 is encoded or dropped is determined to be dropped when B (t)> L and to be encoded when B (t) is L or less. Further, since it takes R (t) / L unit time to transmit the code amount of R (t), the input frame coding rate S (t) before and after the time t is S.
It can be expressed as (t) = L / R (t).

When the input image is encoded, the decoded image changes depending on the quantization precision. If the quantization precision is increased, the coding rate is decreased, and if the coding rate is increased, the quantization precision is decreased and S /
You have to lower the N ratio. This relationship is called the image quality tradeoff function. An example of the image quality trade-off function is shown in FIG. The image quality trade-off function moves to the lower right when the input frame includes large movements or fine patterns, and moves to the upper left when there is little movement. On the other hand, there is one point on the image quality trade-off function for each input image at which the pair of the coding rate and the quantization accuracy described above is visually optimum. The line connecting these points is the objective function, which is also shown in FIG.

In order to accurately obtain the image quality trade-off function, it is necessary to actually encode the frame many times while changing the quantization accuracy and measure the coding rate at that time. However, such processing is unrealistic in terms of processing time. Kato et al., "Spatio-Temporal Distortion Optimal Allocation Coding Control Algorithm in Video Coding Systems," IEICE Transactions B Vol. J71-B No. 8
pp 945-954 August 1988 proposes two methods for determining the image quality tradeoff function from the current coded frame. The first method obtains the prediction error signal DCT coefficient and motion vector of the current input frame at the stage where motion compensation frame prediction and DCT are completed. Here, the image quality trade-off function is accurately calculated from the DCT coefficient histogram for one frame and the motion vector code amount. The second method is to predefine some image quality trade-off function candidates, select a characteristic that matches the encoding result of the previous encoded frame, and select this characteristic from the current encoded frame. Image quality trade-off function.

[0006]

According to the first conventional method, the DCT coefficient histogram of the current input frame is obtained, and then the code amount of the block type and the quantum of the DCT coefficient for each quantization accuracy are obtained based on the histogram. Since the code amount for the indexization is calculated, an enormous amount of calculation is required, which poses a problem in terms of performance, price, space, and power consumption of the processor. Further, for the same reason, a processing delay of at least 1 unit time occurs, so that there is a drawback that it is difficult for both parties to communicate with each other in a videophone or a video conference that requires real-time transmission. In addition, although the processing amount is reduced in the second conventional method, it is necessary to store the information of the candidate of the predetermined image quality trade-off function, Since the image quality trade-off function that matches the coding result of the coded frame is used, it cannot follow changes in the input image. This is a big problem as a moving picture coding control method.

In the conventional frame dropping control, when the generated code amount exceeds the free space of the smoothing buffer, overflow occurs and one frame of an image cannot be sent accurately, so that the image quality is not significantly deteriorated. For the purpose, the retention amount B (t) of the smoothing buffer is the code amount L transmitted per unit time.
Drop pieces until: In actual encoding, since the inter-frame prediction cannot be performed immediately after the scene change, the compression rate does not increase and the generated code amount is large, but since the inter-frame prediction can be performed from the next frame, the compression rate increases and the generated code amount increases. Is halved. However, in the conventional frame dropping control, since the generated code amount of the first frame is large, frame dropping for several frames is performed until the staying amount of the smoothing buffer decreases, so that the image to be coded next is encoded before. The correlation with the converted image is weakened, the compression ratio does not increase even if prediction is performed between frames, and a large amount of code is generated.
The coding rate remains low and does not recover from this vicious cycle.

The present invention has been made in view of the above circumstances, and it is possible to perform motion reproducibility and spatial resolution with simple control.
It is an object of the present invention to provide a highly efficient coding control method in which the balance of noise is kept optimal in terms of visual characteristics.

[0009]

The invention according to claim 1 is the first
If the first coding rate as a result of coding with the above-mentioned quantization accuracy is larger than the coding rate of the optimum operating point in view of the visual characteristics corresponding to the first quantization accuracy, the first coding rate The next encoding that has performed frame dropping according to is performed with the second quantization precision that is more accurate than the first quantization precision, and the first encoding result obtained by encoding with the first quantization precision is performed. When the coding rate is smaller than the coding rate of the optimal operating point in view of the visual characteristics corresponding to the first quantization accuracy, the next coding in which the frame dropping according to the first coding rate is performed is performed. Visual characteristics according to the coding rate of the coding result obtained by performing coding with the third quantization accuracy, which is worse than the first quantization accuracy, and then with the certain quantization accuracy, and the quantization accuracy It is characterized in that the quantization accuracy of the next coding is determined according to the magnitude relationship with the coding rate of the optimum point. There.

According to a second aspect of the present invention, the optimum coding rate in view of the visual characteristics corresponding to each of the quantization precisions described in the first aspect is not a set of points but an area having an upper limit value and a lower limit value, respectively.
If the first coding rate as a result of coding with the above-mentioned quantization accuracy is larger than the upper limit value of the coding rate optimal for the visual characteristics corresponding to the first quantization accuracy, the first coding rate The next encoding that has performed frame dropping according to is performed with the second quantization precision that is more accurate than the first quantization precision, and the first encoding result obtained by encoding with the first quantization precision is performed. When the coding rate is smaller than the lower limit of the optimal coding rate in view of the visual characteristics corresponding to the first quantization accuracy, the next coding in which the frame dropping according to the first coding rate is performed is performed. Encoding is performed with a third quantization precision, which is less accurate than the first quantization precision, and the first encoding rate of the result of encoding with the first quantization precision is the first
When the image is in the region of the optimum coding rate in view of the visual characteristics corresponding to the quantization accuracy of, the next coding in which the frame dropping according to the first coding rate is performed is the first quantization accuracy. Coding is performed in the same way, and the following is performed according to the size relationship between the coding rate of the coding result coded with a certain quantization accuracy and the area of the optimum coding rate in terms of visual characteristics according to the quantization accuracy. It is characterized by determining the quantization accuracy of encoding.

According to a third aspect of the present invention, the difference between the first quantization precision and the second quantization precision or the difference between the first quantization precision and the third quantization precision according to the second aspect is spatially determined. The feature is that the difference in strain is larger than a visually recognizable value.

According to a fourth aspect of the present invention, when the amount of information generated by the coding of the image frame is smaller than the free space of the smoothing buffer having the capacity determined by the allowable maximum value of the coding delay time, this image frame is framed. It is characterized in that the data is stored in the buffer without dropping, and if the amount of information is larger than the free space of the smoothing buffer, the frame is dropped.

According to a fifth aspect of the present invention, the amount of information generated by the encoding of the image frame according to the fourth aspect is approximated from information obtained as a result of encoding the previous image frame or information halfway through the encoding of the current frame. It is characterized by seeking.

The invention of claim 6 is characterized in that the amount of information generated by the encoding of the image frame of claim 5 is set as the amount of information of the frame previously encoded.

The invention of claim 7 is characterized in that the amount of information generated by the encoding of the image frame of claim 5 is obtained from the number of invalid blocks of the current encoded frame.

The invention of claim 8 is characterized in that the coding control system of claim 5 or claim 6 or claim 7 is applied to the frame dropping control method of claim 2 or claim 3.

[0017]

In the invention of claim 1, the image quality trade-off function is a function that monotonically increases to the right, and when the operating point is on the right of the visually optimum operating point, the coding rate is the optimum operating point. The coding rate is larger than the coding rate of, and the coding rate when there is an operating point to the left of the visually optimum operating point is smaller than the coding rate of the optimal operating point. In order to visually move to the optimum operating point, it is possible to judge whether the quantization precision should be increased or decreased from the current operating point.

According to the second aspect of the present invention, by making the coding rate visually optimal for each quantization precision to have a range, stable coding control can be realized even when the input image changes rapidly.

According to the third aspect of the invention, the change in the quantization precision given per unit time becomes large, and the change in the visually optimum operating point due to the change in the input image can be quickly followed.

According to the fourth aspect of the present invention, by encoding the frame in which the correlation between the frames is maintained, the inter-frame prediction efficiency can be improved and the frame drop control with a high coding rate can be realized.

According to the fifth aspect of the present invention, it is possible to judge with a small processing amount whether or not the generated code amount when the image frame is coded is smaller than the free space of the smoothing buffer and there is no need to drop frames.

In the sixth aspect of the present invention, when there is a correlation between the image frames, the generated code amount can be estimated without coding the current image frame.

According to the invention of claim 7, in addition to the invention of claim 6, the information of the number of invalid blocks of the current image is added to improve the estimation accuracy of the generated code amount.

In the invention of claim 8, since the frame dropping control is performed in the operation region for each quantization precision of the invention of claim 2 or 3, the claim 5 or 6 is adopted.
Alternatively, by applying the encoding control method according to the seventh aspect, it is possible to provide a highly efficient encoding control method in which the balance of motion reproducibility, spatial resolution, and noise is optimally maintained in terms of visual characteristics.

[0025]

Embodiment 1 Embodiment 1 of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram for explaining the principle of the coding control system according to the embodiment of the present invention. In FIG. 1, 101 is an image quality trade-off function, which changes according to the size of the temporal motion of the input image or the spatial frequency component, and moves to the lower right when a large motion or a fine pattern is included, When there are few, move to the upper left. The detailed description of the image quality trade-off function and the purpose of the present invention have been described in detail in the prior art, and therefore will not be repeated here. Reference numeral 102 is an upper limit value of the visually optimum coding rate, and 103 is a lower limit value of the visually optimum coding rate. In the conventional technology, the objective function is assumed to be a set of points at which the pair of the coding rate 105 and the quantization precision 104 is visually optimum, but this is actually a value that is subjectively obtained, It varies depending on the viewing distance, angle, and personal preference. Therefore, a visually optimum area is assumed on the image quality trade-off function, and a point in which the maximum coding rate is collected is the upper limit value 102 of the visually optimum coding rate. The lower limit value 103 of the visually optimal coding rate is a collection of points with the minimum coding rate. In other words, the purpose here is to place the operating point in a region between the upper limit value of the visually optimum coding rate and the lower limit value of the visually optimum coding rate.

FIG. 2 shows an algorithm of the coding control system according to the embodiment of the present invention. The operation of the coding control method of the present invention will be described below with reference to FIGS. 1 and 2. First, encoding is performed with the first quantization precision 104A. In FIG. 1, the image quality trade-off function of the input image is 101A or 10A.
If it is 1B, the operating point A or the operating point B is within the visually optimum region, the coding rate does not exceed the upper limit of the visually optimum coding rate, and the visually optimum code Since it does not fall below the lower limit of the conversion rate, the next encoding is performed after performing frame dropping according to the generated code amount while keeping the quantization accuracy unchanged at 104A according to the flow of FIG. If the image quality trade-off function of the input image becomes 101C while repeating this, the operating point becomes the transient operating point C. The coding rate of the operating point C is the quantization precision 104.
Since the upper limit of the visually optimum coding rate for A is exceeded, the quantization accuracy is improved to 104B. After that, after dropping frames according to the generated code amount, encoding is performed with the quantization precision 104C. Assuming that the trade-off function of the input image remains unchanged at 101C, the next operating point becomes the operating point D, which again becomes the visually optimum region. Similarly, when the image quality trade-off function of the input image is 101D when operating at the operating point B, the operating point becomes the transient operating point E. Since the coding rate at this time is below the lower limit of the visually optimum coding rate, the quantization accuracy is set to 104
Lower to C. In encoding after dropping frames according to the generated code amount, the operating point becomes a visually optimum operating point F.

As described above, according to the present embodiment, it is possible to provide a coding control method in which the reproducibility of motion, the spatial resolution, and the balance of noise can be optimally maintained in terms of visual characteristics with simple control. Becomes What is more, the more the quantization accuracy is improved and the smaller the degree of deterioration, the more visually narrowed the optimal operation area can be narrowed down, but the tracking speed for changes in the input image becomes slower,
The control diverges when it cannot follow the rate of change of the input image. On the other hand, when the quantization accuracy is improved or the width of decrease is increased, the tracking speed with respect to a change in the input image is increased, but the operation area is increased and the control accuracy is deteriorated. The control accuracy is set to be near the cognitive limit of visual characteristics.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to the drawings.

FIG. 3 is an explanatory view of the principle of the coding control system according to the embodiment of the present invention, and FIG. 4 shows an algorithm of the coding control system according to the embodiment of the present invention. FIG. 3 also shows the conventional frame dropping control method shown in FIG. In FIG. 3, at time t0, the code amount of R (t0) is generated in the first frame in both the conventional system of FIG. 3 (a) and this system of FIG. 3 (b), and both are written in the smoothing buffer. Time t1
, The amount of staying of the smoothing buffer is R (t0) −L>
L, and in the conventional method, the frame drop of the second frame is determined. On the other hand, in the present invention, the generated code amount R (t1) when the second frame is encoded is estimated, and since this is smaller than the free space of the smoothing buffer, it is encoded according to the algorithm of FIG. 4 and written in the smoothing buffer. At time t2, the staying amount of the conventional smoothing buffer is R (t0) −
2L <L, the coding of the third frame is determined, and the first
Encode the inter-frame prediction error with the frame. In the present invention, the generated code amount R ′ (t
2) is estimated, and since this is smaller than the free space of the smoothing buffer, the inter-frame prediction error with the second frame is encoded according to the algorithm of FIG. 4 and written in the smoothing buffer. At time t3, the smoothing buffer retention amount of the conventional method is R (t0) -L> L, and the conventional method determines the dropping of the fourth frame. On the other hand, the present invention also estimates the generated code amount R (t1) when the fourth frame is encoded, and since this is larger than the free space of the smoothing buffer, the fourth frame is dropped according to the algorithm of FIG. What is important here is that the encoding of the third frame in the conventional method is performed on the inter-frame difference from the first frame,
In the present invention, since the inter-frame difference with the second frame having a stronger correlation is encoded, the compression efficiency of the encoding of the third frame is improved as compared with the conventional method.
That is, with respect to the generated code amount R (t2) of the third frame encoded by the conventional method, the third code encoded by the method of the present invention is used.
The generated code amount R ′ (t2) of the frame becomes small.

As described above, according to the present embodiment, it is possible to perform efficient frame drop control utilizing the correlation between frames. However, as a method of estimating the generated code amount when the image frame is encoded by the method of the present invention, when the generated code amount of the previously encoded frame is used as the estimated value as it is, the correlation between the image frames is strong. Does not cause much error. However, it cannot be applied to images in which the scene changes greatly.
Japanese Patent Application H5-189986 solves this drawback by calculating the number of invalid blocks in the current frame and utilizing the fact that there is a correlation between the number of invalid blocks and the amount of generated information. Further, by applying this method to the frame dropping control of the invention of the first embodiment, a highly efficient code in which the reproducibility of movement, the spatial resolution, and the balance of noise can be optimally maintained in terms of visual characteristics with simple control. It is possible to provide a computerized control method.

[0032]

As described above, according to the present invention, there is provided a highly efficient coding control system in which the balance of motion reproducibility, spatial resolution, and noise can be optimally maintained by simple control in terms of visual characteristics. Can be provided.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of an operation principle in Embodiment 1 of the present invention.

FIG. 2 is a diagram showing a control algorithm in the first embodiment of the present invention.

FIG. 3 is an explanatory diagram of an operation principle in the second embodiment of the present invention.

FIG. 4 is a diagram showing a control algorithm in Example 2 of the present invention.

FIG. 5 is a configuration diagram of an image encoding device according to a conventional technique.

FIG. 6 is an explanatory diagram of the principle of encoding control in the conventional technique.

[Explanation of symbols]

 101 Image quality trade-off function 102 Visually optimal upper limit of coding rate 103 Visually optimal lower limit of coding rate 104 Quantization accuracy 105 Coding rate

Claims (8)

[Claims] 1. When a moving image is compression-encoded with high efficiency and transmitted through a communication line of low bit rate, temporal distortion due to frame dropping corresponding to the encoding rate and spatial distortion due to quantization accuracy are generated. In an image coding apparatus in which an operating point having an optimal balance in terms of visual characteristics is predetermined, a first coding rate as a result of coding with a first quantization accuracy corresponds to the first quantization accuracy. When the coding rate is higher than the optimal operating point in view of the visual characteristics, the next coding in which the frame dropping according to the first coding rate is performed is more accurate than the first quantization accuracy is the second quantization. The encoding rate of the optimum operating point in terms of the visual characteristics corresponding to the first quantization accuracy is the first encoding rate as a result of encoding with the encoding accuracy and encoding with the first quantization accuracy. If the code is smaller, the next code subjected to frame dropping according to the first coding rate Encodes with a third quantization precision, which is lower in precision than the first quantization precision, and similarly, the encoding rate of the encoding result encoded with a certain quantization precision and the visual corresponding to the quantization precision. By determining the quantization accuracy of the next coding according to the magnitude relationship with the coding rate of the point that is optimal in terms of characteristics, the coding parameters for the changing input image are set to a balance between motion reproducibility and spatial resolution. Is a coding control method that controls so that the optimal visual characteristics are maintained. 2. When a moving image is compression-encoded with high efficiency and transmitted through a communication line with a low bit rate, temporal distortion due to dropping frames and spatial distortion due to quantization accuracy are generated according to the encoding rate. In an image coding apparatus in which an operating point whose balance is optimal in terms of visual characteristics is predetermined, the optimal coding rate in terms of visual characteristics corresponding to each quantization accuracy has an upper limit value and a lower limit value instead of a set of points. If the first coding rate as a result of coding with the first quantization accuracy is larger than the upper limit value of the coding rate optimal for the visual characteristics corresponding to the first quantization accuracy, The next encoding, in which frames are dropped according to the first encoding rate, is encoded with the second quantization accuracy that is higher than the first quantization accuracy, and is encoded with the first quantization accuracy. The first coding rate of the result corresponds to the first quantization accuracy. When the coding rate is smaller than the lower limit value of the optimal coding rate in view of visual characteristics, the next coding in which the frame dropping according to the first coding rate is performed is lower in accuracy than the first quantization accuracy. Encoding is performed with encoding accuracy, and the first encoding rate as a result of encoding with first quantization accuracy is within the area of the optimal encoding rate in view of the visual characteristics corresponding to the first quantization accuracy. In a certain case, the next encoding after performing frame dropping according to the first encoding rate is performed with the first quantization accuracy, and similarly, the encoding result of the encoding result obtained with certain quantization accuracy is obtained. The coding parameter for the input image that changes by determining the quantization accuracy of the next coding according to the magnitude relationship between the coding rate and the region of the optimal coding rate in terms of visual characteristics according to the quantization accuracy Controlled so that the balance between motion reproducibility and spatial resolution is optimally maintained in terms of visual characteristics. Coding control scheme. 3. A difference in spatial distortion can be visually recognized as a difference between the first quantization accuracy and the second quantization accuracy or a difference between the first quantization accuracy and the third quantization accuracy. The encoding control method according to claim 2, wherein the size is equal to or larger than a value. 4. When the amount of information generated by encoding an image frame is smaller than the free space of a smoothing buffer having a capacity determined by an allowable maximum value of the encoding delay time, this image frame is buffered without dropping frames. 3. The coding control method according to claim 1, wherein the frame is dropped when the amount of information is larger than the free space of the smoothing buffer. 5. The amount of information generated by encoding an image frame is approximately obtained from information obtained as a result of encoding a previous image frame or information halfway through the encoding of the current frame. The encoding control method described. 6. The coding control method according to claim 5, wherein the information amount generated by the coding of the image frame is set as the information amount of the immediately preceding frame. 7. The encoding control method according to claim 5, wherein the amount of information generated by encoding the image frame is obtained from the number of invalid blocks in the current encoded frame. 8. The method according to any one of claims 5 to 7 is applied to frame drop control.
The encoding control method described.