JP3152148B2 - Image coding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus for realizing encoding that maintains smooth motion in video transmission at a low transmission rate, such as a video conference or video telephone.

[0002]

2. Description of the Related Art In a device for transmitting an image at a relatively low rate, such as a video conference, the amount of codes stored in a transmission buffer is monitored in order to prevent overflow of a transmission buffer. Is a process of stopping image coding. When the encoding of the image is stopped on a frame basis, this is referred to as frame dropping, and the number of frames for which encoding is continuously stopped is referred to as the frame dropping frame number. In moving image coding that allows dropping of frames at a fixed transmission rate, the S / N ratio of a coded frame decreases as the frame rate increases, and the S / N ratio of the coded frame improves as the frame rate decreases. Japanese Patent Application Laid-Open No. 63-116581 discloses a technique that utilizes this relationship to suppress image quality degradation when the image moves rapidly.

FIG. 6 shows a configuration of an image coding apparatus according to the above technique. An original image signal generated at about 30 frames / second per second is input to a moving image input means 60 through an input terminal 60.
1 is supplied. The moving image input means 601 determines whether the currently input frame is to be encoded or to be dropped based on the output value of the dropped frame control circuit 605, and stores only the signal of the coded frame in the frame memory 602 and the statistic It is supplied to the calculation circuit 606. In the frame memory 602, the input signal is delayed by one frame time and supplied to the encoding circuit 603. The encoding circuit 603 quantizes the frame signal supplied from the frame memory 602 using the quantization characteristic output from the quantization characteristic control circuit 609, and further converts the signal into a highly efficient code (for example, a Huffman code). , Buffer memory 604. The buffer memory 604 stores the codes supplied from the coding circuit 603 and outputs the codes to the output terminal 69 at a specified rate. The dropped frame control circuit 605 receives the accumulated amount of codes remaining in the buffer memory 604, calculates the number of dropped frames based on the accumulated amount, and uses the value of the dropped frame number as a moving image input means 601. Output to

On the other hand, a statistic calculation circuit 606 calculates a statistic representing the feature of the coded frame signal supplied from the moving image input means 601 and supplies the statistic to the selector 607. In the correspondence table storage memory 608, a correspondence table indicating the relationship between the statistic and the optimum number of dropped frames is set in advance. The selector 607 includes a correspondence table storage memory 60.
8 is read out from the correspondence table stored in 8 and is supplied to the quantization characteristic control circuit 609. Quantization characteristic control circuit 609
Selects a quantization step width on the basis of the value of the optimal frame drop frame number. For example, an operation is performed to select a larger quantization step width as the value of the number of dropped frames is smaller. The quantization step width selected by the quantization characteristic control circuit 609 is supplied to the encoding circuit 603 and used for encoding a frame.

[0005]

However, in the above configuration, the encoding of the frame to be encoded is started after the statistics of the frame to be encoded are calculated. Therefore, the encoding delay is calculated by the time required for calculating the statistics. And hinders image transmission in real time such as a video conference. Also, since the value of the optimal frame removal frame number used for controlling the quantization characteristic is set in the correspondence table memory 708 in advance, when the image property changes greatly with time, the optimal frame removal frame number also increases rapidly. And the smoothness of the image is impaired.

[0006]

In order to solve the above-mentioned problems, the present invention gradually increases the frame rate until the image quality of the coded image reaches a predetermined threshold region regardless of the nature of the pattern of the input image. And the amount of change in the frame rate is set to 0 when the threshold area is reached, so that the frame rate is automatically converged to the optimum frame rate according to the image quality. By keeping it, the smoothness of the movement can be maintained.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS According to the first aspect of the present invention, an encoding means for dividing an input frame of an image into blocks in advance and performing encoding for each block, and an amount of data stored in a buffer memory. Frame removal control means for determining the number of frame removal frames; image quality parameter value calculation means for calculating image quality parameter values from information of already encoded frames; and target frame when the image quality parameter value is greater than a preset threshold T1. The number of dropped frames is increased, and when the image quality parameter value is smaller than a predetermined threshold T2 (where T2 <T1 ), the target frame dropped frame number is reduced, and the image quality parameter value is smaller than a predetermined threshold T1. When it is larger than T2, control is performed so that the number of target dropped frames is not changed from the immediately preceding value. Means for calculating the number of target dropped frames, an allocated code amount calculating means for calculating the allocated code amount per one frame time of the input image, the target dropped frame number and the allocated code per one frame time. A target code amount calculating unit that calculates a target code amount of the frame using the amount and a quantization characteristic determining unit that determines a quantization characteristic for each of the blocks according to the target code amount.

According to the present invention, it is possible to control the image quality without increasing the encoding delay by calculating the image quality parameter value using the information of the already encoded frame. Further, in the present invention, when the image quality parameter value of a certain encoded frame falls within a target area sandwiched between two preset threshold values, the frame rate at that time is determined to be the optimum frame rate. This allows the actual frame rate to automatically converge to the optimum frame rate according to the picture of the input image without previously setting the optimum value of the frame rate according to the image quality.

According to the present invention, when the image quality parameter value of an already encoded frame is outside the target area, the image quality parameter value of the next encoded frame converges within the target area. The number of target dropped frames is changed by n. In the next frame to be coded, the quantization step width is determined according to the target frame-dropping frame number, so that the actual frame-dropping frame number is the same as the target frame-dropping frame number. Further, by setting the absolute value of n to be small, the time change of the target dropped frame can be made gradual, so that even if the picture changes abruptly, the change of the frame rate becomes gradual, and the movement of the reproduced image can be reduced. Smoothness can be maintained.

According to a second aspect of the present invention, there is provided an encoding means for dividing an input frame of an image into blocks in advance and performing encoding for each block, and the number of frames to be dropped according to the storage amount of a buffer memory. , A picture quality parameter value calculating means for calculating a picture quality parameter value from information of an already coded frame, and a threshold value T for which the picture quality parameter value is set in advance.
When the value is greater than 1, the target frame removal frame number is made larger than the immediately preceding frame removal frame number. When the image quality parameter value is smaller than a predetermined threshold T2 ( T2 <T1 ), the target frame removal frame number is set to the immediately preceding frame removal frame. Target frame dropped frame number calculating means for controlling the target frame dropped frame number so as not to be changed from the immediately preceding frame dropped frame number when the image quality parameter value is smaller than the predetermined threshold value T1 and larger than the threshold value T2. Allocation code amount calculation means for calculating an allocation code amount per one frame time, target code amount calculation means for calculating a target code amount of a frame using the target frame dropped frame number and the allocation code amount, Quantization characteristic determining means for determining a quantization characteristic for each block according to the target code amount And

According to the present invention, the number of target dropped frames is changed within a certain range with reference to the actual dropped frame number. Even when the number of frames is different, it is possible to control so that the local change in the frame rate is always minimized.

Hereinafter, an embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. (Embodiment 1) FIG. 1 is a diagram showing a configuration of an image coding apparatus according to Embodiment 1 of the present invention.

Reference numeral 11 denotes an input terminal to which a digital moving image is input at a constant frame rate; 101, a moving image input means for dividing only information of a frame to be encoded into a plurality of blocks; An encoding circuit for encoding (for example, Huffman code or the like) the information supplied from 101 in block units;
A buffer memory 10 for temporarily storing the codes output from 2 and outputting the codes to the transmission line 19 at a preset speed.
Reference numeral 4 denotes a frame removal control circuit for determining the number of frame removal frames based on the code amount stored in the buffer memory 103;
05 integrates the code amount (generated code amount) output from the encoding circuit 102 in block units or frame units,
A monitoring circuit that supplies the integrated value to a quantization characteristic control circuit 142 and an allocated code amount calculation circuit 130, which will be described later;
Is a code amount calculation circuit for calculating the code amount to be allocated to one frame time from the history of generated code amounts in the past; 142, a quantization characteristic control circuit for determining a quantization step width and supplying it to the coding circuit 102; An image quality parameter calculation circuit for calculating an average value of the quantization step width, a setting circuit 115 for setting an arbitrary threshold value in advance, and a setting circuit 113 for setting
Threshold value set at 15 and image quality parameter calculation circuit 110
The comparator 117 compares the image quality parameters calculated in the step (1) and outputs a signal according to the result. 117 is an adder for calculating the number of target dropped frames. 119 is a storage device for temporarily storing the number of target dropped frames. , 118 are upper and lower limit value setting circuits for limiting the upper limit value and the lower limit value of the target frame drop frame number, and 141 is a multiplier for calculating the target code amount of the frame.

Next, the configuration of the encoding circuit 102 will be described with reference to FIG. In FIGS. 1 and 2, the same components are denoted by the same reference numerals. In FIG. 2, 21 is an input terminal 1
A subtraction circuit for generating a prediction error signal from the signal supplied from the frame memory 28 and the signal supplied from the frame memory 28;
Is a converter for performing discrete cosine transform of the input signal, 23
Is a quantizer for quantizing the discrete cosine-transformed signal based on a quantization step width supplied via a terminal 14, and 24 is a code conversion for converting the quantized signal to a highly efficient code. 25, an inverse quantizer for inversely quantizing the input signal, 26, a converter for performing an inverse discrete cosine transform of the input signal, and 27, an output signal from the converter 26 and a signal stored in the frame memory 28. An adder 28 that adds and generates a locally decoded image is a frame memory that stores the locally decoded image.

Next, the operation of the image coding apparatus shown in FIGS. 1 and 2 will be specifically described with reference to the flowchart of FIG.

A digital moving image signal input to the input terminal 11 at a constant frame rate is supplied to the moving image input means 101 on a frame basis (S1). The moving image input means 101 calculates the sum (j + m) of the number i of the currently input frame, the number j of the most recently encoded frame, and the frame drop interval m output from the frame drop control circuit 104. (S2), and if i = (j + m), divide the input frame information into a plurality of blocks and supply it to the encoding circuit 102 (S3); if i ≠ (j + m), If so, the process returns to the input of the next frame image. The encoding circuit 102 inputs the image information of the k-th block in the frame, performs a discrete cosine transform (S5), and supplies the coefficients obtained by the discrete cosine transform from the quantization characteristic control circuit 142. The quantization is performed using the quantization step width Qk for the k-th block (S6). The quantization step width Qk used here is supplied to and stored in the image quality parameter calculation circuit 110 (S7). Note that S6 and S7 can be reversed.

The quantized coefficient value is supplied to a code converter 44, where it is converted into a highly efficient code, for example, a Huffman code (S8), and stored in the buffer memory 103 via the output line 13 (S8). S9). The monitoring circuit 105 integrates the generated code amount output to the output line 13 in block units and supplies the integrated code amount to the assigned code amount calculation circuit 130 and to the quantization characteristic control circuit 142 (S10). The quantization characteristic control circuit 142 calculates a quantization step width for a block to be encoded based on the input generated code amount (S11). At this time, if it is determined that the next block to be encoded is a block of the current frame, the next block is encoded, and the next block to be encoded is the first block of the next frame to be encoded. If it is determined that the coding of the current coded frame has been completed, the process proceeds to a process of calculating the target code amount of the next coded frame (S12).

The allocated code amount calculating circuit 130 is a monitoring circuit 1
Based on the generated code amount information supplied from 05, the sum of the generated code amounts of the currently coded frames is calculated and stored in its own memory (S13, S14). In the allocated code amount calculation circuit 130, the generated code amounts of a plurality of previously encoded frames are stored in a memory, and the generated code amounts of the currently coded frames and the plurality of previously coded frames are stored. From the generated code amount of the frame, an average allocated code amount A per one frame time is calculated based on Equation 1 (S15),
The signal is supplied to the multiplier 141.

A = ((M + N) × R−M × I) / N (1) In Equation 1, M is the number of image frames input in the past, and N
Is an arbitrary integer of 1 or more, R is a value obtained by dividing a transmission rate by a frame rate of an input image, and I is a generated code amount of a frame actually encoded in an input image of an M frame input in the past. The value obtained by dividing the sum by M. On the other hand, the image quality parameter calculation circuit 110 calculates the average value Qg of the quantization step width of the currently coded frame (S16), and supplies this to the comparator 113. The threshold values T1 and T2 (T1> T2) are set in advance for the set value 115, and are supplied to the comparator 113.

The comparator 113 compares the average value Qg of the quantization step width of the current frame with the threshold values T1 and T2. If Qg> T1, the change amount Δb of the number of target dropped frames is Δb. = 1 (S17, S18), and T1
If>Qg> T2, it is set so that the next target dropped frame number and the immediately preceding change amount △ b = 0 (S1).
9, S20), and when T2> Qg, the change amount of the next target dropped frame number is set so that △ b = −1 (S21, S22), and the set value △ b is added. To the container 117. The adder 117 calculates the number b of the target dropped frames immediately before supplied from the storage 119 and
Is calculated by adding the values of (1) to (2), and the number of frames to be dropped next is calculated (S23). In the upper / lower limit setting circuit 118, the input target frame drop frame number and a preset target frame drop frame number (eg, upper limit 1
5 frames and a lower limit of 2 frames), and if the input target frame drop frame number is larger than the upper limit value, the target frame drop frame number is changed to the upper limit value and the input target frame drop frame number is changed. Is smaller than the lower limit, the target number of dropped frames is changed to the lower limit, the target dropped frame number having a value within the range of the upper and lower limits is supplied to the multiplier 141, and the multiplier 141 The target generated code amount of the next coded frame is calculated from the product of the number of dropped frames and the average allocated code amount A per one frame time, and is supplied to the quantization characteristic control circuit 142 (S24). When the supply is completed, the process moves to the next frame (S25).

The quantization characteristic control circuit 142 converts the frame target code amount T into a target code amount for each block, and the generated code amount for each block supplied from the monitoring circuit 105 is larger than the target code amount for each block. The quantization step width is determined while controlling so as to increase the quantization step width as the size increases, and similarly decreasing the quantization step width as the generated code amount per block is smaller than the block target code amount (S11). .

With the above-described configuration, in the first embodiment, the amount of change in the number of target dropped frames is within ± 1 each time one frame is encoded. By setting the width,
The difference between the actual number of dropped frames and the target dropped frame number is set so as to be small, thereby realizing stable coding with a very small amount of change in the coding frame rate in the short term, In addition, in the long term, the average value of the image quality (SN ratio) is stabilized at a certain level.

In the first embodiment, when the number of target dropped frames is determined, the number of target dropped frames is changed in a direction such that the image quality determined by the image quality parameter is converged between the thresholds T1 and T2. As a result, the frame rate automatically converges to a frame rate value that achieves the image quality level.

In the above description, the amount of change in the number of target dropped frames is ± 1, but may be any other integer.

(Embodiment 2) FIG. 4 is a view showing a second embodiment of the present invention. Portions that operate in the same manner as in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

The target frame dropped frame number calculation circuit 408 includes a comparator 113, a set value circuit 115, and an adder 417. The dropped frame control circuit 104 supplies the number of dropped frames calculated by the method described in the first embodiment to the moving image input means 101 and, at the same time, to the adder 417 via the line 46.

The comparator 113 calculates the average value of the quantization step width output from the image quality parameter calculation circuit 110 and the set value circuit 115 by the method described in the first embodiment.
Threshold value T1 or T2 (where T1> T2 )
A value of +1, 0, or -1 is output to the adder 417 according to the result of the magnitude comparison with.

The adder 417 calculates the sum of the output value from the comparator 113 and the number of dropped frames supplied from the line 46 to determine the target dropped frame number S and supplies it to the multiplier 141. I do.

With the above configuration, in the second embodiment,
Since the target number of dropped frames is changed within a range of + -1 based on the actual number of dropped frames, the number of dropped frames resulting from the encoding of the frame is different from the actual number of dropped frames. However, it is possible to always control such that the local change in the frame rate is minimized.

(Embodiment 3) FIG. 5 is a diagram showing a third embodiment of the present invention, in which an output signal (more specifically, an encoding circuit) of the image encoding apparatus shown in FIG. 1 or FIG. 102
2 or an output of the buffer memory 103).
Hereinafter, the image decoding apparatus will be described with reference to FIG.

First, codes (for example, Huffman codes) input from the input line 501 are stored in the buffer memory 502 and then supplied to the code converter 503. The code converter 503 decodes the input code and extracts the quantized discrete cosine coefficient value and the quantization step width. The quantized discrete cosine coefficient value is supplied to an inverse quantizer 504 together with the quantization step width and inversely quantized. The inversely quantized value is supplied to an inverse discrete cosine transformer 505.
Supplied to In the inverse discrete cosine transformer 505, the input value is subjected to inverse discrete cosine transform for each block and supplied to the adder 506. In the adder 506, the value supplied from the inverse discrete cosine transformer 505 and the frame memory 5
07 to generate a decoded image by adding the pixel value of the immediately preceding decoded frame stored in the frame memory 507.
To be stored. The decoded image stored in the frame memory 507 is output to a display system or the like via an output line 509.

As described above, the output signal of the moving picture coding apparatus shown in FIG. 1 or 4 can be easily decoded by the decoding apparatus shown in FIG.

The output signal of the image coding apparatus shown in FIG. 1 or 4 (more specifically, the output of the coding circuit 102 or the output of the buffer memory 103) is supplied to one end.
The information may be stored and recorded on a medium by a magnetic or optical method, and then the information on the medium may be decoded by the image decoding apparatus in FIG.

[0034]

In the image coding in a system including frame dropping, there is a trade-off between the frame rate and the image quality.
The value of the frame rate that achieves the target value of the image quality differs depending on the type of the input image. In the related art, it was necessary to set an optimum frame rate in advance according to the feature amount of an image, but it is not possible to set an optimum relationship between an optimum frame rate and image quality for various types of input images. If it is possible and the type of image changes abruptly over time, the frame rate and image quality will change abruptly, and the image quality level cannot be stably maintained.

In the present invention, the amount of change in the number of target dropped frames is determined so that the image quality of the coded image falls between the predetermined threshold values Th1 and Th2, and the quantization characteristic is controlled based on this value. , It is possible to automatically converge to a frame rate value that can maintain a constant image quality level for various types of images.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image encoding apparatus according to Embodiment 1 of the present invention.

FIG. 2 is a main part block connection diagram of the image coding apparatus according to the first embodiment;

FIG. 3 is an operation flowchart of the image coding apparatus according to the first embodiment.

FIG. 4 is a block connection diagram of an image encoding device according to Embodiment 2 of the present invention.

FIG. 5 is a block diagram of an image decoding apparatus according to Embodiment 3 of the present invention.

FIG. 6 is a block diagram of a conventional image coding apparatus.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 Moving image input means 102 Encoding circuit 103 Buffer memory 104 Dropout control circuit 105 Monitoring circuit 110 Image quality parameter calculation circuit 113 Comparator 115 Setting circuit 117 Adder 118 Upper / lower limit value setting circuit 119 Storage 130 Assigned code amount calculation circuit 141 Multiplier 142 quantization characteristic control circuit

Claims (4)

(57) [Claims] 1. An encoding means for dividing an input frame of an image into blocks in advance and encoding each of the blocks, a frame removal control means for determining the number of frame removal frames in accordance with the accumulation amount of a buffer memory, and Image quality parameter value calculating means for calculating an image quality parameter value from information of an encoded frame; and when the image quality parameter value is larger than a predetermined threshold T1, increasing the number of target frame-dropped frames, and setting the image quality parameter value in advance. When the image quality parameter value is smaller than the preset threshold value T1 and smaller than the threshold value T2, the target frame drop frame number is reduced from the immediately preceding value when the image quality parameter value is smaller than the predetermined threshold value T2 (where T2 <T1 ). Target frame drop frame characterized by control so as not to change Calculating means, an allocated code amount calculating means for calculating an allocated code amount per one frame time of the input image, and a target code amount of a frame using the target number of dropped frames and the allocated code amount per one frame time. An image coding apparatus comprising: a target code amount calculation unit for calculating; and a quantization characteristic determination unit for determining a quantization characteristic for each block according to the target code amount. 2. An encoding unit that divides an input frame of an image into blocks in advance and performs encoding for each block, a frame removal control unit that determines the number of frame removal frames according to the amount of storage in a buffer memory, Image quality parameter value calculating means for calculating an image quality parameter value from the encoded frame information, and when the image quality parameter value is greater than a predetermined threshold T1, increasing the target frame-dropping frame number from the immediately preceding frame-dropping frame number; When the image quality parameter value is smaller than a preset threshold value T2 (but T2 <T1 ), the target number of dropped frames is made smaller than the immediately preceding dropped frame number, and the image quality parameter value is set in advance. When the value is smaller than the threshold value T1 and larger than the threshold value T2, the target number of dropped frames is set to the immediately preceding dropped frame. A target dropped frame number calculating means for controlling the number of frames not to be changed, an allocated code amount calculating means for calculating an allocated code amount per one frame time, and using the target dropped frame number and the allocated code amount. An image coding apparatus, comprising: a target code amount calculating unit that calculates a target code amount of a frame by using a coding method; and a quantization characteristic determining unit that determines a quantization characteristic for each block according to the target code amount. 3. The image encoding apparatus according to claim 1, wherein the image quality parameter value is an average value of quantization step widths of already encoded frames. 4. A first storage device for storing a generated code amount of a plurality of coded frames, and a second storage device for storing the number of dropped frames generated immediately after coding each of the coded frames.
4. The image coding apparatus according to claim 1, further comprising: a storage unit for calculating an allocated code amount per one frame time from the generated code amount and the number of dropped frames. .