JPH07236138A - Moving image encoding device - Google Patents

Abstract

PURPOSE:To improve picture quality in a specific area while preventing picture quality from being suddenly deteriorated by stepwise controlling a quantizing step at the time of allocating much encoding information to the specific area. CONSTITUTION:Encoding information found out in each block i.s stored in a buffer memory 6 and an area extracting part 13 extracts a specific area from an input picture. A control part 15 controls the size of a quantizing step based upon an encoding position on a current screen which is found our by an encoding part 5 and the buffer occupation amount of encoding information in a buffer memory 6 and allocates the encoding information to the specific area and the other area. In this case, the quantizing step size is stepwise changed in accordance with the information of a current picture obtained from an encoding position calculating part 14 and the quantizing step size is controlled in accordance with the specific area information. Since much encoding information can be allocated to the specific area while preventing picture quality from being suddenly deteriorated, the picture quality of the specific area can be improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding system in which an input image is divided into two-dimensional block units, orthogonally transformed, and quantized and coded. In particular, a specific area is extracted from a screen. The present invention relates to a moving picture coding apparatus that controls the quantization step size for each area.

[0002]

2. Description of the Related Art In recent years, image communication services such as video conferencing and video telephone have been promising as a service that effectively utilizes ISDN, and research on high efficiency encoding for the efficient transmission of such moving images. Is being actively conducted. These studies reduce the amount of information by removing the redundancy contained in the image signal by using the statistical property of the image signal. As such an encoding method, a hybrid encoding method combining motion compensation prediction and discrete cosine transform is well known. However, when transmission is performed at a low bit rate, noise occurs in a coded image and the image quality deteriorates. Therefore, it is desired to improve this. As a method of improving image quality, a method of extracting a specific area from an input image and controlling the quantization step size for each area is being studied. As an example of this, by extracting a face area from the input image and performing quantization on a non-face area (hereinafter referred to as a background area) with a quantization step size larger than that of the face area, a large number of face areas are extracted. There is an idea to allocate the amount of information to improve subjective image quality (for example, RHJMPlompen, et al .: "An image knowledge bas
ed video codec for low bitrates ", SPIE Vol.804 Adv
anced in image processing (1987)). Hereinafter, a conventional example using motion compensation prediction and two-dimensional orthogonal transformation will be described with reference to FIG. In FIG. 6, 22
Is a frame memory unit, 23 is a subtractor, 24 is an orthogonal transformation unit, 25 is a quantization unit, 26 is an encoding unit, 27 is a buffer memory unit, 28 is an inverse quantization unit, 29 is an inverse orthogonal transformation unit, 3
Reference numeral 0 is an adder, 31 is a frame memory unit, 32 is a motion detection unit, 33 is a motion compensation prediction unit, 34 is a region extraction unit, and 35 is a quantization step size control unit.

Now, assume that an image is input to the frame memory unit 22. The input image is a digitized image from a television camera or the like, is accumulated in the frame memory unit 22, and is divided into blocks of N × M pixels (N and M are natural numbers). In the subtractor 23, the frame memory unit 22
Difference between the input image and the motion-compensated prediction value from the motion-compensated prediction unit 33 is calculated for each block, and the orthogonal transform unit 24 performs two-dimensional orthogonal transform on the pixels of each block to quantize the transform coefficient. It is sent to the unit 25. The quantization unit 25 quantizes the transform coefficient according to the quantization step size output from the quantization step size control unit 35. The encoder 26 entropy-encodes the quantized output from the quantizer 25 to generate encoded information. The buffer memory unit 27 stores coded information in order to match the transmission rate of the line. The output from the quantizing unit 25 is also input to the inverse quantizing unit 28, and inverse quantization is performed to obtain transform coefficients. The inverse orthogonal transform unit 29 performs a two-dimensional inverse orthogonal transform on the transform coefficient, and the image obtained by adding the motion compensation prediction value from the motion compensation prediction unit 33 by the adder 30 is the frame memory unit 3.
Accumulated in 1. The image stored in the frame memory unit 31 and the image stored in the frame memory unit 22 are input to the motion detection unit 32, and the motion vector is detected.

The motion compensation prediction unit 33 obtains a motion compensation prediction value from the motion vector and the image stored in the frame memory unit 31. The quantization step size control unit 35 receives valid / invalid information indicating the specific area and information other than the specific area, which is the output of the area extraction unit 34, and the buffer occupancy amount of the coded information in the buffer memory unit 27. The quantization step size is determined based on For example, when the area extraction unit 34 sets the face area as the valid area and the background area as the invalid area, a quantization step size smaller than the background area is selected for the face area based on the quantization step size obtained from the buffer occupancy. To be done.

[0005]

According to the above method, only two kinds of quantization step sizes are set for the face area and the background area, and the quantization step size for the face area is smaller than the background area. Not revealed. Therefore, when this is applied to an actual moving picture coding apparatus, the quantization step size Q determined by the buffer occupancy of the coding information in the buffer memory section changes according to the result extracted by the area extracting section. By defining the amounts dQf and dQb, Q-dQf,
The background area may be quantized using Q + dQb. However, if the quantization step size is determined by such a method, adaptive control of the amount of coded information (hereinafter referred to as the code amount) is not performed, and therefore Q + dQb when the background region is subsequently coded. Since the quantization is performed with the quantization step size of, the code amount is decreased and the buffer occupancy is decreased, and the quantization step size determined by this is a very small value. When the face area is coded in such a state, a smaller quantization step size is selected by Q-dQf, so that the code amount sharply increases and the buffer occupancy rapidly increases.
As a result, there is a problem that the quantization step size determined from the buffer occupancy suddenly increases, and the image quality of the next face area deteriorates. The present invention is a moving image capable of improving the image quality of a specific area without degrading the image quality of the specific area by controlling the quantization step size stepwise when allocating a large amount of encoded information to the specific area. An object is to provide an image encoding device.

[0006]

In order to solve the above-mentioned problems, first, an input image is subjected to orthogonal transform in block units of N × M pixels (N and M are natural numbers) to obtain transform coefficients, and the transform coefficients are calculated. In a moving picture coding apparatus for transmitting coded information generated by quantizing / coding by a quantizing / coding means to a partner side via a line, a buffer for matching the coded information with the transmission rate of the line. Memory means, area extracting means for extracting a specific area from an input image, coding position calculating means for calculating the current coding position on the screen performed by the encoding means, and quantizing the transform coefficient. Quantization step size control means for controlling the quantization step size at the time of performing is provided, the extraction result extracted by the area extraction means, the coding position calculated by the coding position calculation means, and the buffer. Those configured to control the quantization step size by the quantization step size control means on the basis of the buffer occupancy of the coded information in the memory means.

Further, in the above moving picture coding apparatus, the quantization step size determined by the buffer occupancy amount of the coding information in the buffer memory means is coded on the current screen which is being performed by the coding means. The quantization step size is controlled in accordance with the information of the coding position calculation means that gives the position, and the quantization step size is controlled according to the information for distinguishing the specific area and the other area as the extraction result of the area extraction means. Configured,
Further, in the moving picture coding apparatus, a value to be added to the quantization step size determined by the buffer occupancy amount of the coding information in the buffer memory means is the current screen being performed by the coding means. According to the information of the coding position calculating means for giving the above coding position, the block to be coded first in one screen has a larger value than the block to be coded later, and the extraction of the area extracting means is performed. According to the information for distinguishing the specific region and the other region as the result, by adding a value smaller than the region other than the specific region to the specific region and performing quantization, a large amount of coding information is added to the specific region. Is configured to be assigned.

[0008]

The coded information generated by quantizing and coding the transform coefficient obtained by performing the two-dimensional orthogonal transform in block units on the difference between the input image and the motion-compensated predicted value is transmitted through the line. Stored in buffer memory means for matching speed. Further, the extraction result of the area extracting means for extracting a specific area from the input image, the current encoding position on the screen performed by the encoding means, and the buffer occupancy of the encoding information in the buffer memory means Based on the control of the quantization step size, the allocation of the coding information is controlled for the specific area and the area other than the specific area.
As a method of controlling the quantizing step size, the quantizing step size determined by the buffer occupancy of the coding information in the buffer memory means is stepped according to the information of the coding position calculating means which gives the current coding position on the screen. By changing the same, and by controlling the quantization step size in accordance with the information that distinguishes the specific region and the other region that is the extraction result of the region extraction means,
The allocation of encoded information is controlled for a specific area and an area other than the specific area while preventing a sharp deterioration in image quality. Also,
A coding position at which a value added to the quantization step size determined by the buffer occupancy of the coding information in the buffer memory means gives the current coding position on the screen performed by the coding means. According to the information of the calculation means, the block that is encoded first in one screen has a larger value than the block that is encoded later. Further, in accordance with the information that distinguishes the specific region and the other regions as the extraction result of the region extracting means, the specific region is quantized by adding a value smaller than the regions other than the specific region, and the specific region is quantized. A large amount of coded information is assigned to the image data to improve the image quality of a specific area.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an embodiment of the present invention, in which a frame memory unit 1 for accumulating an input image and a difference between the input image and the motion compensation prediction value are connected to the frame memory unit 1 and the motion compensation prediction unit 11. A subtractor 2 to be obtained, an orthogonal transformation unit 3 connected to the subtraction unit 2 for orthogonally transforming a difference between an input image and a motion compensation prediction value in block units and outputting transform coefficients, and an orthogonal transformation unit 3 connected to the orthogonal transformation unit 3. The quantizing unit 4 which quantizes the transform coefficient from the transforming unit 3 according to the quantizing step size determined by the quantizing step size control unit 15, and the quantizing transform coefficient which is connected to the quantizing unit 4 and which is quantized. An encoding unit 5, a buffer memory unit 6 connected to the encoding unit 5 and accumulating the encoded information from the encoding unit 5, and an inverse quantization of the quantized transform coefficient connected to the quantization unit 4. Dequantization unit 7 and a connection to the dequantization unit 7 An inverse orthogonal transform unit 8 for inverse orthogonal transform the transform coefficients from the inverse quantization unit 7, connected to the inverse orthogonal transform unit 8 and the motion compensation prediction unit 12 inverse orthogonal transform unit 8
Adder 9 for adding the motion compensation prediction value and the image obtained in
A frame memory unit 10 connected to the adder 9 for accumulating the output image of the adder 9; a motion detection unit 11 connected to the frame memory unit 10 and the frame memory unit 1 for detecting a motion vector; Motion compensation prediction unit 1 that is connected to the memory unit 10 and the motion detection unit 11 to obtain a motion compensation prediction value
2, an area extraction unit 13 that connects to the frame memory unit 1 and extracts a specific area, an encoding position calculation unit 14 that calculates an encoding position on the current screen, and the buffer memory unit 6
And a quantization step size control unit 15 which is connected to the area extraction unit 13 and the coding position calculation unit 14 and determines a quantization step size.

The operation of the moving picture coding apparatus having the above configuration is as follows. The frame memory unit 1 stores the input image. The difference between the input image accumulated in the frame memory unit 1 in the subtractor 2 and the motion compensation prediction value calculated in the motion compensation prediction unit 12 is calculated by a two-dimensional discrete cosine transform (DCT) for each block of 8 × 8 pixels. ) Is carried out,
DCT by converting time domain signal to frequency domain signal
The coefficient is output to the quantizer 4. The quantization unit 4 quantizes the DCT coefficient according to the quantization step size determined by the quantization step size control unit 15 to obtain high coding efficiency, and reduces the DCT coefficient to be coded. The DCT coefficient quantized in this way is output to the encoding unit 5. The encoding unit 5 performs entropy encoding that appropriately assigns codes to the quantized DCT coefficients, and generates and outputs encoding information composed of a variable length code. The buffer memory unit 6 stores the coded information in order to match the coded information generated by the coding unit 5 with the transmission rate of the line, and outputs the coded information at a constant rate. Further, the amount of coded information that occupies the buffer memory unit 6 (buffer occupancy amount) is input to the quantization step size control unit 15. The inverse quantization unit 7 performs inverse quantization, which is the opposite process to that performed by the quantization unit 4, and outputs the inversely quantized DCT coefficient to the inverse orthogonal transform unit 8. The inverse orthogonal transform unit 8 performs a two-dimensional inverse discrete cosine transform, and the adder 9 performs the inverse orthogonal transform unit 8
Is added to the motion compensation prediction value of the motion compensation prediction unit 12, and the result is stored in the frame memory unit 10. The motion detection unit 11 detects a motion vector between the image in the frame memory unit 10 and the image in the frame memory unit 1, and outputs the motion vector to the motion compensation prediction unit 12. The motion compensation prediction unit 12 obtains a motion compensation prediction value from the image of the frame memory unit 10 and the motion vector of the motion detection unit 11.

The area extracting unit 13 extracts a specific area which is a target for improving image quality. For example, when the moving picture coding apparatus is applied to a video conference or a videophone, the area of general interest is the face, and the facial area is extracted and the image quality of the facial area is improved to improve the subjective image quality. Can be achieved. Therefore, here, a method of extracting the face area with the specific area as the face area will be described. First,
The Cr signal and Cb signal of the 256-gradation input image stored in the frame memory unit 1 are binarized with a threshold value for extracting a skin color region to obtain a binary image in which a region including a face portion is extracted. 2A and 2B show Cr signal and Cb, respectively.
The skin color region obtained by binarizing the signal is shown. Next, the common portion of the skin color region extracted by binarizing the Cr signal and the Cb signal is extracted to obtain (c) of FIG. With this as a face area, a block including pixels in the face area is set as a valid block, and a block not including a pixel is set as an invalid block, and valid / invalid information is input to the quantization step size control unit 15 in block units. The coding position calculation unit 14 obtains the position on the screen which is currently being coded, and inputs the coding position information to the quantization step size control unit. This is 1 as shown in FIG.
If the screen is composed of 99 blocks, encoding is performed in order from the block 1 at the upper left to the block 99 at the lower right. Each time the coding of one block is completed, the block number is counted up to obtain the coding position information. Quantization step size controller 15
Determines the quantization step size based on the buffer occupancy of the buffer memory unit 6, the valid / invalid information of the area extraction unit 13, and the coding position information of the coding position calculation unit 14.
An example of this configuration is shown in FIG. The quantization step size calculation unit 16 determines in advance the relationship between the buffer occupancy and the quantization step size that prevents the buffer memory unit 6 from overflowing or underflowing, and calculates the quantization step size Q according to this relationship. Adder 1
Input to 7 and 18. The face area quantization step size change value setting unit 19 selects an amount dQf for changing the quantization step size Q in the face area by a certain value based on the encoding position information. In general, in a video conference or a videophone, the face area is not located from the top of the screen, and there is a high possibility that about 1/4 of the screen is the background area. Therefore, quantization is performed with a large quantization step size in the upper part of the screen, the code amount is very small, and the quantization step size determined by the quantization step size calculator 16 is a very small value. When the face area is input in such a state, the quantization step size becomes a smaller value. When the quantization is performed with this value, the code amount increases abruptly, and as a result, the quantization step size obtained by the quantization step size calculation unit 16 becomes a large value, and the image quality of the next face area deteriorates rather. It will be. Therefore, in the face area, dQf
Is set to a larger value for a block coded earlier on one screen and gradually decreases for a later block, thereby preventing a rapid increase in the code amount. FIG. 5 shows the setting of the quantization step size change value dQf for the face area based on the coding position information, and dQ is changed for every 11 blocks.
The position where dQf is changed may be every 33 blocks or every block. The background region quantization step size change value setting unit 20 sets the amount dQb for changing the quantization step size Q in the background region by a certain value. Here, dQb is fixed irrespective of the coding position, but it may be changed according to the coding position information as in the face area quantization step size change value. The change values dQf and dQb thus obtained are input to the adders 17 and 18, respectively, and the face area quantization step size Qf and the background area quantization step size Qb are given by equations (1) and (2). Is calculated as follows.

[0012]

[Equation 1]

[0013]

[Equation 2]

For example, when Q = 10, dQb = 16, and the coding position information is 67, dQf is -4 from FIG.
Is set, and Qf and Qb are expressed by equations (3) and (4), respectively.

[0015]

[Equation 3]

[0016]

[Equation 4]

The switch 21 selects the output of the adder 17 or the adder 18 according to the valid / invalid information from the area extracting section 13. That is, when the valid / invalid information is valid, the output of the adder 17 is selected, and when the valid / invalid information is invalid, the output of the adder 18 is selected and Qf or Qb is output as the quantization step size to determine the coding position. The quantization step size can be controlled for each region.

[0018]

The present invention has the following effects. (1) A specific area is extracted from the input image, and the quantization step size is controlled based on the current encoding position on the screen where encoding is performed and the buffer occupancy of the encoding information in the buffer memory. As a result, it is possible to control the allocation of the coding information to the specific area and the area other than the specific area. (2) Information that changes the quantization step size determined by the buffer occupancy of the encoded information in the buffer memory in a stepwise manner according to the current encoding position on the screen, and distinguishes the specific area from other areas Accordingly, by controlling the quantization step size, it is possible to control the allocation of the coding information to the specific area and the area other than the specific area while preventing the sharp deterioration of the image quality. (3) The value to be added to the quantization step size determined by the buffer occupancy of the encoded information in the buffer memory is the value of the block to be encoded first in one screen according to the current encoding position on the screen. The value is larger than that of the block to be coded later, and according to the information that distinguishes the specific area from other areas, a value smaller than the areas other than the specific area is added to the specific area and the quantum is added. By allocating a large amount of encoded information to the specific area by changing the encoding, it is possible to improve the image quality of the specific area while preventing a sharp deterioration of the image quality.

[Brief description of drawings]

FIG. 1 is a block diagram illustrating an embodiment of the present invention.

FIG. 2 is a diagram illustrating extraction of a face area.

[Fig. 3] Fig. 3 is a diagram illustrating an encoding position on one screen.

FIG. 4 is a block diagram illustrating a configuration of a quantization step size control unit.

FIG. 5 is a diagram illustrating setting of a quantization step size change value for face area.

FIG. 6 is a block diagram illustrating a conventional hybrid coding method.

[Description of Codes] 1 frame memory unit 2 subtractor 3 orthogonal transformation unit 4 quantization unit 5 encoding unit 6 buffer memory unit 7 inverse quantization unit 8 inverse orthogonal transformation unit 9 adder 10 frame memory unit 11 motion detection unit 12 Motion compensation prediction unit 13 Region extraction unit 14 Coding position calculation unit 15 Quantization step size control unit

Claims (3)

[Claims] 1. An input image is orthogonally transformed in block units of N × M pixels (N and M are natural numbers) to obtain transform coefficients, and the transform coefficients are quantized / encoded by a quantization / encoding means to generate them. In a moving picture coding apparatus for transmitting the coded information to the other side via a line, buffer memory means for matching the coded information with the transmission rate of the line, and area extraction for extracting a specific area from an input image Means, and coding position calculation means for calculating the current coding position on the screen, which is performed by the coding means,
A quantization step size control unit for controlling a quantization step size when quantizing the transform coefficient is provided, and an extraction result extracted by the region extraction unit and a coding position calculated by the coding position calculation unit, The moving picture coding apparatus is characterized in that the quantizing step size control means controls the quantizing step size based on the buffer occupancy of the coding information in the buffer memory means. 2. A coding position calculating means for giving a quantization step size determined by a buffer occupation amount of coding information in the buffer memory means to a current coding position on the screen being performed by the coding means. And the quantization step size is controlled according to the information for distinguishing the specific region and the other region which is the extraction result of the region extracting means. Item 2. The moving image encoding device according to item 1. 3. A value added by the encoding means for allocating a large amount of encoded information to a specific area with respect to a quantization step size determined by a buffer occupation amount of the encoded information in the buffer memory means. According to the information of the coding position calculation means that gives the current coding position on the screen, the block to be coded first in one screen has a larger value than the block to be coded later. According to the information for distinguishing the specific region and the other regions, which is the extraction result of the region extracting means, the specific region is added with a value smaller than that of the regions other than the specific region and quantized. The moving picture coding device according to claim 1 or 2.