JP2009055236A - Video coder and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform coding wherein deterioration is hardly visually conspicuous by determining conversion block size based on feature information extracted from an input video signal. <P>SOLUTION: In this video coder, a block prediction device 12 divides the respective images of a video signal from an input terminal 10 into macro blocks of 16×16 pixels. An orthogonal transformation device 16 divides the macro blocks from the device 12 into transformation blocks of 8×8 pixels to perform orthogonal transformation. An orthogonal transformation device 18 divides the macro blocks from the device 12 into transformation blocks of 4×4 pixels to perform the orthogonal transformation. Code amount estimation devices 20, 22 estimate generated code amounts by the devices 16, 18, respectively. A feature extraction device 14 extracts a part in which deterioration also tends to be conspicuous on human visual characteristics from the input video signal. A judgment device 26 controls a switching device 4 so as to select output transformation coefficient data of the device 18 to the part in which the deterioration tends to be conspicuous, and controls the switching device 24 so as to select the one with smaller estimation amount of the devices 20, 22 to other parts. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video encoding apparatus and method for compressing and encoding a video signal with high efficiency.

  In recent years, with the advancement of digital signal processing technology, a large amount of digital information such as moving images, still images, and sounds can be encoded with high efficiency, and can be recorded on a recording medium and transmitted via a communication medium.

Such a compression method for high-efficiency encoding of an image includes orthogonal transformation in the encoding process. It is known that coding efficiency can be improved by adaptively changing the block size of orthogonal transform. Patent Document 1 describes a technique for adaptively changing the block size.
Special Table 2003-523651

  The technique described in Patent Document 1 determines a block size according to a change in pixel value. However, when this method is applied, the reproduced image is not always suitable for human viewing. For example, even if it is a slight deterioration mathematically, it is very conspicuous for human beings, or even though there is a large amount of information missing due to encoding, humans have little deterioration. There are things that I don't care about. The conventional methods do not take into consideration such visual characteristics peculiar to human beings and often feel sensory deterioration.

  An object of the present invention is to provide a video encoding apparatus and method for adaptively controlling an encoding block size in consideration of visual characteristics.

  In order to achieve the above object, a video encoding device according to the present invention includes a block dividing unit that divides an image constituting each screen of a video signal into a plurality of macroblocks, And a feature extraction unit that extracts a portion where deterioration is easy to be noticeable, and an image data in units of macroblocks from the block dividing unit for the portion that is not the portion where deterioration appears easily according to the extraction result of the feature extraction unit. Are orthogonally transformed with the first transformation block size, and the second transformation block size smaller than the first transformation block size is converted into the macro block unit image data from the block dividing means for the portion where deterioration is likely to occur. And a transforming means for performing orthogonal transform at.

  The video encoding apparatus according to the present invention includes a block dividing unit that divides an image constituting each screen of the video signal into a plurality of macroblocks, and a visual appearance of deterioration that is easily noticeable in each screen of the video signal. Feature extracting means for extracting a portion, first conversion means for converting the image data in units of macroblocks from the block dividing means into first conversion coefficient data with a first conversion block size, and the block dividing means Second conversion means for converting the image data in units of macroblocks from the image data into second conversion coefficient data with a second conversion block size smaller than the first conversion block size, and the first and second conversions Switching means for selecting one of the coefficient data, first code amount estimating means for estimating a code amount when the first converting means is used, and the second code data A second code amount estimating means for estimating a code amount when the conversion means is used; the switching means is controlled in accordance with the estimated amounts of the first and second code amount estimating means and the extraction result of the feature extracting means; And a determination unit configured to determine, based on the extraction result of the feature extraction unit, that the estimated amount of the first code amount estimation unit is equal to the second code for a portion that is not easily deteriorated. When the amount is smaller than the estimated amount of the amount estimating unit, the switching unit selects the first transform coefficient data, and the estimated amount of the first code amount estimating unit becomes the estimated amount of the second code amount estimating unit. When equal to or greater than this, the switching means selects the second conversion coefficient data, and the switching means causes the switching means to select the second conversion coefficient data for the portion where deterioration easily occurs. Characterized in that to select.

  In the video encoding method according to the present invention, a block division step of dividing an image constituting each screen of the video signal into a plurality of macroblocks, and visual degradation is easily noticeable in each screen of the video signal. A feature extraction step of extracting a deterioration-prone appearance part, and according to the extraction result of the feature extraction step, the macroblock unit image data obtained by the block division step is converted into a first transform block for the part that is not a deterioration-prone appearance part. A transform step of performing orthogonal transform with a size, and orthogonally transforming the image data in units of macroblocks by the block dividing unit with a second transform block size smaller than the first transform block size, with respect to the easily appearing deterioration portion It is characterized by having.

  The video encoding method according to the present invention includes a block division step for dividing an image constituting each screen of a video signal into a plurality of macroblocks, and a visual appearance of deterioration that is easily noticeable in each screen of the video signal. A feature extraction step of extracting a portion, a first conversion step of converting the image data in units of macroblocks in the block division step into first conversion coefficient data in a first conversion block size, and the block division step When the second conversion step of converting the image data in units of macroblocks into the second conversion coefficient data with a second conversion block size smaller than the first conversion block size, and using the first conversion step The code amount when the first code amount estimation step for estimating the code amount of the second conversion step and the second conversion step are used. When the estimated amount of the first code amount estimating step is smaller than the estimated amount of the second code amount estimating step with respect to the second code amount estimating step for estimating When the first transform coefficient data is selected and the estimated amount of the first code amount estimating step is equal to or larger than the estimated amount of the second code amount estimating step, the second transform coefficient is selected. A selection step of selecting data and selecting the second conversion coefficient data for the portion where deterioration is likely to occur.

  In the present invention, since the transform block size is determined based on the feature information extracted from the input video signal, it is possible to perform coding that is less visually noticeable. That is, a video encoding device that outputs a stream suitable for human visual characteristics can be realized.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of the present invention. A video signal is input to the input terminal 10 from the outside. The video signal input from the external video source to the input terminal 10 is supplied to the block prediction device 12 and the feature extraction device 14. The external video source is, for example, an imaging unit of a video camera or a television broadcasting facility.

  The block prediction device 12 divides a picture (image) constituting each screen of the video signal from the input terminal 10 into a plurality of 16 × 16 pixel macroblocks, and performs a prediction process using the correlation within the screen or between the screens. Do. The block prediction device 12 corresponds to the block dividing means in the claims. For example, the prediction processing is performed according to the ISO / IEC13818-2 standard and ITU-TH. H.264 standard. Prediction methods include intra prediction and inter prediction. Intra prediction is a method for calculating a difference value between adjacent macro pixel information with high correlation for each macro block. Inter prediction is a method of searching for a highly correlated pixel block from a reference picture encoded in the past for each macroblock and calculating a difference value.

  The block prediction device 12 uses the 8 × 8 orthogonal transform device 16, the 4 × 4 orthogonal transform device 18, and the 8 × 8 code amount estimation devices 20 and 4 for the difference value calculated by the prediction of either intra prediction or inter prediction. X4 Code amount estimation device 22 is supplied.

  The 8 × 8 orthogonal transform device 16 divides the image data (difference value) in units of macroblocks supplied from the block prediction device 12 into transform blocks of 8 × 8 pixels (first transform block size), and performs two-dimensional discrete Cosine transform. The 4 × 4 orthogonal transform device 18 divides the image data (difference value) in units of macro blocks supplied from the block prediction device 12 into transform blocks of 4 × 4 pixels (second transform block size), and performs two-dimensional discrete processing. Cosine transform. The transform coefficient data output from the 8 × 8 orthogonal transform device 16 corresponds to the first transform coefficient data in the claims, and the transform coefficient data output from the 4 × 4 orthogonal transform device 18 is claimed. This corresponds to the second conversion coefficient data in the range. FIG. 2 shows the relationship between a 16 × 16 pixel macroblock, an 8 × 8 pixel conversion block, and a 4 × 4 pixel conversion block. The 8 × 8 orthogonal transform device 16 and the 4 × 4 orthogonal transform device 18 supply transform coefficient data of two-dimensional orthogonal transform to the switching device 24.

  The 8 × 8 code amount estimation device 20 estimates the generated code amount by the two-dimensional discrete cosine transform of the 8 × 8 orthogonal transform device 16. The 4 × 4 code amount estimation device 22 estimates the generated code amount by the two-dimensional discrete cosine transform of the 4 × 4 orthogonal transform device 18. The 8 × 8 code amount estimation device 20 corresponds to first code amount estimation means in the claims, and the 4 × 4 orthogonal transform device 18 corresponds to second code amount estimation means. For example, each of the code amount estimation devices 20 and 22 calculates a variance value of each block in each transform block size from the difference value supplied from the block prediction device 12, and outputs the sum as an estimated value of the generated code amount. . Since blocks with small variance are generally composed of low-frequency components, the code amount after orthogonal transformation tends to be small. On the other hand, blocks with large variance are generally composed of high-frequency components, so the amount of code after orthogonal transformation tends to be large. Here, the dispersion value is used as the evaluation value. However, the power of the high frequency component may be calculated by Fourier transform or the like, or the correct code amount may be calculated by actually performing orthogonal transform.

  The 8 × 8 code amount estimation device 20 and the 4 × 4 code amount estimation device 22 supply the estimation result to the determination device 26. Further, output data of the feature extraction device 14 is also supplied to the determination device 26. The determination device 26 controls the switching device 24 according to the code amount estimation results from the code amount estimation devices 20 and 22 and the extraction result of the feature extraction device 14. That is, the determination device 26 functions as a switching control device for the switching device 24.

  The feature extraction device 14 extracts, from the video signal from the input terminal 10, a portion having important feature information in terms of human visual characteristics for each macroblock to be processed by the block prediction device 12. For example, intricate areas such as grasslands and trees tend to be less worrisome to human eyes even if they deteriorate. On the other hand, flat portions such as the sky and walls, and edge portions such as material boundaries tend to feel visually unnatural even if they are slightly degraded. Alternatively, an area that is easily visually observed, such as a human face, can also be said to be an area where deterioration due to encoding is conspicuous. The feature extraction device 14 extracts a portion where deterioration is conspicuous in the human visual characteristics, that is, a portion where deterioration appears easily.

  Specifically, the feature extraction device 14 first extracts a flat portion where deterioration is conspicuous and a high-frequency block where deterioration is not conspicuous by frequency component analysis in units of macroblocks. For example, Hadamard transform can be used for the analysis of frequency components. It is determined whether or not a region that is not a flat portion includes a horizontal, vertical, or diagonal edge, and if it includes an edge, it is assumed that the block is easily deteriorated.

  Furthermore, the feature extraction device 14 detects a person with a known face detection technique 鵜 or skin color detection technique for the entire screen or a partial area, and whether the processing target macroblock is included in the detected range. Determine whether or not. When the macroblock is included in a range that is regarded as a person (or face), it is assumed that the block is easily deteriorated even in a region that is not a flat portion.

  Furthermore, the feature extraction apparatus 14 refers to a plurality of screens to detect a moving object for the entire screen or a partial region, and determines whether or not the processing target macroblock is included in the range detected as the moving object. judge. When a moving object is detected, it is determined whether it is a low-speed moving object or a high-speed moving object with reference to a predetermined value. When the macroblock is included in a range that is regarded as a low-speed moving object, even if the block is determined not to be a flat portion, it is assumed that the block is a block where deterioration is conspicuous (a portion where deterioration appears easily). If it is included in a range that is regarded as a high-speed moving object, it is assumed that the block is a block in which deterioration is not conspicuous even if the block is determined to be a flat portion (deterioration difficult portion).

  The feature extraction device 14 supplies an index value or a flag indicating an easy-to-occur deterioration portion determined by such complex analysis to the determination device 26 as an extraction result. The feature extraction device 14 individually extracts any one of a flat part, an edge part, a target subject part, a person part, a skin color part, and a slow-moving object part of the video region as the deterioration-prone appearance part.

  The determination device 26 basically controls the switching device 24 so as to select the transform coefficient data of the orthogonal transform block size with the smaller estimated code amount value from the code amount estimating devices 20 and 22. However, from the feature extraction result of the feature extraction device 14, the determination device 26 causes the switching device 24 to select the output of the 4 × 4 orthogonal transformation device 18 for a macro block that is a block that is easily deteriorated. This is because, as schematically shown in FIG. 3, the smaller the block, the smaller the deterioration, and the larger the block, the larger the deterioration. That is, for the flat part, the edge part, and the subject of interest (such as a face, a person, or an object moving at a low speed), a 4 × 4 pixel conversion block is used.

  The feature extraction device 14 classifies macroblocks into three types: blocks that are prone to deterioration (prone portions where deterioration appears easily), blocks that make deterioration difficult to see (portions where deterioration appears difficult), and intermediate blocks (deterioration appearance intermediate portions). May be. In this case, for example, the determination device 26 causes the switching device 24 to select the output of the 4 × 4 orthogonal transform device 18 for blocks that are prone to deterioration, and 8 × 8 for the switching device 24 for blocks that are difficult to deteriorate. The output of the orthogonal transformation device 16 is selected. Then, the determination device 26 causes the switching device 24 to select the transform coefficient data of the orthogonal transform block size with the smaller generated code amount estimation value from the code amount estimation devices 20 and 22 for the intermediate block.

  The determination device 26 may further control the switching device 24 in consideration of the bit rate or compression rate of the compressed stream data.

  The quantizer 28 quantizes the orthogonal transform coefficient data selected by the switching device 24 and supplies the quantized data to the entropy encoder 30. The entropy encoding device 30 performs arithmetic encoding or variable length encoding on the quantized transform coefficient data from the quantizing device 28, and outputs a code string from the stream output terminal 32 to the outside.

  With the above configuration, by selecting two types of orthogonal transform block sizes in consideration of human visual characteristics, it is possible to perform encoding that is visually inconspicuous on the playback screen. A video encoding apparatus that outputs a stream suitable for human visual characteristics can be realized.

  In addition, it is possible to encode with a smaller code amount by adaptively selecting two types of orthogonal transform block sizes according to the estimated amount of generated code. If the code amount is the same, there is less information deterioration. Encoding is possible. Although examples of combinations of transform block sizes of 8 × 8 pixels and 4 × 4 pixels have been described, it is apparent that the present invention can be applied to other combinations of transform block sizes.

It is a schematic block diagram of one Example of this invention. It is a size comparison figure of a macroblock and two types of conversion blocks. It is a schematic diagram explaining the conversion block size and the spread of deterioration.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Input terminal 12 Block prediction apparatus 14 Feature extraction apparatus 16 8 * 8 orthogonal transformation apparatus 18 4 * 4 orthogonal transformation apparatus 20 8 * 8 code amount estimation part 22 4 * 4 code amount estimation part 24 Switching apparatus 26 Determination apparatus 28 Quantization Device 30 Entropy Coding Device 32 Stream Output Terminal

Claims (13)

Block dividing means for dividing an image constituting each screen of the video signal into a plurality of macroblocks;
In each screen of the video signal, feature extraction means for extracting easy-to-develop portions that are prominent in visual deterioration; and
According to the extraction result of the feature extraction unit, the macro block unit image data from the block dividing unit is orthogonally transformed with a first transform block size to the portion that is not the deterioration-prone portion, and the deterioration-prone portion And a transform unit that orthogonally transforms the image data in units of macroblocks from the block dividing unit with a second transform block size smaller than the first transform block size. apparatus.   The apparatus according to claim 1, further comprising: a quantizing unit that quantizes transform coefficient data output from the transforming unit; and an entropy encoding unit that entropy encodes output data of the quantizing unit. Video encoding device. The converting means is
First conversion means for converting the image data in units of macroblocks from the block dividing means into first conversion coefficient data with a first conversion block size;
Second conversion means for converting the image data in units of macroblocks from the block dividing means into second conversion coefficient data with a second conversion block size smaller than the first conversion block size;
3. The video encoding apparatus according to claim 1, further comprising a switching unit that selects one of the first and second transform coefficient data in accordance with an extraction result of the feature extraction unit. Block dividing means for dividing an image constituting each screen of the video signal into a plurality of macroblocks;
In each screen of the video signal, feature extraction means for extracting easy-to-develop portions that are prominent in visual deterioration; and
First conversion means for converting the image data in units of macroblocks from the block dividing means into first conversion coefficient data with a first conversion block size;
Second conversion means for converting the image data in units of macroblocks from the block dividing means into second conversion coefficient data with a second conversion block size smaller than the first conversion block size;
Switching means for selecting one of the first and second conversion coefficient data;
First code amount estimating means for estimating a code amount when the first converting means is used;
Second code amount estimating means for estimating a code amount when using the second conversion means;
Determining means for controlling the switching means according to the estimated amounts of the first and second code amount estimating means and the extraction result of the feature extracting means;
According to the extraction result of the feature extraction unit, the determination unit has an estimated amount of the first code amount estimating unit smaller than an estimated amount of the second code amount estimating unit with respect to a portion that is not easily deteriorated. If the switching means selects the first transform coefficient data, and the estimated amount of the first code amount estimating means is equal to or larger than the estimated amount of the second code amount estimating means The video coding apparatus characterized in that the switching means selects the second transform coefficient data and causes the switching means to select the second transform coefficient data for the portion where deterioration easily occurs. . The feature extraction means further extracts a deterioration-appearing difficult portion where visual deterioration is not conspicuous and a deterioration appearance intermediate portion between the deterioration-appearing easy portion and the deterioration-appearing difficult portion in each screen of the video signal. ,
The determination means is
When the estimated amount of the first code amount estimating unit is smaller than the estimated amount of the second code amount estimating unit with respect to the deterioration appearance intermediate portion, the first conversion coefficient data is selected by the switching unit. When the estimated amount of the first code amount estimating unit is equal to or larger than the estimated amount of the second code amount estimating unit, the switching unit selects the second transform coefficient data,
For the easy-to-occur deterioration portion, the switching means selects the second conversion coefficient data,
5. The video encoding apparatus according to claim 4, wherein the first transform coefficient data is selected by the switching unit for the portion where deterioration is difficult to appear.   6. The method according to claim 4, further comprising: a quantizing unit that quantizes transform coefficient data output from the switching unit; and an entropy encoding unit that entropy encodes output data of the quantizing unit. The video encoding device described in 1.   The feature extraction unit extracts any one of a flat part, an edge part, a target object part, a person part, a skin color part, and a low-speed moving object part of the video area as the easily appearing deterioration part. The video encoding device according to item 1. A block dividing step of dividing the image constituting each screen of the video signal into a plurality of macro blocks;
In each screen of the video signal, a feature extraction step of extracting a deterioration appearance easy portion where visual deterioration is conspicuous, and
According to the extraction result of the feature extraction step, the macroblock unit image data obtained by the block division step is orthogonally transformed with a first transform block size for the portion that is not the deterioration-prone appearance portion, and the deterioration appearance-prone portion is obtained. On the other hand, a video encoding method comprising: a transform step of orthogonally transforming the image data in units of macroblocks by the block division step with a second transform block size smaller than the first transform block size.   The video code according to claim 8, further comprising: a quantization step for quantizing the transform coefficient data in the transform step; and an entropy encoding step for entropy encoding the output data of the quantization step. Method. A block dividing step of dividing an image constituting each screen of the video signal into a plurality of macroblocks;
In each screen of the video signal, a feature extraction step of extracting a deterioration appearance easy portion where visual deterioration is conspicuous, and
A first conversion step of converting the image data in units of macroblocks by the block division step into first conversion coefficient data with a first conversion block size;
A second conversion step of converting the image data in units of macroblocks by the block division step into second conversion coefficient data with a second conversion block size smaller than the first conversion block size;
A first code amount estimating step for estimating a code amount when using the first conversion step;
A second code amount estimating step for estimating a code amount when using the second conversion step;
When the estimated amount of the first code amount estimating step is smaller than the estimated amount of the second code amount estimating step for the portion that is not easy to appear deterioration, the first transform coefficient data is selected, When the estimated amount of the first code amount estimating step is equal to or larger than the estimated amount of the second code amount estimating step, the second transform coefficient data is selected, and the deterioration easily occurring portion And a selection step of selecting the second transform coefficient data. The feature extracting step further extracts, in each screen of the video signal, a portion where deterioration is difficult to appear visually, and a portion where deterioration appears easily between the portion where deterioration is easy to appear and the portion where deterioration is difficult to appear. ,
The selection step includes
When the estimated amount of the first code amount estimating step is smaller than the estimated amount of the second code amount estimating step for the deterioration appearance intermediate portion, the first transform coefficient data is selected, and the first When the estimated amount of the one code amount estimating step is equal to or larger than the estimated amount of the second code amount estimating step, the second transform coefficient data is selected, and the deterioration easily appearing portion is selected. 11. The video encoding method according to claim 10, wherein the second transform coefficient data is selected, and the first transform coefficient data is selected for the portion where deterioration is difficult to occur.   12. The method according to claim 10, further comprising: a quantization step for quantizing the transform coefficient data selected in the selection step; and an entropy encoding step for entropy encoding the output data of the quantization step. The video encoding method described in 1.   The feature extraction step extracts any one of a flat part, an edge part, a target object part, a person part, a skin color part, and a low-speed moving object part of an image area as a part where deterioration appears easily. 2. The video encoding method according to item 1.

Images