JP2007104342A - Device and method for image encoding - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To encode a picture within a target code amount while its low processing load as well as superior encoding result are both realized. <P>SOLUTION: From an input picture, two or more pieces of statistical information for block unit are detected, and weighting is carried out for each statistical information to calculate a vision sensitivity level. The vision sensitivity level is classified to a vision sensitivity classes, while the input picture is separated into two or more regions, and how many high order classes of vision sensitivity classes for each region are included is counted. According to the counted value, an encoding amount is allotted to each region, and a quantize scale value is set per block unit for each region so that encoding is performed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image encoding apparatus and an image encoding method, and more particularly to an image encoding apparatus and an image encoding method suitable for encoding a moving image.

  When encoding moving image data, it is known to divide each frame (picture) of moving image data into blocks each composed of a predetermined number of pixels and perform encoding in units of the blocks. For example, in the moving picture experts group (MPEG) method, each block is subjected to orthogonal transform to be converted into a spatial frequency component, and the spatial frequency component is quantized and then entropy encoded.

  The MPEG system defines a decoding process of an encoded stream, and a certain degree of freedom remains in the encoding process. Therefore, it is possible to change the encoding parameter and the encoding algorithm within the range of the standard, and the control of the image quality and the code amount can be realized. One important technique of an algorithm for determining image quality and code amount is code amount control.

  The algorithms for realizing the code amount control can be broadly classified into two types: CBR (Constant Bit Rate) method and VBR (Variable Bit Rate) method. The CBR method is a method in which a constant code amount is always assigned, and the VBR method is a method in which codes are adaptively assigned according to the encoding difficulty level.

  In general, the MPEG method requires a larger amount of code in a picture with a high encoding difficulty level to maintain the same image quality as a picture with a low encoding difficulty level. When encoding by the CBR method, the picture quality is deteriorated in a picture having a high encoding difficulty level compared to a picture having a low encoding difficulty level. In order to prevent this, if a high bit rate is set in accordance with a picture having a high degree of difficulty in encoding, the amount of code becomes excessive in other pictures.

  On the other hand, in the VBR system, a high bit rate is assigned to a picture with a high degree of encoding difficulty, and a low bit rate is assigned to a picture with a low degree of encoding difficulty. It can be suppressed to the limit. Therefore, in general, if the amount of codes is the same, it can be said that an image encoded using the VBR method has better image quality than an image encoded using the CBR method.

  In the MPEG method, encoding processing is performed in units of blocks. Therefore, after assigning a target code amount for a picture by the CBR method or VBR method, it is necessary to assign a target code amount for each block. In addition, an image encoding device in an imaging apparatus such as a camera needs to perform real-time encoding, and processing speed is also required. For this reason, there is a technique in which a picture is divided into sub-areas including a plurality of blocks, orthogonal transform and quantization are performed in each sub-area, the number of effective coefficients is counted, and code amount control is performed based on the distribution. Yes (see, for example, Patent Document 1).

JP 2003-304402 A

  In the conventional technique described in Patent Document 1, the processing load is reduced by counting effective coefficients in units of macroblocks (MB) in each subarea and performing code amount control according to the count value. However, since the human visual characteristic is not considered in the code amount control, the image quality after encoding is not always good.

  The present invention has been made in view of the problems of the prior art, and an object of the present invention is to provide an image encoding apparatus and an image encoding capable of obtaining an encoding result having a good image quality while reducing a processing load. It is to provide a method.

  In order to achieve the above-described object, according to the present invention, in an image encoding apparatus that encodes an input picture within a given target code amount, a human is used for each of a plurality of blocks constituting the input picture. Statistical information detection means for detecting a plurality of statistical information related to visual characteristics; and visual sensitivity class classification means for classifying each of the plurality of blocks into a plurality of classes with respect to the level of visual sensitivity based on the plurality of statistical information; The input picture is divided into a plurality of first areas in one direction in the horizontal or vertical direction, and a predetermined visual sensitivity among the blocks included in the first area is high for each of the first areas. A first counting unit that counts blocks classified into classes, and a first code amount allocating unit that allocates a target code amount for each first area according to a target code amount and a counting result of the first counting unit. Each of the first regions is divided into a plurality of second regions in the other direction in the horizontal or vertical direction, and a predetermined one of the blocks included in the second region is predetermined for each second region. Second counting means for counting blocks classified into a class having high visual sensitivity, and a target code for each second area according to the target code amount for the first area and the counting result of the second counting means. A second code amount allocating unit for allocating the amount, a coding parameter calculating unit for calculating a coding parameter from the target code amount and statistical information of the second region, and encoding an input picture according to the coding parameter And an encoding means.

  Further, the above-described object is to provide a plurality of blocks related to a human visual characteristic for each of a plurality of blocks constituting an input picture in an image encoding apparatus that encodes an input picture within a given target code amount. Statistical information detecting means for detecting statistical information, visual sensitivity class classification means for classifying each of the plurality of blocks into a plurality of classes with respect to the level of visual sensitivity based on the plurality of statistical information, and the input picture horizontally or A block that is divided into a plurality of first regions in one vertical direction and is classified into a predetermined class with high visual sensitivity among the blocks included in the first region for each first region. First counting means for counting, first code amount assigning means for assigning a target code amount for each first area according to the target code amount and the counting result of the first counting means, and an input picture, The second area is divided into a plurality of second areas in the flat or vertical direction, and each of the second areas is classified into a predetermined high visual sensitivity class among the blocks included in the second area. A second counting unit that counts blocks, a second code amount assigning unit that assigns a target code amount for each second region, in accordance with the target code amount and the counting result of the second counting unit; Coding parameter calculating means for calculating a coding parameter from a target code quantity for the region, a target code amount for the second region, and statistical information; and a coding means for coding an input picture according to the coding parameter. It is also achieved by an image encoding device characterized by comprising.

  With such a configuration, according to the present invention, it is possible to obtain an encoding result having good image quality while reducing the processing load.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing a configuration example of an image encoding device according to the first embodiment of the present invention.
An input image 201 represents moving image data and is input in units of pictures. The statistical information detection unit 202 detects information such as activity, luminance, spatial frequency, skin color information, gradation, and the like in MB (macroblock) units. The visual sensitivity level calculation unit 203 weights the result of the statistical information detection unit by the visual sensitivity level table 204 and calculates the visual sensitivity level. The visual sensitivity class classification unit 205 sets a visual sensitivity class for each MB according to the visual sensitivity level.

  The Row_grp class counting unit 206 divides the picture into Mr Row_grp in the horizontal direction, and counts the number of MBs of the upper Mr classes existing in each Row_grp. The Cor_grp class counting unit 207 divides each Row_grp into Mc Cor_grp in the vertical direction, and counts the number of MBs of the upper Mc classes existing in each Cow_grp.

  The Row_grp intra-code amount allocation unit 208 allocates a corresponding code amount Tr to each Row_grp from the count result of the Row_grp intra-class counter 206 and a predetermined target code amount of the entire picture. The code amount allocation unit 209 within the Cow_grp further obtains a code amount Tc for each Cor_grp from the count result of the class counter unit 206 within Cor_grp and the code amount Tr allocated to the Row_grp by the code amount allocation unit 208 within Row_grp. Assign to each.

  The coding parameter calculation unit 209 calculates a quantization scale value for each MB from the activity, which is one detection result of the statistical information detection unit 202, and the code amount Tc for Cor_grp allocated by the Cor_grp intra-code amount allocation unit 209. . The encoding unit 211 performs orthogonal transform on the input image 201 in units of blocks and converts the input image 201 into a spatial frequency component. Then, the spatial frequency component is subjected to quantization processing based on the quantization scale value calculated by the encoding parameter calculation unit 209 and then subjected to entropy encoding.

  Next, the operation of the image coding apparatus according to the present embodiment will be described with reference to the flowchart shown in FIG.

  The input image 201 is input to the statistical information detection unit 202 and the encoding unit 211. The statistical information detection unit 202 divides the input image 201 into macro blocks (MB) of, for example, 8 × 8 pixels or 16 × 16 pixels, and human visual characteristics such as luminance, spatial frequency, skin color information, gradation, activity, etc. in MB units. Statistical information related to is detected (step S1).

  Activities include, for example, the average sum of squares of the difference between the average brightness value of each pixel in the MB and the brightness value of each pixel, the dispersion value of the brightness value, the absolute value of the average deviation of the brightness value, and the difference between adjacent pixels It can be obtained as an absolute value sum. In this embodiment, the statistical information detection unit 202 outputs luminance, spatial frequency, skin color information, and gradation to the visual sensitivity level calculation unit 203, and outputs the activity to the encoding parameter calculation unit 210.

  Next, the visual sensitivity level calculation unit 203 calculates the visual sensitivity level according to the weight of the visual sensitivity level table 104 from the plurality of input statistical information (step S2). An example of the visual sensitivity table is shown in FIG. As shown in FIG. 3, the visual sensitivity table is a table in which statistical information is associated with its weighted count value.

When the visual sensitivity table of FIG. 3 is used, the visual sensitivity level H_level is
H_level = Sh × Ch + Sy × Cy + Sf × Cf + Sg × Cg
= Sh × 0.5 + Sy × 0.5 + Sf × 0.5 + Sg × 0.5
It can be calculated by the following formula.

  The calculated H_level is output to the visual sensitivity class classification unit 205. The visual sensitivity class classification unit 205 classifies each MB into a visual sensitivity class H_class according to the visual sensitivity level H_level (step S3).

  For example, this classification is performed based on the criteria shown in FIG. In this case, when H_level ≧ 30, H_class = 1 and it is determined as the most important MB. In the present embodiment, there are four types of H_class, 1-4.

  Next, the in-Row_grp class counting unit 206 divides the picture into Mr groups (Row_grp) in the horizontal direction (step S4). Then, the number of MBs of a higher class (for example, H_class is 1 and 2) among the plurality of MBs present in each Row_grp is counted (step S5). Then, the code amount allocation unit 208 within Row_grp allocates a code amount Tr to each Row_grp from the target code amount of the entire picture and the coefficient result of the class counter 206 within Row_grp.

  Further, in the Cor_grp class counting unit 207, Row_grp is divided into Mc groups (Cor_grp) in a direction (vertical direction) 90 degrees different from the horizontal direction, and the upper multiple classes (for example, H_class is 1) existing in each Cor_grp. And 2) count the number of MBs. In the Cor_grp intra-code amount allocation unit 209, as in the Row_grp intra-code amount allocation unit 208, the code amount Tc is calculated and allocated to each Cor_grp from the coefficient results of the Row_grp intra-code amount Tr and the Cor_grp intra-class counting unit 207. (Step S7). The code amount Tc is output to the encoding parameter calculation unit 210.

The above operation will be described using a specific relationship with a picture.
FIG. 5 is a diagram for explaining the relationship between group division and code amount allocation in the image coding apparatus according to the present embodiment.
In FIG. 5, the picture is divided horizontally into five Row_grp (horizontal direction), and each Row_grp is divided into seven Cor_grp vertically long (vertical direction). The visual characteristic class (H_class) is set in MB units. In the drawing, a part of visual characteristic classes 1 to 4 corresponding to H_class 1 to 4 is shown.

  FIG. 6 shows an example of code amount allocation. In FIG. 6, the code amount is T1> T2> T3> T4, and a larger amount of code is assigned to Cor_grp that includes more upper class MBs. Specifically, numerical values corresponding to H_class are integrated, and the code amount Tc is determined stepwise according to the integration result for each Cor_grp.

  The encoding parameter calculation unit 210 equally distributes the code amount to each MB included in the Cor_grp from the activity and the code amount Tc input from the statistical information detection unit 202 (step S8). Then, a quantization scale value corresponding to the code amount is calculated for each MB based on, for example, the MPEG TM5 (Test Mode 5) method (step S9). The encoding unit 211 performs orthogonal transformation on each MB of the input image 201 to convert it to a spatial frequency component, and then quantizes the spatial frequency component based on the quantization scale value input from the encoding parameter calculation unit 210. Then, entropy encoding is performed on the quantization result, and the result is output as an encoded stream.

  FIG. 7 is a diagram illustrating a code amount transition of one picture in the image coding apparatus according to the present embodiment. In the case of equal distribution, a certain code amount increases for each Row_grp. However, according to the code amount allocation according to the present invention, not only is the degree of encoding difficult, but it is also determined to be important from human visual characteristics. A large amount of code is allocated to the block, and a small amount of code is allocated to the block that is not. Therefore, it can be seen that each Row_grp has a different code amount.

  As described above, according to the present embodiment, by performing code amount control in consideration of human visual characteristics, highly accurate code amount control can be realized, and image quality can be improved. Also, instead of assigning a code amount in units of blocks, a picture is divided into a plurality of regions each including a plurality of blocks, and a code amount is assigned to each divided region, and then a code amount is assigned to each block. Assign. Therefore, the processing load is small and high-speed processing can be realized.

(Second Embodiment)
FIG. 8 is a block diagram showing a configuration example of an image encoding device according to the second embodiment of the present invention. FIG. 9 is a flowchart for explaining the operation of the image coding apparatus according to the present embodiment. 8 and 9, the same reference numerals are assigned to the same components as those in the first embodiment, and the description thereof is omitted.

Hereinafter, a configuration different from the first embodiment will be described.
In the present embodiment, as shown in FIG. 10, in the Row_grp class counter 306 and the Cor_grp2 class counter 307, the pictures are independently divided into Row_grp and Cor_grp2, respectively.

  Cor_grp2 is obtained by dividing the picture vertically (in the vertical direction) into Mc groups in the Cor_grp2 class counter 307. That is, in the example of FIG. 10, one Cor_grp2 is composed of 5 MB. Then, the number of MBs of higher-order multiple classes (for example, H_class is 1 and 2) existing in each Cow_grp2 is counted and output to the Cor_grp2 intra-code amount allocation unit 309. The Cor_grp2 intra-code amount assigning unit 309 assigns the code amount Tc for each Cor_grp2 from the target code amount of the entire picture and the coefficient result of the Cor_grp2 intra-class counting unit 307 (step S7 ').

  The encoding parameter calculation unit 310 receives the activity, which is one detection result of the statistical information detection unit 202, the code amount Tr allocated to Row_grp by the intra-row_grp code amount allocation unit 208 in step S6, and the above Tc. . Based on these input values, the code amount for each MB is determined (step S8 '). Then, a quantization scale value corresponding to the code amount is calculated and output to the encoding unit 311 (step S9).

  Since each MB belongs to each of Row_grp and Cor_grp, the code amount is determined using Tr of the Row_grp to which the MB belongs, Tc of the Cor_grp, and the activity. For example, if the activity is constant, the code amount for MBs belonging to Row_grp and Cor_grp with both Tr and Tc being highest is the largest, and the code amount for MBs belonging to Row_grp and Cor_grp being low for both Tr and Tc is the smallest. A decision is made.

  As described above, according to the present embodiment, a code amount is assigned to a region that is larger in the vertical direction than in the first embodiment. By assigning the code amount of the area divided in the horizontal direction and the area divided in the vertical direction in parallel and then calculating the encoding parameter for each block, the processing speed is higher than that of the image encoding apparatus of the first embodiment. I can expect.

(Other embodiments)
In the above-described embodiment, the number of blocks classified into a plurality of upper classes in the group is counted. However, the number of blocks classified into at least the highest class may be counted.

  In addition, a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly or to a system or apparatus having a computer that can execute the program using wired / wireless communication. The present invention includes a case where an equivalent function is achieved by a computer executing the supplied program.

  Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, a magnetic recording medium such as a flexible disk, a hard disk, a magnetic tape, MO, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-R, DVD- There are optical / magneto-optical storage media such as RW, and non-volatile semiconductor memory.

  As a program supply method using wired / wireless communication, a computer program forming the present invention on a server on a computer network, or a computer forming the present invention on a client computer such as a compressed file including an automatic installation function A method of storing a data file (program data file) that can be a program and downloading the program data file to a connected client computer can be used. In this case, the program data file can be divided into a plurality of segment files, and the segment files can be arranged on different servers.

  That is, the present invention includes a server device that allows a plurality of users to download a program data file for realizing the functional processing of the present invention on a computer.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to the user, and key information for decrypting the encryption for a user who satisfies a predetermined condition is provided via a homepage via the Internet, for example. It is also possible to realize the program by downloading it from the computer and executing the encrypted program using the key information and installing it on the computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU of the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

It is a block diagram which shows the structural example of the image coding apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart explaining operation | movement of the image coding apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of a visual sensitivity level table. It is a figure which shows the example of a visual sensitivity class classification | category. It is a figure for demonstrating the relationship between the group division | segmentation and code amount allocation in the image coding apparatus of 1st Embodiment. It is a figure which shows the example of code amount allocation in the image coding apparatus of 1st Embodiment. It is a figure which shows the code amount transition of 1 picture in the image coding apparatus of 1st Embodiment. It is a block diagram which shows the structural example of the image coding apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the image coding apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the area | region division example in 2nd Embodiment.

Claims (9)

In an image encoding device that encodes an input picture within a given target code amount,
Statistical information detecting means for detecting a plurality of statistical information related to human visual characteristics for each of a plurality of blocks constituting the input picture;
Visual sensitivity class classification means for classifying each of the plurality of blocks into a plurality of classes with respect to the level of visual sensitivity based on the plurality of statistical information;
The input picture is divided into a plurality of first areas in one direction in the horizontal or vertical direction, and for each of the first areas, a predetermined visual sensitivity of the blocks included in the first area is determined. A first counting means for counting blocks classified into a higher class;
First code amount assigning means for assigning a target code amount for each of the first areas according to the target code amount and the counting result of the first counting means;
Each of the first regions is divided into a plurality of second regions in other directions in the horizontal or vertical direction, and predetermined among the blocks included in the second region for each second region. A second counting means for counting blocks classified into a class having high visual sensitivity;
Second code amount assigning means for assigning a target code amount for each of the second regions according to a target code amount for the first region and a counting result of the second counting means;
Coding parameter calculating means for calculating a coding parameter from the target code amount of the second region and the statistical information;
An image coding apparatus comprising: coding means for coding an input picture according to the coding parameter.   The encoding parameter calculation means assigns a target code amount for the second region equally to a plurality of blocks included in the second region, and the assigned code amount and the statistical information for each block The image encoding apparatus according to claim 1, wherein the encoding parameter is calculated from In an image encoding device that encodes an input picture within a given target code amount,
Statistical information detecting means for detecting a plurality of statistical information related to human visual characteristics for each of a plurality of blocks constituting the input picture;
Visual sensitivity class classification means for classifying each of the plurality of blocks into a plurality of classes with respect to the level of visual sensitivity based on the plurality of statistical information;
The input picture is divided into a plurality of first areas in one direction in the horizontal or vertical direction, and for each of the first areas, a predetermined visual sensitivity of the blocks included in the first area is determined. A first counting means for counting blocks classified into a higher class;
First code amount assigning means for assigning a target code amount for each of the first areas according to the target code amount and the counting result of the first counting means;
The input picture is divided into a plurality of second areas in other directions in the horizontal or vertical direction, and for each second area, a predetermined visual sensitivity of the blocks included in the second area is determined. A second counting means for counting blocks classified into a higher class;
Second code amount assigning means for assigning a target code amount for each second region in accordance with the target code amount and the counting result of the second counting means;
Encoding parameter calculation means for calculating an encoding parameter from the target code amount for the first region, the target code amount for the second region, and the statistical information;
An image coding apparatus comprising: coding means for coding an input picture according to the coding parameter.   The encoding parameter calculation means allocates a code amount for each block based on a target code amount for the first and second regions, and the code amount is allocated from the allocated code amount and the statistical information for each block. 4. The image encoding apparatus according to claim 3, wherein an encoding parameter is calculated.   The image coding apparatus according to any one of claims 1 to 4, wherein the coding parameter calculation unit uses an activity for each block as the statistical information.   The image coding apparatus according to claim 1, wherein the coding parameter is a quantization scale. An image encoding method for encoding an input picture within a given target code amount,
A statistical information detecting step for detecting a plurality of statistical information related to human visual characteristics for each of a plurality of blocks constituting the input picture;
Visual sensitivity class classification step for classifying each of the plurality of blocks into a plurality of classes with respect to high or low visual sensitivity based on the plurality of statistical information;
The input picture is divided into a plurality of first areas in one direction in the horizontal or vertical direction, and a predetermined visual sensitivity of the blocks included in the first area is determined for each first area. A first counting step for counting blocks classified into a higher class;
A first code amount assigning step for assigning a target code amount for each of the first regions in accordance with the target code amount and the counting result of the first counting step;
Each of the first regions is divided into a plurality of second regions in other directions in the horizontal or vertical direction, and predetermined among the blocks included in the second region for each second region. A second counting step for counting blocks classified into a class having high visual sensitivity;
A second code amount assigning step for assigning a target code amount for each second region in accordance with a target code amount for the first region and a counting result of the second counting step;
An encoding parameter calculating step for calculating an encoding parameter from the target code amount of the second region and the statistical information;
And a coding step of coding an input picture in accordance with the coding parameter. An image encoding method for encoding an input picture within a given target code amount,
A statistical information detecting step for detecting a plurality of statistical information related to human visual characteristics for each of a plurality of blocks constituting the input picture;
Visual sensitivity class classification step for classifying each of the plurality of blocks into a plurality of classes with respect to high or low visual sensitivity based on the plurality of statistical information;
The input picture is divided into a plurality of first areas in one direction in the horizontal or vertical direction, and for each of the first areas, a predetermined visual sensitivity of the blocks included in the first area is determined. A first counting step for counting blocks classified into a higher class;
A first code amount assigning step for assigning a target code amount for each of the first regions according to the target code amount and the counting result of the first counting step;
The input picture is divided into a plurality of second areas in other directions in the horizontal or vertical direction, and for each second area, a predetermined visual sensitivity of the blocks included in the second area is determined. A second counting step for counting blocks classified into a higher class;
A second code amount assigning step for assigning a target code amount for each of the second regions in accordance with the target code amount and the counting result of the second counting step;
An encoding parameter calculating step for calculating an encoding parameter from the target code amount for the first region, the target code amount for the second region, and the statistical information;
An image encoding method comprising: an encoding step of encoding an input picture in accordance with the encoding parameter.   The program which makes a computer implement | achieve the image coding method of Claim 7 or Claim 8.