JP2006270390A - Image encoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding device for realizing the attainment of an improvement in the quality of an image by the enhancement of a motion-vector encoding efficiency at a small arithmetic cost in the image encoding device dividing the image at block units and conducting an in-frame motion compensation utilizing a motion vector. <P>SOLUTION: When the quantity of codes generated at the time of a block processing surpasses the target quantity of the codes, an image block processed immediately before the division of the image at block units in the order of a block scanning and the attribute of a block are compared. When the attributes coincide, the result of the search of the motion vector is used as it is as the motion vector of the block. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image encoding apparatus that encodes a moving image.

  Recording / reproducing and transmitting captured moving images as digital data has become widespread. At this time, it is common to handle the moving image by performing compression coding processing from the viewpoint of reducing the amount of data. As a moving image compression coding method, for example, as in the MPEG method, orthogonal transform coding using a spatial direction (within a frame) correlation and motion compensated prediction coding using a time direction correlation (between frames). Both are often used.

  Motion compensation predictive coding used in MPEG divides each frame into block units, performs block matching between the previous frame and the current frame, or the current frame and the next frame, and detects each block of the current frame. The motion vector and block matching error are encoded.

  Here, regarding the coding of motion vectors, using the fact that the correlation of “motion” between adjacent blocks is high, coding is performed not on the motion vector itself but on the difference from the previous motion vector. .

  For example, in FIG. 2, when 101 is a past frame and 111 is a current frame, the blocks 103 and 104 constituting the shooting target (object) 102 included in the past frame 101 are moved shootings in the current frame 111, respectively. It is considered that the object (object) 112 has moved to the constituent blocks 113 and 114, and the motion vectors are 116 and 117, respectively, and the two motion vectors can be regarded as the same.

  In this case, when each of the blocks 113 and 114 is encoded, the motion vector of the block 113 (the difference value from the motion vector of the previous block) is first encoded, and then the block error with the block 103 is encoded. For the block 114 that follows, the motion vectors are equal, so that a difference value of 0 and a block error with the block 104 are encoded. Here, regarding the coding of the motion vector difference value, variable length coding (VLC) is performed in which short code words are assigned in descending order of occurrence frequency, and short codes are assigned in ascending order of vector difference.

  Regarding the motion vector of the block 114, the motion vector search result becomes the motion vector 118 depending on the conditions at the time of the search process, that is, the prediction error with the block 105 may be smaller. However, in this case, since the difference from the motion vector of the immediately preceding block 113 is not 0, the code amount in differential vector encoding increases. In this way, by searching for a motion vector only by simple block matching, a motion vector that opposes the motion of the entire object to be imaged is generated, which may cause an increase in the amount of code when encoding the difference vector.

  As described above, the motion vector encoding efficiency is improved as the motion vector is closer to the immediately preceding block in the block scan order. However, there is no guarantee that the motion vectors between adjacent blocks are necessarily correlated, and since motion vector coding is prioritized, block errors are large, that is, if a motion vector that is not appropriate as a position for motion prediction is selected, the entire coding is performed. Since efficiency deteriorates, it is important to balance both block error and difference vector. Further, when considering processing in a portable device such as a digital camera, simple processing (with a small number of computations) in consideration of restrictions such as circuit scale, operating frequency, and power consumption is desirable.

  The improvement in coding efficiency means that, when considered at the same bit rate, the floating bits can be allocated to other blocks with a high degree of difficulty in coding without causing a reduction in image quality due to the improvement in coding efficiency. This means that high image quality can be achieved for the entire moving image.

  In the “motion vector correction control method” disclosed in Patent Document 1, as a technique for performing an overall optimal motion vector search process in both motion vector and block error encoding, first, a normal motion vector search process is performed. If the result is extremely different from the surrounding block and the color block is flat, it is determined that a motion vector is erroneously detected because there are many flat blocks, and the motion vector search result of the surrounding block Apply as it is.

In addition, in the “motion vector detection method and circuit” disclosed in Patent Document 2, if there is a prediction error that is the smallest and less than a predetermined value among the motion vectors of adjacent blocks, the motion vector of the block is used as it is. If the prediction error exceeds the predetermined value, the motion vector search range is determined from the direction and speed of the motion vector obtained in the adjacent block, and the motion vector search is performed. This is a motion vector search method aiming at improving the hit rate in motion vector search using the correlation of motion vectors between adjacent blocks.
Japanese Patent No. 3418799 Japanese Unexamined Patent Publication No. 2000-201328

  However, in Patent Document 1, although it is a process of reducing the “variation” of the vector by aligning with the motion vectors of the peripheral blocks, it is basically a correction process specialized for a flat color image, For each process, it is necessary to perform a motion vector comparison with a peripheral block and a block flatness determination process. In Patent Document 2, it is necessary to perform a motion vector comparison process for adjacent blocks every time.

  That is, in the prior art, the processing load when searching for a motion vector is very large.

  In order to solve the above-described problems, the present invention provides an image coding apparatus for realizing high image quality by improving motion vector coding efficiency at a low calculation cost.

  An image encoding apparatus according to the present invention detects an attribute of a block and outputs an attribute parameter in an image encoding apparatus that performs inter-frame motion compensation using a motion vector by dividing an image into blocks. A block, a MV search processing unit that performs motion vector search of the block and outputs a motion vector, a memory that holds the attribute parameter and the motion vector of the frame, and monitors a generated code amount up to the encoding point of the block And an attribute parameter of a code amount monitor that outputs a comparison result with a target code amount set in advance and a neighboring block whose generated code amount exceeds the target code amount and whose block scan order is one or more before. If they match, the motion vector of the neighboring block held in the memory is used. An MV encoding process selection unit that outputs a motion vector obtained by the search processing unit; and an encoding unit that performs inter-frame encoding processing using the motion vector selected by the encoding process selection unit. And

  According to the first aspect of the present invention, when the generated code amount at the time of the block processing exceeds the target code amount, the attribute of the block is compared with the image block processed immediately before in the block scan order. If the attributes match, the motion vector search result is used as it is as the motion vector of the block, replacing the motion vector without performing block matching processing for motion vector re-search, and improving the encoding efficiency I can do it.

  In addition, according to the second to fourth aspects of the present invention, by using the color characteristics of each block as an attribute comparison index between blocks, the attribute parameter for each block can be calculated without using a complicated determination process.

  According to the invention described in claim 5 of the present application, it is possible to reduce the circuit scale by performing the block attribute comparison using the skin color detector shared with other parts of the encoding process.

  According to the sixth aspect of the present invention, the coding distortion value is calculated when the motion vector result of the image block processed immediately before in the block scan order is used for the block as it is, and compared with a predetermined reference value. In addition, by determining whether or not to perform motion vector replacement processing, it is possible to prevent the case where the block error increases and the code amount as a whole increases when the previous block and the motion vector do not match.

  Further, according to the invention described in claim 7 of the present application, when calculating the coding distortion value which is an index as to whether or not the motion vector search result of the immediately preceding block is used in the block scan order, the difference value between the blocks is distributed. By using a distributed arithmetic unit that is shared with other parts of the encoding process, the circuit scale can be reduced.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to examples.

  FIG. 1 is a diagram of an image encoding apparatus according to the present invention, which best represents the features of the present invention.

  In FIG. 1, a motion vector searcher 200 includes a block attribute detection unit 201, a block attribute / MV search result storage memory 202, an MV encoding process selection unit 203, and an MV search processing unit 204. The pixel data is read in units, the block attribute detection unit 201 detects the attribute of the processing target image block, and writes it in the block attribute / MV search result storage memory 202. The attribute parameter of the written target block is the neighborhood of the block processing determination unit 203 when the current code amount exceeds the target value after the normal MV search processing is performed in the MV search processing unit. It is read together with the attribute parameter of the block, and it is determined whether or not it has the same attribute as the neighboring block based on a preset criterion.

  Based on the determination result of the block attribute, the MV search processing unit 204 uses the motion vector search result of the neighboring block as it is when the same attribute as that of the neighboring block is used as it is from the block attribute / MV search result storage memory 202. The motion vector search result is read and output as it is. When the attribute is not the same as that of the neighboring block, the motion vector search processing result of the block is output as it is.

  The final motion vector search processing result of the block is written in the block attribute / MV search result storage memory 202 and then encoded by the encoder 230 in units of blocks. At that time, the code amount of the entire frame up to the block is counted by the code amount monitor 220 and used for the next processing of the block processing determination unit 203.

(Embodiment 1)
FIG. 3 is a flowchart of the motion vector searcher according to the first embodiment of the present invention. In each frame, blocks are selected one by one in the order of block scanning, and after performing motion vector search processing, encoding is performed in units of blocks.

  First, in process 301, a motion vector search process that is generally used, that is, a block matching process with a reference frame is performed to obtain a candidate motion vector, and then an attribute parameter of the block is calculated in process 302. . In this embodiment, color information is used as a block attribute parameter. If the color information matches that of the adjacent block, that is, if the color is the same as that of the adjacent block, both are considered to be the same attribute block, that is, the same object and moving in the same direction.

  In FIG. 2, it is determined that the two blocks belong to the same object 112 and move with the same motion vector when the color information which is an attribute of the blocks 114 and 113 matches.

  In this embodiment, color difference components Cb and Cr of each pixel in the block are used as color information.

  When Cb and Cr are taken as axes in FIG. 5, if each component takes a value within a range 504 separated from the reference color 501 by the threshold values Thcb and Thcr, the same attribute is obtained. To do.

  That is, in FIG. 5, the point 502 has the same attribute as the point 501, but the point 503 has a different attribute. In this embodiment, average values AvgCb and AvgCr of the color difference components of the pixels in each block are obtained, and if the average color difference component value of the reference frame falls within the range from the color difference component average value of the frame to the threshold values Thcb and Thcr, both blocks. Are considered to have the same attribute.

  In each block, the code amount up to the time point when each block is encoded is accumulated in processing 307 described later. In branch 303, it is determined whether the code amount of the previous block is larger or smaller than the target code amount obtained from a predetermined target bit rate.

  If the target code amount is exceeded, the process proceeds to branch 304; otherwise, the process skips 304 and 305 and proceeds to process 306. In the branch 304, it is determined from the block attribute calculation result in the process 302 whether or not the previous block in the block scan order has the same attribute, that is, the same color in this embodiment, and if it is the same, the process proceeds to the process 305. If 305 is skipped, the process proceeds to process 306. In process 305, the result of the normal motion vector search process obtained in process 301 is replaced with the motion vector search result of the immediately preceding block regarded as the same attribute. In process 306, based on the selected motion vector search result, a difference between the block indicated by the motion vector in the reference frame and the block is obtained, and then subjected to DCT operation, quantization, and variable-length coding. Perform the encoding process. After obtaining the code amount up to the current block of the current frame in the process 307, if there is no unprocessed block in the frame, the encoding of the current frame is terminated and the process proceeds to the next frame. Block is selected and the process returns to the process 301.

  In the encoding process of the frame 111 in FIG. 2, the block 113 performs the encoding process with the motion vector 116, and then moves to the block 114. If the result of the normal motion vector search process is the motion vector 118, the motion vector difference However, if the code amount at the time of processing exceeds the target value, the blocks 113 and 114 are based on the respective block attributes (color information). Are determined to have the same attribute, the motion vector search result of block 113 is used as it is. That is, in the process in block 114, the motion vector 118 that is the normal motion vector search result is replaced with the motion vector 117 that is the motion vector search result of the immediately preceding block, and the encoding process is performed.

  When the code amount exceeds the target value, when performing the processing from the motion vector search process again for the purpose of reducing the code amount of motion vector coding, block matching processing (each block between blocks) It is necessary to check whether or not the code amount has been reduced again after performing the difference calculation at the pixel and obtaining the minimum value) and performing the encoding. On the other hand, in this embodiment, attribute determination between blocks, which is a simple calculation process for obtaining an average value of Cb and Cr of each pixel of a block constituted by n × n, for example, calculation of color information is performed. Basically, the result of the motion vector is simply used as it is, and no other operation is required.

  As described above, the coding efficiency of the motion vector can be increased with a small amount of calculation.

(Embodiment 2)
In FIG. 2, assuming that the object 112 indicates a face portion of a person and has a uniform skin color, the determination of the same attribute of the blocks 113 and 114 included therein is performed by determining the skin color component in the block. Is possible by comparing

  In addition, the skin color is easily noticeable by humans, and the compression coding “rough” is easily noticeable, so special processing such as changing the degree of filtering of the skin color part of the image or increasing the amount of code to be assigned is added. For this purpose, there exists an image encoding device provided with a skin color detection device. In this embodiment, this skin color detecting device is used for attribute comparison between blocks, that is, if the skin color, it is regarded as the same attribute and the coding process is performed for each block.

  Hereinafter, the processing flow is the same as that of the first embodiment, and therefore, the description thereof is omitted. The circuit scale can be reduced by combining the skin color detection device used for other processing with the block attribute calculation processing of the present embodiment.

(Embodiment 3)
FIG. 4 is a flowchart of the motion vector searcher according to the third embodiment of the present invention. Processes 401 to 405 are the same as processes 301 to 305 of the first embodiment, respectively, and a description thereof will be omitted.

  When the code amount up to the immediately preceding block exceeds the target value and has the same attribute as that of the immediately preceding block, in step 406, the motion vector search result of the immediately preceding block is used to obtain a block matching error with the reference frame, and the variance thereof Find the value.

  Here, using the equation of the classic Rate-Distortion model, equation (1) is established when the assigned code amount is R, the quantization distortion amount is D, and the variance of the input value is S.

  From (1), the amount of quantization distortion is expressed by equation (2).

  Approximate the amount of quantization distortion using equation (2), determine the encoding result when using the motion vector search result of the immediately preceding block, and then use the search result of the immediately preceding block or It is determined whether to use the search result.

  For example, if the expression (1) calculated using the motion vector search result of the immediately preceding block is equal to or greater than a predetermined value, the distortion of the encoded image is increased by using the motion vector, that is, the same image quality. In this case, the code amount increases, and it is determined that the motion vector search result of the block should be used as it is based on the use of the result of the immediately preceding block.

  As described above, by calculating the variance of the block difference values, the result of actual encoding can be predicted in advance, so that the accuracy of motion vector selection can be improved.

  In addition, since the calculation process of variance is often used for code amount control, the circuit scale can be reduced by sharing the arithmetic circuit for distribution.

The figure showing an example of a structure of the motion vector searcher of the image coding apparatus in this application The figure showing an example of the motion vector search process of the image coding apparatus in this application The figure showing an example of the flowchart of the motion vector search process of Embodiment 1, 2 of this application The figure showing an example of the flowchart of the motion vector search process of Embodiment 3 of this application. The figure showing description of attribute determination by the color difference component of Embodiment 1 of this application

Explanation of symbols

200 motion vector search unit 201 block attribute detection unit 202 block attribute / MV search result memory 203 MV encoding process selection unit 204 MV search processing unit 210 frame memory 220 code amount monitor 230 encoder

Claims (7)

In an image encoding device that divides an image into blocks and performs inter-frame motion compensation using a motion vector,
A block attribute detection unit that detects an attribute of the block and outputs an attribute parameter;
An MV search processing unit that performs a motion vector search of the block and outputs a motion vector;
A memory for storing attribute parameters and motion vectors of the frame;
A code amount monitor that monitors the generated code amount up to the encoding point of the block and outputs a comparison result with a target code amount set in advance;
When the generated code amount exceeds the target code amount and the attribute parameters of the neighboring blocks whose block scanning order is one or more before match, the motion vector of the neighboring block held in the memory is Otherwise, an MV encoding process selection unit that outputs a motion vector obtained by the MV search processing unit,
An image encoding apparatus comprising: an encoding unit that performs inter-frame encoding processing using the motion vector selected by the encoding process selection unit.   The image encoding apparatus according to claim 1, wherein the color characteristic of each block is used as the attribute parameter of the block.   The image coding apparatus according to claim 1 or 2, wherein a color difference component of each pixel in the block is used as a block attribute parameter comparison method.   As a block attribute parameter comparison method, the average values AvgCb and AvgCr of the color difference components Cb and Cr of each pixel in the block are calculated for each block, and if the difference between the average values is within a reference value, the block and The image coding apparatus according to claim 3, wherein neighboring blocks are regarded as having the same attribute.   As a block attribute parameter comparison method, a skin color detector is provided, skin color detection processing is performed for each pixel in the block, and when both the processing target block and the comparison target block have skin color pixels of a reference number or more, The image coding apparatus according to claim 1, wherein neighboring blocks are regarded as having the same attribute.   In the MV encoding process selection unit, when a neighboring block and the block have the same attribute, an encoding distortion value when encoding is performed using a motion vector search result of the neighboring block, and the encoding distortion value is determined in advance. 6. The image coding apparatus according to claim 1, wherein when the value is smaller than the value, the motion vector detection result in the neighboring block is used as it is as the motion vector of the block.   The image encoding apparatus according to claim 6, wherein, as a method of calculating an encoding distortion value, variance in a difference value between a pixel sample of a prediction block obtained from a motion vector and a pixel sample of the block is used. .