JP2006191175A - Image coding apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion vector arithmetic apparatus suppressing a computation complexity required for motion compensation prediction even when there are many reference candidate frame pictures. <P>SOLUTION: A motion vector detection section 111 of an image coding apparatus for carrying out arithmetic processing to apply motion compensation prediction to an image coding object image includes: a reference candidate image storage section 1120 for storing a plurality of reference candidate images being candidates of referenced images in the arithmetic processing; a reduced image generating section 1113 for generating reduced images of each reference candidate image and the image coding object image; a correlation comparison section 1111 for calculating the correlation between the reduced image of each reference candidate image and the reduced image of the image coding object image; a motion vector control section 1119 for selecting the reference candidate image being an candidate of the arithmetic processing among the reference candidate images depending on a degree of the correlation; and a motion vector arithmetic section 1116 for applying the arithmetic processing to the selected reference candidate image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image coding apparatus, and more particularly to a technique for reducing the amount of calculation for performing motion compensation prediction.

Conventionally, a technique called motion compensation prediction has been used as a technique for compressing and encoding a large amount of digital image data.
Motion-compensated prediction is to detect a similar part and a motion vector by dividing a frame image to be encoded into small blocks and searching the frame image before and after each block, and the detected motion vector and detection This is a technique for efficiently compressing and encoding image data by encoding a difference value of pixel data with the similar part.

In the MPEG2 and MPEG4 standards, one or two frame images are searched for (hereinafter referred to as “reference candidate frame images”). For example, as shown in FIG. When is a P picture, the frame image immediately before the encoding target image is set as the reference candidate frame image.
In addition, in the MPEG4AVC (Advanced Video Coding, ITU-T H.264) standard recently standardized, as shown in FIG. 9B, three or more (up to 16) reference candidate frame images are searched. Furthermore, as shown in 1 to 8 of FIG. 10, the block size that is the pixel block to be searched is also searched for more types of block sizes than in the MPEG2 and MPEG4 standards. can do.

As described above, in the MPEG4AVC standard, by increasing the types of reference candidate frame images that can be searched and the sizes of blocks to be searched, the detection rate of similar parts in motion vector prediction can be increased, and a large-capacity digital image can be obtained. Data compression efficiency can be increased.
On the other hand, in order to perform motion compensation prediction, an enormous amount of computation is required. Therefore, as a conventional technique for suppressing this computation amount, in a block to be searched, the number of pixels to be computed is thinned out for each block. A technique for reducing the amount of calculation required for detecting similar parts is disclosed in Japanese Patent Application Laid-Open No. 2004-133830.
Patent-3269031

However, the above-described conventional technique can be expected to be effective in suppressing the amount of calculation for motion compensation prediction in the MPEG2 and MPEG4 standards with a small number of reference candidate frame images, but in the MPEG4AVC standard with a large number of reference candidate frame images. There is a problem that it cannot sufficiently cope with the suppression of the amount of calculation for motion compensation prediction.
Therefore, an object of the present invention is to provide a motion vector calculation device capable of suppressing the amount of calculation for motion compensation prediction even when the number of reference candidate frame images is large.

In order to solve the above-described problem, the present invention is a motion vector arithmetic device that performs arithmetic processing for performing motion compensation prediction on an image encoding target image, and is a candidate for an image to be referred to in the arithmetic processing. A reference candidate image to be subjected to the arithmetic processing is selected according to a degree of correlation between the storage means storing a plurality of reference candidate images and each of the reference candidate images and the image encoding target image. Selection means for selecting, and arithmetic means for performing the arithmetic processing on the selected reference candidate image are provided.

  Further, the motion vector calculation device includes a reduced image generation unit that generates a reduced image of each reference candidate image and a reduced image of the image encoding target image, and the image code for the reduced image of each reference candidate image. Calculating means for calculating a correlation degree indicating a degree of correlation between the reduction target image and the reduced image, and the selecting means selects the reference candidate image from the reference candidate images according to the degree of correlation indicated by the correlation degree. It is also possible to select a reference candidate image to be subject to arithmetic processing.

By providing the above configuration, the present invention provides the number of reference candidate frame images to be calculated in advance even when the number of reference candidate frame images used in the calculation processing is large in the calculation processing for performing motion compensation prediction. Since it can narrow down, the amount of calculations required for motion compensation prediction can be suppressed.
Here, the calculation means is configured to reduce the reduced image of each reference candidate image and the encoding target based on the pixel values of the pixels constituting the reduced image of each reference candidate image and the reduced image of the encoding target image. The degree of correlation between the image and the reduced image may be calculated.

As a result, the degree of correlation is calculated based on the pixel value of the reduced image of each reference candidate frame image, so that it is possible to preferentially select a reference candidate frame image whose pixel value approximates the encoding target frame image. Therefore, it is possible to perform motion compensation prediction with almost the same accuracy as when the reference candidate frame image is not narrowed down while suppressing the calculation amount.
Here, the motion vector calculation device further includes a detection unit that detects a remaining battery level, and the selection unit includes the selected reference candidate image when the detected remaining battery level is smaller than a threshold value. The specific reference candidate image may be reselected, and the calculation unit may perform the calculation process on the reselected reference candidate image.

As a result, when the remaining battery level is low, the number of reference candidate frame images to be subjected to the calculation process is narrowed down compared to the case where the remaining battery level is present, so the power consumption required for the calculation process can be reduced. The calculation process can be continued for a long time by effectively using less power.
Here, the calculation process is performed on the selected reference candidate image in units of a plurality of types of pixel blocks, and the motion vector calculation device further includes a detection unit that detects a remaining battery level, and the calculation The means may suppress the execution of the arithmetic processing for some types of pixel block units among the plurality of types when the detected remaining battery level is smaller than a threshold value.

As a result, even when the arithmetic processing is performed in units of a plurality of types of pixel blocks, the types of pixel blocks to be subject to arithmetic processing can be limited according to the remaining battery level, so that it is necessary for the arithmetic processing. The power consumption can be reduced, and the arithmetic processing can be continued for a long time by effectively using the small amount of power.
Here, the motion vector calculation device includes a reduced image storage unit that stores a reduced image of each reference candidate image generated by the reduced image generation unit, and the reduced image generation unit stores the reduced image of each reference candidate image. Determining means for determining whether or not a reduced image is stored in the reduced image storage means; and when a reduced image of each reference candidate image is stored in the reduced image storage means, the reference candidate image is reduced. It is good also as having the suppression means which suppresses reproduction | regeneration of an image.

  Thereby, since the reduced image is not regenerated for the reduced image of the reference candidate frame image that has already been stored, it is possible to suppress the execution of the extra calculation processing required for generating the reduced image.

Embodiments of the present invention will be specifically described below with reference to the drawings.
[Embodiment 1]
FIG. 1 is a functional block diagram illustrating a configuration of an image encoding device 100 according to the first embodiment.
The image coding apparatus 100 includes a subtracter 11, an adder 12, a frame memory unit 101, an orthogonal transform unit 102, a quantization unit 103, an entropy coding unit 104, an inverse quantization unit 105, an inverse orthogonal transform unit 106, a deblocking The filter unit 107, the prediction memory unit 108, the intra prediction unit 109, the inter prediction unit 110, and the motion vector detection unit 111 are configured.

The frame memory unit 101 is a pre-processing unit (not shown), in which the image order for performing motion compensation processing is changed, and is converted from raster scan format image data in units of horizontal scanning lines to block scan format image data, The input frame image of the moving image to be encoded (hereinafter referred to as “encoding target frame image”) is stored.
The subtractor 11 is a predetermined pixel unit (for example, 16 pixels × 16 pixels) that is a unit of encoding from the encoding target frame image stored in the frame memory unit 101 (hereinafter referred to as “encoding target pixel block”). And the difference between the extracted encoding target pixel block and the prediction pixel block described later input from the inter prediction unit 110 is calculated, and the calculation result is output to the orthogonal transform unit 102 as difference image data. To do.

The orthogonal transform unit 102 performs discrete cosine transform on the difference image data input from the subtractor 11 and outputs the result to the quantization unit 103.
The quantization unit 103 quantizes the difference image data input from the orthogonal transform unit 102 and outputs the quantized image data to the entropy encoding unit 104 and the inverse quantization unit 105.
The entropy encoding unit 104 encodes the difference image data input from the quantization unit 103 and the motion vector input from the motion vector detection unit 111 as a pair.

The inverse quantization unit 105 and the inverse orthogonal transform unit 106 decode the difference image data input from the quantization unit 103 into the original difference image data.
The adder 12 adds the difference image data input from the inverse orthogonal transform unit 106 and the prediction pixel block input from the inter prediction unit 110, and adds the encoding target pixel block ( Hereinafter, the “previous encoding target pixel block” is regenerated.

The deblocking filter unit 107 performs a deblocking filter process on a part of the encoding target pixel block generated by the adder 12 and outputs the result to the prediction memory unit 108.
Here, the “deblocking filter process” refers to a filter process for reducing block distortion that occurs when the encoding target pixel block is encoded.
The prediction memory unit 108 stores pixel blocks constituting a reference candidate frame image, which are targets for motion compensation prediction.

Specifically, the immediately preceding encoding target pixel block sequentially regenerated by the adder 12 is stored. As a result, the immediately previous encoding target pixel block, which is a constituent element of the reference candidate frame image, is sequentially stored in the prediction memory unit 108. As a result, the reference candidate frame image is sequentially stored in the prediction memory unit 108. Will be.
The inter prediction unit 110 reads the prediction pixel block detected by the motion vector detection unit 111 from the prediction memory unit 108 and outputs the prediction pixel block to the subtracter 11 and the adder 12.

Here, the “predicted pixel block” refers to a pixel block that is detected by the motion vector detection unit 111 and is most similar to the encoding target pixel block.
The motion vector detection unit 111 includes pixel blocks in a specific search range (a predetermined region of each reference candidate frame image selected by a reference image selection process described later) among the pixel blocks stored in the prediction memory unit 108. , A calculation process for performing motion compensated prediction is performed, a prediction pixel block and a motion vector between the prediction pixel block and the encoding target pixel block are detected, and the detected prediction pixel block is detected by the inter prediction unit 110. And the detected motion vector is output to the entropy encoding unit 104.

  Specifically, the motion vector detection unit 111 reads the reference candidate frame image from the prediction memory unit 108, and the pixel value of each pixel in the encoding target pixel block and a predetermined search range in the read reference candidate frame image. The total sum of the absolute values of the difference values (hereinafter referred to as “inter-pixel difference values”) from the pixel values of the pixels at the corresponding positions of the respective pixel blocks (hereinafter referred to as “search pixel block”). A search pixel block that is calculated for each block and has the smallest sum of difference values is detected as a prediction pixel block, and a motion vector from the detected prediction pixel block to the encoding target pixel block is detected (this detection method is referred to as “block Matching method ”)), the detected prediction pixel block is output to the inter prediction unit 110, and the detected motion vector is output to the entropy encoding unit 10. And outputs it to.

  The switch circuit 13 is connected to the inter prediction unit 110 and is output from the inter prediction unit 110 when performing interframe predictive coding by a control signal from a control unit which is a component of an image coding device (not shown). When the prediction pixel block to be processed is input to the subtractor 11 and the intra-frame encoding process is performed by the control signal from the control unit (inter-frame prediction encoding is not performed), the prediction pixel block is connected to the intra prediction unit 109 side. The data output from the intra prediction unit 109 is zero.

Therefore, when connected to the intra prediction unit 109, the subtractor 11 does not calculate the difference, and the extracted pixel block is output to the orthogonal exchange unit 102 as it is.
Next, the configuration of the motion vector detection unit 111 will be described. FIG. 2 is a functional block diagram showing the configuration of the motion vector detection unit 111. The motion vector detection unit 111 includes a correlation degree comparison unit 1111, a reduced encoding target image storage unit 1112, a reduced image generation unit 1113, a reduced reference candidate image storage unit 1114, a clock generation unit 1115, a motion vector calculation unit 1116, and an encoding target. The image storage unit 1117 includes a remaining battery level detection unit 1118, a motion vector calculation control unit 1119, and a reference candidate image storage unit 1120.

The correlation degree comparison unit 1111 has a preset number (here, for example, 10) of reference frame images (hereinafter referred to as “reference candidate frame images”) to be searched for motion compensation prediction. For each of the reduced images, the degree of correlation of the encoding target frame image with the reduced image (hereinafter referred to as “reduced image correlation”) is calculated according to the following Equation 1, and the calculation result is notified to the motion vector arithmetic control unit 1119. To do.
[Equation 1]
Reduced image correlation = Σ | X _ij −Y _ij |, (0 ≦ i ≦ 45, 0 ≦ j ≦ 45)
Here, X _ij represents the pixel value of each pixel of the encoding target frame image, and Y _ij represents the pixel value of each pixel of the reference candidate frame image.

  According to Equation 1, the sum of absolute values of the difference values of the pixel values of the corresponding pixels in the reduced images of the encoding target frame image and the reference candidate frame image is calculated as the reduced image correlation, and the magnitude relationship of the calculated values Thus, the degree of correlation of the reduced image (hereinafter referred to as “correlation”) can be evaluated. That is, the smaller the reduced image correlation value, the higher the correlation.

The reduced encoded image storage unit 1112 stores the reduced image of the encoding target frame image generated by the reduced image generation unit 1113.
The reduced image generation unit 1113 generates reduced images obtained by reducing the encoding target frame images stored in the encoding target image storage unit 1117 and the reference candidate frame images stored in the reference candidate image storage unit 1120, respectively. To do.

  Specifically, for each frame image to be reduced, as shown in FIG. 4A, a reduction process for rounding each pixel block of 16 × 16 pixels constituting the frame image into one pixel using Equation 2 below. As shown in FIG. 4B, the frame image composed of 720 × 480 pixels is reduced to 1/256, and a reduced image composed of 45 × 30 pixels is generated.

Here, as shown by “ref_idx” in FIG. 9B, the encoding target frame image and the reference candidate frame image are assigned a picture number for identifying the frame image with a sequential number, and the picture number. Is also attached to the generated reduced image.
[Equation 2]
Small picture = Σ (X _ij ) / 256, (0 ≦ i ≦ 15, 0 ≦ j ≦ 1)
5)
Here, X _ij represents a pixel value of each pixel constituting a pixel block of 16 × 16 pixels.

Further, the reduced image generating unit 1113 stores the reduced image of the generated encoding target frame image in the reduced encoding target image storage unit 1112 and the generated reduced image of each reference candidate frame image in the reduced reference candidate image storage unit 1114. Store.
The reduced reference candidate image storage unit 1114 stores a reduced image of each reference candidate frame image generated by the reduced image generation unit 1113.

The clock generation unit 1115 generates a clock having a specific frequency and outputs the generated clock to the motion vector calculation unit 1116, and reduces the clock frequency to a predetermined level in response to an instruction from the motion vector calculation control unit 1119.
The motion vector calculation unit 1116 performs a calculation process for performing motion compensation prediction on each search pixel block and encoding target pixel block of the reference candidate frame image selected by the motion vector calculation control unit 1119, and performs motion vector calculation control. It performs according to the mode instruct | indicated by the part 1119, and detects the prediction pixel block and the motion vector between the said prediction pixel block and an encoding object pixel block.

  Specifically, the motion vector calculation unit 1116 uses Equation 3 to calculate the sum of inter-pixel difference values from the encoding target pixel block for each search pixel block (hereinafter referred to as “inter-pixel difference value sum”). , And the search pixel block having the smallest calculated inter-pixel difference value is detected as a prediction pixel block, the position of the detected prediction pixel block in the reference candidate frame image, and the encoding target frame of the encoding target block Based on the positional relationship with the position in the image, as shown in FIG. 3, a motion vector from the prediction pixel block to the encoding target pixel block is detected.

Here, when the mode instruction by the motion vector calculation control unit 1119 is a “first mode instruction” which will be described later, the motion vector calculation unit 1116 performs an inter-pixel difference value for each type of pixel search pixel block. When the mode instruction by the motion vector calculation control unit 1119 is a “second mode instruction” to be described later, an inter-pixel difference value sum is calculated for each of the three types of pixel-by-pixel search pixel blocks. .
[Equation 3]
Inter-pixel difference value sum = Σ | x _ij −y _ij |, (0 ≦ i ≦ 15, 0 ≦ j ≦ 15)
Here, x _ij represents the pixel value of each pixel of the encoding target pixel block, and y _ij represents the pixel value of each pixel of the search pixel block.

The sum of pixel difference values is calculated by Equation 3, and a search pixel block that indicates the minimum value among the pixel difference value sums calculated for each search pixel block is detected as a predicted pixel block.
The encoding target image storage unit 1117 stores the encoding target frame image read from the frame memory unit 101.

The remaining battery level detection unit 1118 detects the remaining battery level and notifies the motion vector calculation control unit 1119 of the detection result.
The motion vector calculation control unit 1119 performs calculation processing mode selection processing and reference candidate frame image notification processing.
First, calculation processing mode selection processing will be described.

  When notified of the detection result of the remaining battery level from the remaining battery level unit 1118, the motion vector calculation control unit 1119 determines whether or not the remaining battery level is equal to or lower than the threshold value. An instruction to execute a calculation process in one mode (hereinafter referred to as “first mode instruction”) is notified to the motion vector calculation unit 1116, and further, the clock generation unit 1115 is instructed to reduce the clock frequency to a predetermined level. .

Thereby, since the clock to the motion vector calculation unit 1116 is lowered, the power consumption can be suppressed, and the electric energy can be controlled according to the remaining battery level.
On the other hand, if not less than the threshold value, the motion vector calculation control unit 1119 notifies the motion vector calculation unit 1116 of a calculation processing execution instruction in the second mode (hereinafter referred to as “second mode instruction”).

Here, the “first mode instruction” refers to an instruction to execute a calculation process for performing motion compensation prediction in a search pixel block of one type of pixel, and the “second mode instruction” refers to a motion This is an instruction to execute arithmetic processing for performing compensation prediction in three types of pixel-by-pixel search pixel blocks.
Next, reference candidate frame image notification processing will be described.

The motion vector calculation control unit 1119 compares the reduced image correlation values for the reference candidate frame images notified from the correlation level comparing unit 1111, selects the three reference candidate frame images in ascending order of the values, and selects them. The motion vector calculation unit 1116 is notified of the reference candidate frame image thus obtained.
The reference candidate image storage unit 1120 stores the reference candidate frame image read from the prediction memory unit 108.
<Operation>
Next, reference image selection processing performed by the motion vector detection unit 111 will be described. FIG. 5 is a flowchart showing the operation of the above processing. The above operation will be described below with reference to FIG.

  The reduced image generating unit 1113 reads out the encoding target frame image stored in the encoding target image storage unit 1117 (step S501), generates the reduced image (step S502), and the reduced encoding target image storage unit 1112. Further, the picture number of the reference candidate frame image is specified based on the picture number of the read target frame image (for example, 10 consecutive frame images before the target frame image are reference candidate frames). When set as an image, a picture number corresponding to a value obtained by subtracting 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 from the picture number of the encoding target frame image, respectively. Specified as the picture number of the reference candidate frame image) and the picture number that matches the picture number of the specified reference candidate frame image Depending on whether or not a reduced image is stored in the reduced reference candidate image storage unit 1114, a reduced image of the reference candidate frame image searched for the read encoding target frame image is stored in the reduced reference candidate image storage unit 1114. It is determined whether or not (step S503).

  In step S503, when the reduced image of the reference candidate frame image is not stored in the reduced reference candidate image storage unit 1114 (step S503: N), the reduced image generation unit 1113 uses the reference candidate frame image as the reference candidate image storage unit. Reading from 1120 (step S504), a reduced image of the reference candidate frame image is generated and stored in the reduced reference candidate image storage unit 1114 (step S505).

In step S503, when the reduced image of the reference candidate frame image is stored in the reduced reference candidate image storage unit 1114 (step S503: Y), the process proceeds to step S506 described later.
Next, the correlation degree comparison unit 1111 performs the encoding target frame image stored in the reduction encoding target image storage unit 1112 for the reduced images of the respective reference candidate frame images stored in the reduction reference candidate image storage unit 1114. The reduced image correlation with the reduced image is calculated (step S506), the calculation result is notified to the motion vector arithmetic control unit 1119, and the reduced image correlation is calculated for the reduced images of all the reference candidate frame images. Is determined (step S507).

If the determination in step S507 is affirmative (step S507: Y), the process proceeds to step S508 described later, and if the determination in step S507 is negative (step S507: N), the process proceeds to step S503. To do.
Next, the motion vector calculation control unit 1119 compares the reduced image correlation values for the reference candidate frame images notified from the correlation comparison unit 1111 and selects three reference candidate frame images in ascending order of the values. In step S508, the selected reference candidate frame image is notified to the motion vector calculation unit 1116.

Next, the operation of the block size selection process performed by the motion vector detection unit 111 will be described. FIG. 7 is a flowchart showing the operation of the above processing. The above operation will be described below with reference to FIG.
The remaining battery level detection unit 1118 detects the remaining battery level (step S701), and notifies the motion vector calculation control unit 1119 of the detection result.

Next, when notified of the remaining battery level detection result from the remaining battery level unit 1118, the motion vector calculation control unit 1119 determines whether or not the remaining battery level is equal to or less than a threshold value (step S702).
If the determination in step S702 is not less than or equal to the threshold (step S702: N), the motion vector calculation control unit 1119 notifies the motion vector calculation unit 1116 of the second mode instruction.

  Next, when the mode instruction by the motion vector calculation control unit 1119 is a “second mode instruction” which will be described later, the motion vector calculation unit 1116 first searches each search in units of 16 × 16 pixels shown in 1 of FIG. An arithmetic process for performing motion compensation prediction (hereinafter referred to as “arithmetic process 1”) is performed for the pixel block and the encoding target pixel block (step S703), and the calculated inter-pixel difference value for each search pixel block. The search pixel block having the smallest sum is detected as a pixel block that is a candidate for the prediction pixel block (hereinafter, “candidate prediction pixel block”), and a motion vector from the detected candidate prediction pixel block to the encoding target pixel block is obtained. As shown in 4 of FIG. 10, the encoding target pixel block of the 16 × 16 pixel unit which is detected and further processed first A calculation process for performing motion compensation prediction (hereinafter referred to as “calculation process 2”) is similarly performed on each of the divided pixel block to be encoded in units of 8 × 8 pixels (step S704), and a candidate prediction pixel. A motion vector from the block and the candidate prediction pixel block to the encoding target pixel block is detected, and each of the encoding target pixel blocks in units of 8 × 8 pixels divided into four is further divided into four (8) in FIG. Arithmetic processing (hereinafter, “arithmetic processing 3”) for performing motion compensation prediction in the same manner for each of the encoding target pixel blocks in 4 × 4 pixel units, which is 16 divisions from the first 16 × 16 pixel encoding target pixel block. (Step S705) and the motion from the candidate prediction pixel block to the encoding target pixel block. The sum of the inter-pixel difference value indicated by the candidate prediction pixel block detected by the calculation process 1 and the magnitude of the motion vector detected by the calculation process 1 (hereinafter referred to as “evaluation value 1”) is detected. And the sum of the sum of pixel difference values indicated by each candidate prediction pixel block detected by the calculation process 2 and the total sum of the magnitudes of the respective motion vectors detected by the calculation process 2 (hereinafter referred to as “evaluation value 2”). And the sum of the sum of inter-pixel difference values indicated by each candidate prediction pixel block detected by the arithmetic processing 3 and the total sum of the magnitudes of the respective motion vectors detected by the arithmetic processing 3 (hereinafter referred to as “ (Referred to as “evaluation value 3”) (step S706), the magnitude relationships of the calculated evaluation values 1 to 3 are compared, and the smallest candidate is detected and the motion detected. The vectors are output to the inter prediction unit 110 and the entropy encoding unit 104, respectively (step S707).

When the determination in step S702 is equal to or less than the threshold (step S702: Y), the motion vector calculation control unit 1119 notifies the motion vector calculation unit 1116 of the first mode instruction.
Next, when the mode instruction by the motion vector calculation control unit 1119 is a “first mode instruction” to be described later, the motion vector calculation unit 1116 performs a calculation process 1 in units of 16 × 16 pixels shown in 1 of FIG. In step S708), for each search pixel block, a search pixel block having the minimum calculated inter-pixel difference value sum is detected as a prediction pixel block, and a motion vector from the detected prediction pixel block to the encoding target pixel block is further detected. It detects (step S709), and outputs the detected prediction pixel block and motion vector to the inter prediction unit 110 and the entropy coding unit 104, respectively (step S710).
<Computation amount suppression effect>
Next, the effect of suppressing the calculation processing amount for performing motion compensation prediction by narrowing down the reference candidate frame image based on the reduced image performed in the first embodiment will be described with a specific example.

In general, the arithmetic processing is performed by calculating an inter-pixel difference value for each pixel in an encoding target pixel block and each search pixel block corresponding thereto, and the inter-pixel difference value is calculated by integer pixel accuracy. , 1/2 pixel (half pel) accuracy and 1/4 pixel (quarter pel) accuracy.
As shown in FIG. 6, the integer pixel accuracy means calculating a pixel difference value at intervals of one pixel indicated by circles, and half-pel accuracy is indicated by circles and triangles as shown in FIG. This means that the inter-pixel difference value is calculated at half-pixel intervals. The quarter-pel accuracy means that the inter-pixel difference value is calculated at quarter-pixel intervals indicated by circles, triangles, and stars as shown in FIG. That means.

  As a procedure of the above arithmetic processing, first, arithmetic processing with integer pixel accuracy is performed on each reference candidate frame image, and a search pixel block having a minimum inter-pixel difference value is detected, and the detected search is performed. For the search pixel block located in a region around the position of the search pixel block in the reference candidate frame image to which the pixel block belongs, calculation processing of half-pel accuracy and quarter-pel accuracy is further performed, and finally the prediction pixel block and the motion vector Are detected.

Therefore, since the calculation amount suppression effect by narrowing down the reference candidate frame image based on the reduced image is mainly exhibited in the calculation processing with integer pixel accuracy, hereinafter, the calculation amount in the calculation processing with integer pixel accuracy (hereinafter referred to as “the calculation amount A”). ”) Is compared with the case of the conventional method, the calculation amount suppression effect will be described.
The calculation amount A of the conventional method is calculated by the following mathematical formula 4.
[Equation 4]
Conventional calculation amount A = (number of operations per pixel) x (number of pixels in the encoding target pixel block) x (number of search pixel blocks) x (number of reference candidate frame images)
Here, since the pixel value is composed of luminance data and two types of color difference data, the “number of operations per pixel” is 3.

The “number of pixels in the encoding target pixel block” is set to 256 since the pixel size of the encoding target pixel block is 16 × 16 pixels in the first embodiment.
The “number of search pixel blocks” is 256 here.
In addition, the “number of reference candidate frame images” is 10 in the first embodiment, so it is 10.

When the above values are applied to Equation 4, the calculation amount A of the conventional method is 1,966,080.
Next, the calculation amount A in the first embodiment is calculated by the following mathematical formula 5.
[Equation 5]
Calculation amount A in Embodiment 1 = (Amount of calculation required for generating reduced image) + (Amount of calculation required for calculating reduced image correlation) × (Number of reference candidate frame images) + (Calculation based on narrowed reference candidate frame images) Amount A)
Here, the amount of calculation required for generating the reduced image is calculated by Equation 6 shown below.
[Equation 6]
Amount of calculation required to generate reduced image = (number of operations per pixel) × (number of reduced image generation) × (number of operations required to calculate each pixel value in reduced image)
“Number of operations per pixel” is 3, “Number of reduced image generations” is 11, including the encoding target frame image in Embodiment 1, and “Number of operations required to calculate each pixel value in the reduced image” Since the number of pixels of the reduced image is 1,350 (45 × 30), when these values are applied to Equation 6, the amount of calculation required to generate the reduced image is 44,550.

Further, “the amount of calculation required for calculating the reduced image correlation degree” is calculated by Expression 7 shown below.
[Equation 7]
Amount of calculation required to calculate reduced image correlation = (number of operations per pixel) × (number of pixels of reduced image)
Since “the number of operations per pixel” is 3, and “the number of pixels of the reduced image” is 1,350, the amount of calculation required for calculating the reduced image correlation is 4,050.

Further, the “calculation amount A based on the narrowing-down reference candidate frame image” is calculated by the following formula 8.
[Equation 8]
Calculation amount A = (number of calculations per pixel) x (number of pixels in the encoding target pixel block) x (number of search pixel blocks) x (number of reference candidate frame images after narrowing down)
Since the “number of reference candidate frame images after narrowing down” is 3 in the first embodiment, the amount of calculation A based on the narrowed reference candidate frame images is 589,824 when the corresponding values are applied to Equation 8. .

Furthermore, when the value of each element is applied to Formula 5, the calculation amount A in the first embodiment is 646,524.
Therefore, the calculation amount A (646,524) in the first embodiment is significantly reduced compared to the calculation amount A (1,966,080) of the conventional method, and the reference based on the reduced image performed in the first embodiment is performed. It can be seen that the calculation processing amount for performing motion compensation prediction is effectively suppressed by narrowing down the candidate frame images.

(Embodiment 2)
In the first embodiment, the number of types of pixel units of the search pixel block to be subjected to the calculation process for performing motion compensation prediction is changed according to the remaining battery level. Depending on the amount, the calculation processing is different in that the number of reference candidate frame images to be searched is changed.

Hereinafter, the description will focus on the differences from the first embodiment.
Since the configurations of the image encoding device and the motion vector detection unit according to the embodiment are the same as those in Embodiment 1, description thereof will be omitted.
<Operation>
Hereinafter, differences in the reference image selection processing operation performed by the motion vector detection unit 111 in Embodiment 2 from Embodiment 1 will be described.

FIG. 8 is a flowchart showing the operation of the above processing. The difference will be described below with reference to FIG.
Since the process of step S801-step S807 is the same as the process of step S501-step S507 in Embodiment 1 shown in FIG. 5, description is abbreviate | omitted.
If the determination in step S807 is affirmative (step S807: Y), the remaining battery level detection unit 1118 detects the remaining battery level (step S808), and notifies the motion vector calculation control unit 1119 of the detection result.

Next, when notified of the result of detection of the remaining battery level from the remaining battery level unit 1118, the motion vector calculation control unit 1119 determines whether or not the remaining battery level is equal to or less than a threshold value (step S809).
When the determination in step S809 is negative (step S809: N), the motion vector calculation control unit 1119 compares the value of the reduced image correlation degree for each reference candidate frame image notified from the correlation degree comparison unit 1111. Then, three reference candidate frame images are selected in ascending order of values (step S810), and the selected reference candidate frame images are notified to the motion vector calculation unit 1116.

When the determination in step S809 is affirmative (step S809: Y), the motion vector calculation control unit 1119 compares the reduced image correlation degree values for the respective reference candidate frame images notified from the correlation degree comparison unit 1111. The reference candidate frame image having the smallest value is selected (step S811), and the selected reference candidate frame image is notified to the motion vector calculation unit 1116.
<Supplement>
As mentioned above, although the motion vector calculating apparatus 100 which concerns on this invention was demonstrated based on Embodiment IV and 2, it is needless to say that this invention is not limited to these Embodiments.
(1) For example, in the first and second embodiments, a reduced image correlation between the reduced image of the reference candidate frame image and the reduced image of the encoding target frame image is calculated, and based on the calculated reduced image correlation. The number of reference candidate frame images is reduced, but without generating a reduced image, the characteristic values (for example, the luminance value and the color difference value) indicated by each pixel of the reference candidate frame image and each of the encoding target frame images By comparing the characteristic values indicated by the pixels, the reference candidate frame images may be selected in descending order of correlation with the encoding target frame image, and the number of reference candidate frame images may be narrowed down.
(2) In Embodiments 1 and 2, in the reference image selection process, the number of reference candidate frame images to be selected is 10, but the number of reference candidate frame images to be selected is 10. There is no limitation as long as it is plural.

Further, the number of reference candidates to be selected is not limited to three as the calculation processing target for motion compensation prediction, and may be a number smaller than the number of reference candidate frame images to be selected.
(3) In the first and second embodiments, the sum of absolute values of the difference values of the corresponding pixel values in the reduced images of the encoding target frame image and the reference candidate frame image is used as the reduced image correlation degree. Although calculated, the sum of the squares of the difference values of the pixel values may be calculated as the reduced image correlation instead of the sum of the absolute values of the difference values of the pixel values.
(4) In the first embodiment, in the block size selection process, the calculation process is executed by the search pixel block of three types of pixels in accordance with the second mode instruction, but is not limited to the three types. As long as there are multiple types.

Further, the pixel unit for executing the arithmetic processing by the first mode instruction is not limited to one type, and may be any number less than the number of pixel units to be executed by the first mode instruction.
(5) In the second embodiment, in the reference image selection process, when the remaining battery level is equal to or less than the threshold value, one reference candidate frame image is selected. The number may be smaller than the number of reference candidate frame images selected based on the degree of correlation.

  In an image encoding device, it can be used as a technique for reducing the amount of calculation for performing motion compensation prediction.

2 is a functional block diagram showing a configuration of an image encoding device 100. FIG. 3 is a functional block diagram illustrating a configuration of a motion vector detection unit 111. FIG. It is the figure which represented the prediction pixel block and motion vector which were detected by the motion vector calculating part 1116 with the schematic diagram. A specific example of the contents of the reduced image generation processing will be shown. It is a flowchart which shows the operation | movement of a reference image selection process. The specific example of the pixel space | interval in each pixel precision used for calculation of the difference value between pixels is shown. It is a flowchart which shows the operation | movement of a block size selection process. It is a flowchart which shows the operation | movement of a reference image selection process. It is the figure which represented the difference of the reference candidate frame image searched in the motion compensation prediction in each standard of MPEG2 / 4 and MPEG4AVC with the schematic diagram. It is the figure which represented the kind of size of the block used as search object with the schematic diagram in the motion compensation prediction in the MPEG4AVC standard.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Subtractor 12 Adder 101 Frame memory part 102 Orthogonal transformation part 103 Quantization part 104 Entropy encoding part 105 Inverse quantization part 106 Inverse orthogonal transformation part 107 Deblock filter part 108 Prediction memory part 109 Intra prediction part 110 Inter prediction part DESCRIPTION OF SYMBOLS 111 Motion vector detection part 1111 Correlation degree comparison part 1112 Reduction encoding object image storage part 1113 Reduction image generation part 1114 Reduction candidate image storage part 1115 Clock generation part 1116 Motion vector calculation part 1117 Encoding object image storage part 1118 Battery remaining charge detection Unit 1119 Motion vector calculation control unit 1120 Reference candidate image storage unit

Claims (6)

A motion vector arithmetic device that performs arithmetic processing for performing motion compensation prediction on an image encoding target image,
Storage means for storing a plurality of reference candidate images, which are candidates for images to be referred to in the arithmetic processing;
A selection unit that selects a reference candidate image to be subjected to the arithmetic processing from each of the reference candidate images according to a degree of correlation with the image encoding target image;
A motion vector arithmetic device comprising: arithmetic means for performing the arithmetic processing on the selected reference candidate image. The motion vector computing device is:
Reduced image generation means for generating a reduced image of each reference candidate image and a reduced image of the image encoding target image;
Calculation means for calculating a degree of correlation indicating a degree of correlation between the reduced image of each reference candidate image and the reduced image of the image encoding target image;
The motion vector according to claim 1, wherein the selection unit selects a reference candidate image to be subjected to the arithmetic processing from the reference candidate images according to a degree of correlation indicated by a correlation degree. Arithmetic unit. The calculation means reduces the reduced image of each reference candidate image and the encoding target image based on pixel values of pixels constituting the reduced image of each reference candidate image and the reduced image of the encoding target image, respectively. The motion vector calculation device according to claim 2, wherein the degree of correlation with an image is calculated. The motion vector calculation device further includes detection means for detecting a remaining battery level,
The selection means reselects a specific reference candidate image among the selected reference candidate images when the detected battery remaining power level is smaller than a threshold value,
The motion vector arithmetic device according to claim 2 or 3, wherein the arithmetic means performs the arithmetic processing on the reselected reference candidate image. The arithmetic processing is performed in units of a plurality of types of pixel blocks on the selected reference candidate image.
The motion vector calculation device further includes detection means for detecting a remaining battery level,
The calculation means suppresses execution of the calculation processing for some types of pixel block units among the plurality of types when the detected remaining battery level is smaller than a threshold value. Item 4. The motion vector calculation device according to Item 2 or 3. The motion vector calculation device includes reduced image storage means for storing a reduced image of each reference candidate image generated by the reduced image generation means,
The reduced image generation means includes:
Determining means for determining whether or not a reduced image of each reference candidate image is stored in the reduced image storage means;
6. A suppression unit that suppresses regeneration of a reduced image of the reference candidate image when the reduced image of each reference candidate image is stored in the reduced image storage unit. The motion vector calculation device according to any one of the above.

Images