JP4789719B2 - Motion vector detection apparatus, motion vector detection method, computer program, and storage medium - Google Patents

Description

  The present invention relates to a motion vector detection device, a motion vector detection method, a computer program, and a storage medium.

  In recent years, the digitalization of information related to multimedia has been rapidly progressing, and the demand for higher image quality of video information has increased accordingly. As a concrete example, it can be mentioned that a conventional SD of 720 × 480 pixels of broadcasting media is being transferred to HD of 1920 × 1080 pixels. However, this demand for high image quality causes an increase in digital data at the same time, and a compression encoding technique and a decoding technique exceeding the conventional performance are required.

  In response to these demands, standardization work of an encoding compression method using inter-frame prediction using correlation between images in the activities of ITU-T SG16 and ISO / IEC JTC1 / SC29 / WG11 is in progress.

  Among these, H.264 is an encoding method that is said to realize the most efficient encoding at present. H.264 / MPEG-4 PART10 (AVC) (hereinafter referred to as H.264). H. H.264 encoding / decoding specifications are disclosed in Patent Document 1, for example.

  H. As one of the technologies newly introduced in H.264, a plurality of types of pixel block partitions for performing predictive coding as shown in FIG. 2 are prepared, the amount of motion in fine pixel units is detected, and the one with the smallest prediction error is prepared. There is a technique for selecting and further reducing the code amount. Such a partition is called a macroblock partition.

  Here, the macroblock partition will be described in detail with reference to FIG. H. In H.264, as shown in FIG. 2 (i), 16 × 16 pixels, which is the size used in MPEG2, is set as a macroblock type having the maximum block size. In addition, it is possible to select a macroblock partition to be used for predictive coding from a total of four types including (ii) to (iv) in FIG. The selection of the macroblock partition is equivalent to selecting the macroblock size used for predictive encoding from any of 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, and 8 × 8 pixels. is there.

  Furthermore, the macroblock composed of 8 × 8 pixels shown in FIG. 2 (iv) can be divided into finer submacroblocks. Here, as shown in FIGS. 2 (v) to (viii), a sub-macro block having a block size of a minimum of 4 × 4 pixels is used and divided into any of four types of sub-macro block partitions. Is possible. In this case, the sub-macroblock partition is selected by selecting a sub-macroblock size used for predictive coding from 8 × 8 pixels, 8 × 4 pixels, 4 × 8 pixels, and 4 × 4 pixels. Is equivalent.

  That is, the macroblock partition used for predictive coding is selected from 19 types in total. Among these, three types are the number of macroblock partitions (FIG. 2 (i) to (iii)) that are not the target of the sub macroblock partition. Further, the remaining 16 types depend on the product of the number of macroblocks subject to sub-macroblock partitions in FIG. 2 (iv): 4 and the number of sub-macroblock partitions: 4 in FIGS. 2 (v) to (viii). .

  H. In H.264, as shown in FIG. 5, a reference frame with good coding efficiency is selected from a plurality of reference frames (RF1 to RF5) for each macroblock in the encoding target frame (CF), and which frame is used for each block. Can be specified. Therefore, different reference frames may be selected even for macroblocks in the same encoding target frame CF. In this way, H.C. In H.264, a plurality of search hierarchies for motion vector detection are set using a plurality of reference frames.

As a result, the motion information is searched in finer image units, and the accuracy of the motion information is improved.
JP 2005-167720 A

  However, compared to MPEG 2 having only one type of macroblock, H.264 has as many as 19 types of partitions, and if the motion vectors are evaluated for all the blocks included in the partitions, a huge amount of computation is required. H. In H.264, since motion vector detection can be performed using a plurality of search layers (a plurality of reference frames), if all partitions are evaluated for each search layer, an enormous number of operations are required.

  In addition, in the case of a mobile device such as a video camcorder, an increase in calculation load leads to an increase in driving battery consumption, and there is a problem in that shooting time is shortened.

  The present invention has been made in view of such problems, and greatly reduces the amount of calculation while maintaining search accuracy, and performs motion search using a plurality of block sizes with high efficiency to detect a motion vector. It aims to make it possible.

In order to solve the above-described problem, the present invention provides a motion vector detection apparatus that detects a motion vector using processing target image data and a plurality of reference image data, and includes a first block size in the processing target image data. Determining the second image data having the first block size in the first reference image data from among the plurality of reference image data, wherein the first matching error with the first image data having the minimum is determined. A first motion vector detecting means for detecting a first motion vector based on the first image data and the second image data; and the first image data is smaller than the first block size. A plurality of third image data having a second block size is divided into a plurality of third image data from the first matching error that is minimized. First extracting means for extracting a plurality of second matching errors response, each of said plurality of extracted by the first extraction means second matching error is compared with a predetermined threshold value, the second The comparison result between the matching error and the predetermined threshold value is determined to match any of a plurality of predetermined conditions, and motion vectors are detected using the plurality of reference image data according to the matched conditions And a first determining means for determining a third block size used for.

  According to the present invention, it is possible to significantly reduce the amount of calculation while maintaining search accuracy, and to detect a motion vector by performing a motion search with a plurality of block sizes with high efficiency.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

  First, FIG. 1a shows an example of the overall configuration of an image compression coding apparatus as an image processing apparatus according to an embodiment of the present invention.

  The moving image compression encoding apparatus includes a camera unit 200, a subtracter 2001, an integer conversion unit 2002, a quantization unit 2003, an entropy encoding unit 2004, an inverse quantization unit 2005, an inverse integer conversion unit 2006, and an adder 2007. . In addition, frame memories 2008 and 2012, an intra prediction unit 2009, switches 2010 and 2015, a deblocking filter 2011, an inter prediction unit 2013, and a motion detection unit 2014 are further included. Then, the image data input from the camera unit 200 is divided to form a block, and an encoding process is performed on a block basis to output encoded data. Subsequently, H.C. The H.264 encoding process will be described.

  First, the subtracter 2001 subtracts predicted image data from image data input from the camera unit, and outputs image residual data. The generation of predicted image data will be described later. The integer transform unit 2002 performs orthogonal transform processing on the image residual data output from the subtractor 2001 by DCT transform or the like, and outputs transform coefficients. Then, the quantization unit 2003 quantizes the transform coefficient using a predetermined quantization parameter. The entropy encoding unit 2004 receives the transform coefficient quantized by the quantization unit 2003, entropy encodes it, and outputs it as encoded data.

  On the other hand, the transform coefficient quantized by the quantization unit 2003 is also used for generating predicted image data. The inverse quantization unit 2005 performs inverse quantization on the transform coefficient quantized by the quantization unit 2003. Further, the inverse integer transform unit 2006 performs inverse integer transform on the transform coefficient inversely quantized by the inverse quantization unit 2005 by inverse DCT transform or the like, and outputs the result as decoded image residual data. An adder 2007 adds the decoded image residual data and the predicted image data, and outputs the result as reconstructed image data.

  The reconstructed image data is recorded in the frame memory 2008, but is recorded via the deblocking filter 2011 when the deblocking filter process is performed. Further, when the deblocking filter process is not performed, it is recorded in the frame memory 2012 without going through the deblocking filter 2011. The switch 2010 is a selection unit that selects whether to perform deblocking filter processing. Of the reconstructed image data, data that may be referred to in subsequent predictions is stored in the frame memory 2008 or 2012 as reference frame data for a period of time. The deblocking filter 2011 is used to remove noise.

  The intra prediction unit 2009 performs intra-frame prediction processing using the image data recorded in the frame memory 2008, and generates predicted image data. Also, the inter prediction unit 2013 performs inter-frame prediction processing based on the motion vector information detected by the motion detection unit 2014 using the reference image data recorded in the frame memory 2012, and generates predicted image data. The motion detection unit 2014 detects the motion vector in the input image data using the reference image data recorded in the frame memory 2012, and outputs the detected motion vector information to the inter prediction unit 2013 and the entropy encoding unit 2004. The switch 2015 is a selection unit for selecting which of intra prediction and inter prediction is used. One of the output from the intra prediction unit 2009 and the output from the inter prediction unit 2013 is selected, and the selected predicted image data is output to the subtracter 2001 and the adder 2007.

  Next, a specific configuration and operation of a motion detection unit as a motion vector detection device corresponding to the present embodiment will be described. FIG. 1 b is a diagram illustrating an example of a specific configuration of a motion detection unit corresponding to the present embodiment. FIG. 3 is a flowchart showing an example of processing for detecting a motion vector and determining a macroblock partition for one macroblock in input image data in the motion detection unit corresponding to the present embodiment. In this embodiment, the encoding method is H.264. The case of H.264 will be described as an example, but the present invention may be applied to other encoding methods.

  Input image data, which is an image signal of a motion vector detection target (processing target), is input from the camera unit 200 to the large block motion vector detection unit 3 at the input terminal 1. In addition, an image signal that has been encoded and locally decoded is input from the frame memory 2012 to the input terminal 2 as reference image data, and is stored in the reference image storage unit 4.

  In step S100, the search hierarchy is set to the highest search hierarchy. In the present embodiment, motion vector detection is performed for a plurality of search layers, that is, a plurality of reference frames, and the reference frame encoded most recently in the input image data is set as the “highest search layer”. Of the reference frames, the oldest encoded time is referred to as the “lowest search hierarchy”. When the search hierarchy is updated, the "uppermost search hierarchy" to the "lowermost search hierarchy" are updated in the order of encoding time. For example, in the case of FIG. 5, RF1 corresponds to “the highest search hierarchy”, and RF5 corresponds to “the lowest search hierarchy”. When the search hierarchy is updated, the search hierarchy is updated in the order of RF1, RF2, RF3, RF4, and RF5. In FIG. Although the search hierarchy is five according to the H.264 standard, this is merely an example, and the number of hierarchies is not limited to this.

  Next, in step S101, a matching position setting process is performed. Here, the setting of the matching position means determination of the position of the reference image data in the reference frame (search hierarchy) for the input image data that is the frame to be encoded. If a motion vector has not been detected between the reference frame and the input image data in the previous search layer, zero is set as the initial motion vector, and the matching position is set to the same block position as the input image data. Set. At this time, the large block search mode is set to OFF. The large block search mode is a mode in which a search in a macro block (large block) of 16 × 16 pixel size is performed through a lower search hierarchy. When the large block search mode is set to ON, the search using the medium block or the small block is not performed thereafter. If a motion vector is detected in a higher search hierarchy and the large block search mode is on, the matching position is set with the motion vector as the initial motion vector.

  Next, in step S102, the large block motion vector detection unit 3 calculates a matching error MAEbig. This matching error MAEbig is calculated for the macroblock (large block) of 16 × 16 pixel size of the input image data from the input terminal 1 and the reference image data of the large block input from the reference image storage unit 4. The matching error can be calculated, for example, by obtaining a difference between pixel values of both blocks to be compared. The matching error MAEbig is calculated for each search layer, and finally, the motion vector when the matching error MAEbig in the search range (all search layers) becomes the smallest and the matching error at that time are obtained. This motion vector is referred to as a “final motion vector”.

Next, in step S103, it is determined whether or not the large block search mode is set to ON. If the large block search mode is on (“YES” in step S103), the process proceeds to step S118. In step S118, it is determined whether the search hierarchy is the lowest search hierarchy. If it is determined that the search hierarchy is the lowest ("YES" in step S118), the process proceeds to step S117. In step S <b> 117, the final macroblock partition / final motion vector determination unit 11 receives information indicating that the macroblock partition is a large block and the final motion vector from the large block motion vector detection unit 3 . On the other hand, when it is determined that it is not the lowest search hierarchy (“NO” in step S118), the process proceeds to step S119, one search hierarchy is updated, and one lower search hierarchy from step S101 is updated. Continue processing.

  On the other hand, if the large block search mode is off (“NO” in step S103), the process proceeds to step S104. In step S104, the motion vector obtained in step S102 and the matching error MAEbig are transmitted to the middle block error extraction unit 5.

  The middle block error extraction unit 5 divides the large block 400 into 8 × 8 pixel macro blocks 401 to 404 (hereinafter referred to as middle blocks) as shown in FIG. Extract errors. The matching errors extracted here are MAEmid (0) for the middle block 401, MAEmid (1) for the middle block 402, MAEmid (2) for the middle block 403, and MAEmid (3) for the middle block 404. Each extracted matching error is transmitted to the first macroblock partition determination unit 6. 4A shows that when the matching error MAEbig of the large block 400 is 120, the matching errors of each middle block are MAEmid (0) = 10, MAEmid (1) = 10, MAEmid (2) = 30, MAEmid ( 3) An example of = 70 is shown.

  Next, in step S105, the first macroblock (MB) partition determination unit 6 compares the matching error of each middle block in the current search hierarchy with a threshold value, and determines a macroblock partition under the following conditions. . Here, any one of the macroblock partitions (i) to (iv) in FIG. 2 is selected. That is, here, any block size of 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, or 8 × 8 pixels is selected. In the following, the relationship Thl <Thm is established between the threshold Thl and Thm.

(Condition 1)
When MAEmid (0) <Thl, MAEmid (1) <Thl, MAEmid (2) <Thl, MAEmid (3) <Thl,
The macroblock partition is set to (i) in FIG.

(Condition 2)
When MAEmid (0) <Thl, MAEmid (1) <Thl, MAEmid (2) <Thl, MAEmid (3) ≧ Thl,
When MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
When MAEmid (1) + MAEmid (3) ≧ Thm, MAEmid (2) + MAEmid (3) <Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (3) ≧ Thm, MAEmid (2) + MAEmid (3) ≧ Thm
The macroblock partition is set to (iv) in FIG.

(Condition 3)
When MAEmid (0) <Thl, MAEmid (1) <Thl, MAEmid (2) ≧ Thl, MAEmid (3) <Thl,
When MAEmid (0) + MAEmid (2) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (0) + MAEmid (2) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
When MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (2) + MAEmid (3) <Thm
The macroblock partition is set to (ii) in FIG.
When MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (2) + MAEmid (3) ≧ Thm
The macroblock partition is set to (iv) in FIG.

(Condition 4)
When MAEmid (0) <Thl, MAEmid (1) <Thl, MAEmid (2) ≧ Thl, MAEmid (3) ≧ Thl,
The macroblock partition is set to (ii) in FIG.

(Condition 5)
When MAEmid (0) <Thl, MAEmid (1) ≧ Thl, MAEmid (2) <Thl, MAEmid (3) <Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (1) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (1) + MAEmid (3) ≧ Thm
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm,
The macroblock partition is set to (iii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (1) + MAEmid (3) ≧ Thm
The macroblock partition is set to (iv) in FIG.

(Condition 6)
When MAEmid (0) <Thl, MAEmid (1) ≧ Thl, MAEmid (2) <Thl, MAEmid (3) ≧ Thl,
The macroblock partition is set to (iii) in FIG.

(Condition 7)
When MAEmid (0) <Thl, MAEmid (1) ≧ Thl, MAEmid (2) ≧ Thl, MAEmid (3) <Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

(Condition 8)
When MAEmid (0) <Thl, MAEmid (1) ≧ Thl, MAEmid (2) ≧ Thl, MAEmid (3) ≧ Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

(Condition 9)
When MAEmid (0) ≧ Thl, MAEmid (1) <Thl, MAEmid (2) <Thl, MAEmid (3) <Thl,
When MAEmid (0) + MAEmid (1) <Thm, MAEmid (0) + MAEmid (2) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (0) + MAEmid (1) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm
The macroblock partition is set to (ii) in FIG.
When MAEmid (0) + MAEmid (1) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm,
The macroblock partition is set to (iii) in FIG.
When MAEmid (0) + MAEmid (1) ≧ Thm, MAEmid (0) + MAEmid (2) ≧ Thm
The macroblock partition is set to (iv) in FIG.

(Condition 10)
When MAEmid (0) ≧ Thl, MAEmid (1) <Thl, MAEmid (2) <Thl, MAEmid (3) ≧ Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

(Condition 11)
When MAEmid (0) ≧ Thl, MAEmid (1) <Thl, MAEmid (2) ≧ Thl, MAEmid (3) <Thl,
The macroblock partition is set to (ii) in FIG.

(Condition 12)
When MAEmid (0) ≧ Thl, MAEmid (1) <Thl, MAEmid (2) ≧ Thl, MAEmid (3) ≧ Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

(Condition 13)
When MAEmid (0) ≧ Thl, MAEmid (1) ≧ Thl, MAEmid (2) <Thl, MAEmid (3) <Thl,
The macroblock partition is set to (ii) in FIG.

(Condition 14)
When MAEmid (0) ≧ Thl, MAEmid (1) ≧ Thl, MAEmid (2) <Thl, MAEmid (3) ≧ Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

(Condition 15)
When MAEmid (0) ≧ Thl, MAEmid (1) ≧ Thl, MAEmid (2) ≧ Thl, MAEmid (3) <Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

(Condition 16)
When MAEmid (0) ≧ Thl, MAEmid (1) ≧ Thl, MAEmid (2) ≧ Thl, MAEmid (3) ≧ Thl,
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) <Thm,
Macroblock partitions are set to (ii) and (iii) in FIG.
When MAEmid (1) + MAEmid (0) <Thm, MAEmid (0) + MAEmid (2) ≧ Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (ii) in FIG.
When MAEmid (1) + MAEmid (0) ≧ Thm, MAEmid (0) + MAEmid (2) <Thm, MAEmid (1) + MAEmid (3) <Thm, MAEmid (2) + MAEmid (3) ≧ Thm,
The macroblock partition is set to (iii) in FIG.
Otherwise, the macroblock partition is set to (iv) in FIG.

  Here, for example, assuming that Thl = 15 and Thm = 40, in the example of FIG. 4A, the case of the above (condition 4) is applied, and (ii) of FIG. 2 is selected as the macroblock partition. When the macroblock partition becomes (i) in FIG. 2, the large block search mode is set to ON. When a macroblock partition other than (i) in FIG. 2 is selected, the large block search mode setting is kept off.

  Next, in step S106, it is determined again whether the large block search mode is set to ON. As described above, in step S105, when the type (i) of FIG. 2 is selected as the macroblock partition, the large block search mode is set to ON. Therefore, when it is determined that the large block search mode is set to ON (“YES” in step S106), the process proceeds to step S120, one search hierarchy is updated, and the process returns to step S101. In step S101, a new matching position is set using the motion vector already calculated by the large block motion vector detection unit 3 in step S102, and the search in the large block is repeated up to the lowest layer.

  As described above, by limiting the macroblock partitions used in the search of the lower hierarchy, the arithmetic processing can be greatly reduced as compared with the case where the search is performed in all the macroblock partitions.

  On the other hand, when it is determined that the large block search mode is set to OFF (“NO” in step S106), the process proceeds to step S107. In step S107, the middle block motion vector detection unit 7 acquires a macroblock partition and a motion vector from the first macroblock partition determination unit 6, and determines a matching position using the motion vector.

  Next, in step S108, a matching error MAEmid with reference image data input from the reference image storage unit 4 is calculated for each block using the acquired macroblock partition. The matching error MAEmid is calculated for each search layer, and finally, a motion vector when the matching error MAEmid in the search range (all search layers) becomes the smallest and the matching error at that time are obtained. This motion vector is referred to as a “final motion vector”. At this time, when a macro block partition other than FIG. 2 (iv) is selected, the middle block search mode is set to ON. On the other hand, when FIG. 2 (iv) is selected, the middle block search mode is set to OFF.

  Next, in step S109, it is determined whether or not the middle block search mode is set to ON. The middle block search mode is a mode in which a search in “medium block” is performed through a lower search hierarchy. When the middle block search mode is set to ON, the search using the large block or the small block is not performed thereafter.

  If it is determined in step S109 that the middle block search mode is set to ON (“YES” in step S109), the process proceeds to step S121. In step S121, it is determined whether the search hierarchy is the lowest search hierarchy. If it is determined that the search hierarchy is the lowest ("YES" in step S121), the process proceeds to step S117. In step S117, the final macroblock partition / final motion vector determination unit 11 receives the macroblock partition information and the final motion vector at that time from the middle block motion vector detection unit 7. On the other hand, if it is determined that it is not the lowest search hierarchy (“NO” in step S121), the process proceeds to step S122, one search hierarchy is updated, and one lower search hierarchy from step S107 is updated. Continue processing.

  As a result, by limiting the macroblock partitions used in the search of the lower hierarchy, the arithmetic processing can be greatly reduced as compared with the case where the search is performed in all the macroblock partitions.

  If it is determined in step S109 that the medium block search mode is set to OFF (“NO” in step S109), the process proceeds to step S110. In step S110, it is determined whether or not the macroblock partition used in the middle block error extraction unit 5 has the pattern shown in FIG. If it is determined that the pattern is not the pattern shown in FIG. 2 (iv) (“NO” in step S110), the process proceeds to step S121 and the above processing is performed. On the other hand, if it is determined that the pattern is that of FIG. 2 (iv) (“YES” in step S110), the process proceeds to step S111.

  In step S <b> 111, the small block error extraction unit 8 acquires each matching error and motion vector in the middle block from the middle block motion vector search unit 7.

  The small block error extraction unit 8 further divides the 8 × 8 pixel macroblock 410 into sub-macroblocks 411 to 414 (hereinafter referred to as small blocks) having a 4 × 4 pixel size, as shown in FIG. 4B. After that, the matching error for each small block is extracted. The matching errors extracted here are MAElow (0) for the small block 411, MAElow (1) for the small block 412, MAElow (2) for the small block 413, and MAElow (3) for the small block 414. Each extracted matching error is transmitted to the sub macroblock partition determination unit 9. 4B, when the matching error MAElow of the macroblock 410 is 30, the matching error of each small block is MAElow (0) = 3, MAElow (1) = 3, MAElow (2) = 7, MAElow ( 3) = 17 example.

  Note that the process of step S111 is executed for each of the 8 × 8 pixel macroblocks in FIG. Thereby, a sub macroblock partition is determined for each macroblock, and a motion vector corresponding to the determined submacroblock partition is detected.

  In step S112, the sub-macroblock (SMB) partition determination unit 9 compares the matching error of each small block in the current search hierarchy with a threshold value, and determines a sub-macroblock partition under the following conditions. Here, any one of the sub-macroblock partitions (v) to (viii) in FIG. 2 is selected. That is, here, any block size of 8 × 8 pixels, 8 × 4 pixels, 4 × 8 pixels, or 4 × 4 pixels is selected. In the following, a relationship of Thll <Thml is established between the threshold values Thll and Thml.

(Condition 1 ')
When MAElow (0) <Thll, MAElow (1) <Thll, MAElow (2) <Thll, MAElow (3) <Thll,
The sub macroblock partition is set to (v) in FIG.

(Condition 2 ')
When MAElow (0) <Thll, MAElow (1) <Thll, MAElow (2) <Thll, MAElow (3) ≧ Thll,
When MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml,
The sub macroblock partitions are set to (vi) and (vii) in FIG.
When MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml,
The sub macroblock partition is set to (vii) in FIG.
When MAElow (1) + MAElow (3) ≧ Thml, MAElow (2) + MAElow (3) <Thml,
The sub macroblock partition is set to (vi) in FIG.
When MAElow (1) + MAElow (3) ≧ Thml, MAElow (2) + MAElow (3) ≧ Thml,
The sub macroblock partition is set to (viii) in FIG.

(Condition 3 ')
When MAElow (0) <Thll, MAElow (1) <Thll, MAElow (2) ≧ Thll, MAElow (3) <Thll,
When MAElow (0) + MAElow (2) <Thml, MAElow (2) + MAElow (3) <Thml,
The sub macroblock partitions are set to (vi) and (vii) in FIG.
When MAElow (0) + MAElow (2) <Thml, MAElow (2) + MAElow (3) ≧ Thml,
The sub macroblock partition is set to (vii) in FIG.
When MAElow (0) + MAElow (2) ≧ Thml, MAElow (2) + MAElow (3) <Thml,
The sub macroblock partition is set to (vi) in FIG.
When MAElow (0) + MAElow (2) ≧ Thml, MAElow (2) + MAElow (3) ≧ Thml,
The sub macroblock partition is set to (viii) in FIG.

(Condition 4 ')
When MAElow (0) <Thll, MAElow (1) <Thll, MAElow (2) ≧ Thll, MAElow (3) ≧ Thll
The sub macroblock partition is set to (vi) in FIG.

(Condition 5 ')
When MAElow (0) <Thll, MAElow (1) ≧ Thll, MAElow (2) <Thll, MAElow (3) <Thll,
When MAElow (1) + MAElow (0) <Thml, MAElow (1) + MAElow (3) <Thml,
The sub macroblock partitions are set to (vi) and (vii) in FIG.
When MAElow (1) + MAElow (0) <Thml, MAElow (1) + MAElow (3) ≧ Thml,
The sub macroblock partition is set to (vi) in FIG.
When MAElow (1) + MAElow (0) ≧ Thml, MAElow (1) + MAElow (3) <Thml,
The sub macroblock partition is set to (vii) in FIG.
When MAElow (1) + MAElow (0) ≧ Thml, MAElow (1) + MAElow (3) ≧ Thml,
The sub macroblock partition is set to (viii) in FIG.

(Condition 6 ')
When MAElow (0) <Thll, MAElow (1) ≧ Thll, MAElow (2) <Thll, MAElow (3) ≧ Thll,
The sub macroblock partition is set to (vii) in FIG.

(Condition 7 ')
When MAElow (0) <Thll, MAElow (1) ≧ Thll, MAElow (2) ≧ Thll, MAElow (3) <Thll,
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

(Condition 8 ')
When MAElow (0) <Thll, MAElow (1) ≧ Thll, MAElow (2) ≧ Thll, MAElow (3) ≧ Thll
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

(Condition 9 ')
When MAElow (0) ≧ Thll, MAElow (1) <Thll, MAElow (2) <Thll, MAElow (3) <Thll,
When MAElow (0) + MAElow (1) <Thml, MAElow (0) + MAElow (2) <Thml,
The sub macroblock partitions are set to (vi) and (vii) in FIG.
When MAElow (0) + MAElow (1) <Thml, MAElow (0) + MAElow (2) ≧ Thml,
The sub macroblock partition is set to (vi) in FIG.
When MAElow (0) + MAElow (1) ≧ Thml, MAElow (0) + MAElow (2) <Thml,
The sub macroblock partition is set to (vii) in FIG.
When MAElow (0) + MAElow (1) ≧ Thml, MAElow (0) + MAElow (2) ≧ Thml,
The sub macroblock partition is set to (viii) in FIG.

(Condition 10 ')
When MAElow (0) ≧ Thll, MAElow (1) <Thll, MAElow (2) <Thll, MAElow (3) ≧ Thll,
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

(Condition 11 ')
When MAElow (0) ≧ Thll, MAElow (1) <Thll, MAElow (2) ≧ Thll, MAElow (3) <Thll,
The sub macroblock partition is set to (vi) in FIG.

(Condition 12 ')
When MAElow (0) ≧ Thll, MAElow (1) <Thll, MAElow (2) ≧ Thll, MAElow (3) ≧ Thll,
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

(Condition 13 ')
When MAElow (0) ≧ Thll, MAElow (1) ≧ Thll, MAElow (2) <Thll, MAElow (3) <Thll,
The sub macroblock partition is set to (vi) in FIG.

(Condition 14 ')
When MAElow (0) ≧ Thll, MAElow (1) ≧ Thll, MAElow (2) <Thll, MAElow (3) ≧ Thll
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

(Condition 15 ')
When MAElow (0) ≧ Thll, MAElow (1) ≧ Thll, MAElow (2) ≧ Thll, MAElow (3) <Thll,
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

(Condition 16 ')
When MAElow (0) ≧ Thll, MAElow (1) ≧ Thll, MAElow (2) ≧ Thll, MAElow (3) ≧ Thll,
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) <Thml
The sub macroblock partitions are set to (vi) and (vii) in FIG.
MAElow (1) + MAElow (0) <Thml, MAElow (0) + MAElow (2) ≧ Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vi) in FIG.
MAElow (1) + MAElow (0) ≧ Thml, MAElow (0) + MAElow (2) <Thml, MAElow (1) + MAElow (3) <Thml, MAElow (2) + MAElow (3) ≧ Thml
The sub macroblock partition is set to (vii) in FIG.
In other cases, the sub-macroblock partition is set to (viii) in FIG.

  Here, for example, assuming that Thll = 4 and Thml = 10, in the example of FIG. 4B, the above-mentioned case (condition 4 ′) is applied, and (vi) of FIG. 2 is selected as the sub macroblock partition. . When the sub macroblock partition becomes (v) in FIG. 2, the middle block search mode is set to ON. In addition, when a sub macroblock partition other than (v) in FIG. 2 is selected, the small block search mode is set to ON while the setting of the medium block search mode remains OFF.

  Next, in step S113, it is determined whether or not the small block search mode is set to ON. When the small block search mode is on (“YES” in step S113), the process proceeds to step S114. In this case, a sub-macroblock partition other than that of FIG. 2 (v) is selected as the 8 × 8 pixel macroblock. On the other hand, when the small block search mode is OFF (“NO” in step S113), the medium block search mode is set to ON, and the process proceeds to step S121. In this case, since the sub macroblock partition of FIG. 2 (v) is selected for the 8 × 8 pixel macroblock, motion vector detection is performed for the middle block of 8 × 8 pixels.

  In step S <b> 114, the small block motion vector search unit 10 acquires the sub macroblock partition information and the motion vector from the middle block motion vector detection unit 7. The small block motion vector search unit 10 sets a new matching position using the motion vector.

  Next, in step S115, the small block motion vector search unit 10 calculates a matching error MAElow in the search range with reference image data input from the reference image storage unit 4 in the sub macroblock partition. The matching error MAElow is calculated for each search layer, and finally, the motion vector when the matching error MAElow in the search range (all search layers) becomes the smallest and the matching error at that time are obtained. This motion vector is referred to as a “final motion vector”.

  In a succeeding step S116, it is determined whether or not the search hierarchy is the lowest search hierarchy. If it is determined that the search hierarchy is the lowest ("YES" in step S116), the process proceeds to step S117. In step S117, the final macroblock partition / final motion vector determination unit 11 receives the sub-macroblock partition information and the final motion vector at that time from the small block motion vector search unit 10. On the other hand, when it is determined that it is not the lowest search hierarchy (“NO” in step S116), the process proceeds to step S123, one search hierarchy is updated, and one search hierarchy from step S114 is updated. Continue processing.

  In step S117, the final macroblock partition / final motion vector determination unit 11 receives macroblock partition information and final motion vector information. These pieces of information are transmitted from any one of the large block motion vector detection unit 3, the medium block motion vector detection unit 7, and the small block motion vector detection unit 10 as described above. In subsequent step S <b> 124, the final macroblock partition / final motion vector determination unit 11 transmits the received information to the inter prediction unit 2013 and the entropy coding unit 2004 outside the motion detection unit 2014.

  In the above embodiment, the search pixel accuracy can be implemented as one pixel in both the horizontal direction and the vertical direction for all block sizes. On the other hand, the search accuracy may be varied depending on the search hierarchy.

For example, consider a case in which the search accuracy in the horizontal direction is skipped by one pixel in the upper layer, and the search is performed with one pixel accuracy in the horizontal and vertical directions in the lower layer.
Here, when the size of the macroblock is 16 × vertical 16 when searching at one pixel accuracy in both the horizontal and vertical directions in the upper layer, the search accuracy is equivalent to the same search range with one horizontal pixel skip. The macroblock size to be performed is horizontal 8 × vertical 16. At this time, the block is divided horizontally 4 × vertically 8 and can be compared with a threshold value. According to the result, the block size to be used in the lower hierarchy is determined, and a search with one pixel accuracy can be performed in the horizontal and vertical directions in the lower hierarchy.

  According to the embodiments of the invention described above, it is possible to narrow down appropriate macroblock partitions to be used for a plurality of search hierarchies from a large size macroblock type. Thereby, it is not necessary to perform motion search for all macroblock partitions and sub-macroblock partitions for each of a plurality of reference frames. Therefore, the calculation load regarding motion detection can be significantly reduced. As a result, the battery consumption of the apparatus can be reduced, and long-time shooting can be realized even with a video camera system compatible with high image quality.

[Other Embodiments]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the above-described functions to the system, and the system reading and executing the program codes. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. In addition, an operating system (OS) running on a computer performs part or all of actual processing based on an instruction of the program code, and the above-described functions are realized by the processing.

  Furthermore, you may implement | achieve with the following forms. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, based on the instruction of the program code, the case where the above-described functions are realized by the CPU included in the function expansion card or the function expansion unit performing part or all of the actual processing is also included.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

It is a figure which shows an example of the whole structure of the image processing apparatus corresponding to embodiment of invention. It is a figure which shows an example of a structure of the motion vector detection apparatus corresponding to embodiment of invention. It is a figure which shows an example of the block size used for a motion vector detection. It is a flowchart which shows an example of the process which detects a motion vector about one macroblock in input image data, and determines a macroblock partition in the motion detection part corresponding to embodiment of invention. It is a figure for demonstrating the breakdown of the matching error in a large block, a medium block, and a small block corresponding to embodiment of invention. It is a figure for demonstrating the concept which detects motion information from a some reference frame.

Claims (12)

A motion vector detection device that detects a motion vector using processing target image data and a plurality of reference image data,
The first block in the first reference image data among the plurality of reference image data, wherein the first matching error with the first image data having the first block size in the processing target image data is minimized. First motion vector detecting means for determining second image data having a size and detecting a first motion vector based on the first image data and the second image data;
The first image data is divided into a plurality of third image data having a second block size smaller than the first block size, and the plurality of third image data is calculated based on the first matching error that is minimized. First extraction means for extracting a plurality of second matching errors corresponding to each of the image data;
Each of the plurality of second matching errors extracted by the first extracting means is compared with a predetermined threshold value, and a plurality of predetermined comparison results between each second matching error and the predetermined threshold value are determined. First determination means for determining which of the conditions is met, and determining a third block size used for motion vector detection using the plurality of reference image data according to the adapted conditions. A motion vector detection device characterized by the above. If the third block size matches the first block size,
The first motion vector detection means further detects the first motion vector using reference image data other than the first reference image data using the first block size. The motion vector detection device according to claim 1.   The first determining means determines the third block size as the first block size when there is a second matching error larger than a first threshold among the plurality of second matching errors. The motion vector detection device according to claim 1, wherein the block size is determined according to a predetermined condition. When the third block size is smaller than the first block size and larger than the second block size,
A fifth image having the third block size in the plurality of reference image data, in which a third matching error with the fourth image data having the third block size in the processing target image data is minimized. 4. The apparatus according to claim 1, further comprising second motion vector detecting means for determining data and detecting a second motion vector based on the fourth image data and the fifth image data. The motion vector detection device according to any one of the above. When the third block size is the second block size,
A fifth block having the second block size in the first reference image data, wherein a third matching error with the fourth image data having the second block size in the first image data is minimized. Second motion vector detection means for determining a second motion vector based on the fourth image data and the fifth image data;
The fourth image data is divided into a plurality of sixth image data having a fourth block size smaller than the second block size, and the plurality of sixth image data are calculated based on the third matching error that is minimized. Second extraction means for extracting a plurality of fourth matching errors corresponding to each of the image data;
Second determination means for determining a fifth block size used for motion vector detection using the plurality of reference image data based on the plurality of fourth matching errors extracted by the second extraction means. The motion vector detection device according to claim 1, further comprising: When the fifth block size matches the second block size,
The second motion vector detection means further detects the second motion vector using reference image data other than the first reference image data using the second block size. The motion vector detection device according to claim 5.   The second determining means determines the fifth block size as the second block size when there is a fourth matching error larger than a second threshold among the plurality of fourth matching errors. The motion vector detection device according to claim 5, wherein the block size is determined according to a predetermined condition. If the fifth block size does not match the second block size,
An eighth image having the fifth block size in the plurality of reference image data, in which a fifth matching error with the seventh image data having the fifth block size in the processing target image data is minimized. 8. The apparatus according to claim 5, further comprising third motion vector detecting means for determining data and detecting a third motion vector based on the seventh image data and the eighth image data. The motion vector detection device according to any one of the above. The first block size is a block size of 16 pixels × 16 pixels,
The second block size is a block size of 8 pixels × 8 pixels,
The motion vector detection device according to claim 5, wherein the fourth block size is a block size of 4 pixels × 4 pixels. A motion vector detection method for detecting a motion vector using processing target image data and a plurality of reference image data,
The first block in the first reference image data among the plurality of reference image data, wherein the first matching error with the first image data having the first block size in the processing target image data is minimized. A first motion vector detection step of determining second image data having a size and detecting a first motion vector based on the first image data and the second image data;
The first image data is divided into a plurality of third image data having a second block size smaller than the first block size, and the plurality of third image data is calculated based on the first matching error that is minimized. A first extraction step of extracting a plurality of second matching errors corresponding to each of the image data;
Each of the plurality of second matching errors extracted in the first extraction step is compared with a predetermined threshold, and a comparison result between each second matching error and the predetermined threshold is determined in advance. A first determination step of determining which of the conditions is met, and determining a third block size used for motion vector detection using the plurality of reference image data according to the adapted conditions. A motion vector detection method characterized by the above. A computer program for causing a computer to execute the motion vector detection method according to claim 10. A computer-readable storage medium storing the computer program according to claim 11.

Images