JP4547881B2 - Motion vector detection device and detection method, matching block table generation device and generation method, program for executing each method, and computer-readable medium storing the program - Google Patents

Description

  The present invention is, for example, a motion vector detection device and detection method suitable for application when detecting a motion vector for performing motion compensation in a motion compensation prediction encoding device, a matching block table generation device and generation method, and The present invention relates to a program for executing each method.

  Specifically, according to the present invention, when a motion vector corresponding to a target pixel position of a predetermined frame is detected by performing a block matching process, a pixel block having a block pattern corresponding to a feature amount related to the target pixel position is used. The present invention relates to a motion vector detection device or the like that attempts to detect a motion vector with high accuracy.

  For example, in a process of compressing and encoding an image signal such as an MPEG2 (Moving Picture Experts Group) system, an encoding process based on a correlation between two adjacent frames, so-called motion compensation interframe prediction, is used. In motion compensation inter-frame prediction, it is necessary to detect a motion vector for each pixel unit or for a pixel block unit of a predetermined size between two adjacent frames (one is described as a target frame and the other is referred to as a reference frame).

  As a method for detecting a motion vector, a block matching method has been conventionally used (for example, see Patent Document 1). FIG. 22 shows a configuration example of a motion vector detection apparatus 300 that detects a motion vector according to the block matching method.

  The motion vector detection apparatus 300 includes an input terminal 301 to which an image signal is input, a frame memory 302 that stores an image signal of a target frame Fc, and a frame memory 303 that stores an image signal of a reference frame Fr. Yes. When an image signal of a frame from the input terminal 301 is supplied to the frame memory 302 and written, the image signal of the previous frame stored in the frame memory 302 is read out, supplied to the frame memory 303, and written. It is.

  In addition, the motion vector detection device 300 uses the image signal of the target frame Fc stored in the frame memory 302 and the image signal of the reference frame Fr stored in the frame memory 303 to each pixel of the target frame Fc. A motion detection unit 304 that detects a motion vector corresponding to the position, and an output terminal 305 that outputs the motion vector detected by the motion detection unit 304 are provided.

  The motion detection unit 304 detects a motion vector corresponding to each pixel position of the target frame Fc as described above according to the block matching method. A motion vector detection process in the motion detection unit 304 will be described with reference to FIGS. 23 and 24. FIG. 23 shows the correspondence between the target frame Fc and the reference frame Fr. FIG. 24 shows the procedure of the detection process.

The motion detection unit 304 sequentially designates all pixel positions in the target frame Fc as the target pixel position, and applies the reference block Fr to a reference block having a predetermined size (L × L pixels) centered on the target pixel position. The sum of absolute differences of the pixel values of the corresponding pixel pair with the reference block (same size as the base block) moved within the provided search area SR is calculated according to the equation (1).
ΣiΣj = | Fc (i, j) −Frn (i, j) | (1)

  Here, Fc (i, j) is the pixel value of the pixel of the base block, and Frn (i, j) is the pixel value of the pixel of the reference block with the identification number n. Also, Σi means a summation operation when i is incremented by 1 from 1 to L, and Σj means a summation operation when j is incremented by 1 from 1 to L.

  Then, the difference vector between the pixel position at the center of the reference block and the target pixel position when the sum of absolute differences of the pixel values of the corresponding pixel pairs of the base block and the reference block is minimized corresponds to the target pixel position. As a motion vector.

  Specifically, the motion detection unit 304 performs the following process on the target pixel position of the target frame Fc.

In step S1, a search area SR having a size larger than that of the reference block is set in the reference frame Fr with the same pixel position as the target pixel position of the target frame Fc as the center. In step S2, a variable min that stores the minimum value of the sum of absolute differences is initialized to the maximum value. For example, when the pixel value of one pixel is 8 bits and the size of the reference block is 4 × 4 pixels, the variable min is initialized to 4096 (= 2 ⁸ × 16).

  Next, in step S3, the identification number n of the reference block to be moved within the search area SR is initialized to 1. In step S4, a variable sum for storing the calculation result of the sum of absolute differences is initialized to zero.

  Next, in step S5, the difference absolute value sum of the pixel values of the pixel pairs located at the corresponding positions of the reference block of the target frame Fc and the reference block of the identification number n in the search area SR set in the reference frame Fr. Is substituted into the variable sum. In step S6, the variable sum and the variable min, which are the calculation results of step S5, are compared to determine whether the variable sum is smaller than the variable min. If it is determined that the variable sum is smaller than the variable min, the process proceeds to step S7.

  In step S7, the variable min is replaced with the variable sum, and the identification number n of the reference block is stored as a motion vector number. After the process of step S7, the process proceeds to step S8. If it is determined in step S6 described above that the variable sum is not smaller than the variable min, the process of step S7 is skipped and the process proceeds to step S8.

  In step S8, it is determined whether or not the reference block identification number n is the maximum value, that is, whether or not the reference block has been moved to the entire search area SR. If it is determined that the identification number n of the reference block is not the maximum value, the process proceeds to step S9. In step S9, the identification number n of the reference block is incremented, the process returns to step S4, and the subsequent processes are repeated.

  If it is determined in step S8 that the reference block identification number n is the maximum value, that is, if the reference block has been moved to the entire search area SR, the process proceeds to step S10. In this step S10, the difference vector between the pixel position at the center of the reference block corresponding to the identification number n stored as the motion vector number and the target pixel position of the target frame Fc corresponds to the target pixel position of the target frame Fc. As a motion vector.

Japanese Patent No. 3277417

  As described above, when motion vector detection is performed by the block matching method, how to set the block pattern of the pixel block when performing block matching becomes a problem. That is, by setting the pixel block to be small, it is possible to cope with small movements, but on the contrary, it is vulnerable to noise, resulting in variations in the results. On the other hand, by setting a large pixel block, it is resistant to noise and stabilizes the result, but on the contrary, it cannot cope with small movement.

  An object of the present invention is to accurately detect a motion vector using a pixel block having an appropriate block pattern.

The motion vector detection apparatus according to the present invention is a motion vector detection apparatus that detects a motion vector by performing block matching processing using the first to third image signals, and includes a target pixel from the image signal of the first frame. Feature amount acquisition means for extracting a plurality of pixel data located in the vicinity of the position and obtaining a feature amount related to the target pixel position based on the plurality of pixel data, and a target pixel acquired from the image signal of the first frame For each block pattern, the pixel block according to the position, the image signal of the second frame that is the previous frame of the first frame, and the image signal of the third frame that is the subsequent frame of the first frame are used. Determined based on the evaluation value indicating the degree of correlation obtained from the pixel block corresponding to the target pixel position obtained from the motion compensated image signal obtained A storage unit storing a matching block table indicating a correspondence relationship between the block pattern and the feature amount related to the target pixel position, and a feature amount obtained by the feature amount acquisition unit from the matching block table stored in the storage unit A block pattern information acquisition unit that obtains block pattern information that can identify the positions of the pixels constituting the corresponding block pattern, and a block pattern indicated by the block pattern information obtained by the block pattern information acquisition unit, of the first frame Matching between the first pixel block corresponding to the target pixel position and each of the plurality of second pixel blocks of the second frame of the block pattern indicated by the block pattern information obtained by the block pattern information acquisition unit Matching calculation means for performing calculations and this matching operation Based on the matching operation result corresponding to a plurality of second pixel blocks obtained by means comprises a motion vector obtaining means for obtaining a motion vector corresponding to the target pixel position of the first frame, the matching block table The first frame image signal and the motion compensated image signal are used for each combination of the pixel position and block pattern of the first frame based on a plurality of pixel data located in the vicinity of the pixel position. In addition to obtaining the feature amount, an evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position is calculated from each of the image signal of the first frame and the motion compensated image signal, and the calculated By adding the evaluation value to the address value corresponding to the block pattern and the calculated feature amount, for each feature amount, A block evaluation table showing a correspondence relationship between each of the block patterns and the accumulated value of the evaluation values is generated. Based on the generated block evaluation table of each feature amount, each feature amount is used in the block matching. This is generated by determining the block pattern of the power pixel block .

The motion vector detection method according to the present invention is a motion vector detection method for detecting a motion vector by performing block matching processing using the image signals of the first to third frames, and the image of the first frame. A first step of extracting a plurality of pixel data located in the vicinity of the target pixel position from the signal and obtaining a feature amount related to the target pixel position based on the plurality of pixel data, and an image signal of the first frame Using the obtained pixel block at the target pixel position, the image signal of the second frame that is the previous frame of the first frame, and the image signal of the third frame that is the subsequent frame of the first frame An evaluation value indicating the degree of correlation obtained from the pixel block corresponding to the target pixel position obtained from the motion compensated image signal obtained for each block pattern. The positions of the pixels constituting the block pattern corresponding to the feature amount obtained in the first step are identified from the matching block table indicating the correspondence between the block pattern determined based on the feature amount and the feature amount related to the target pixel position. A second step of obtaining possible block pattern information, a first pixel block corresponding to the target pixel position of the first frame of the block pattern indicated by the block pattern information obtained in the second step, A third step of performing a matching operation between each of the plurality of second pixel blocks of the second frame of the block pattern indicated by the block pattern information obtained in step 2, and in the third step Based on the matching calculation result corresponding to the plurality of second pixel blocks obtained, the target pixel position of the first frame And a fourth step of obtaining a motion vector corresponding to said matching block table, the image signal and using the above-described motion compensation image signal of the first frame, the pixel position and the block pattern of the first frame For each combination, a feature amount is obtained based on a plurality of pixel data located in the vicinity of the pixel position, and the pixel position is determined from each of the image signal of the first frame and the motion compensation image signal. An evaluation value indicating the degree of correlation of the obtained pixel block is calculated, and the calculated evaluation value is added to the value of the address corresponding to the block pattern and the calculated feature value, thereby obtaining each feature value. In addition, a block evaluation table showing the correspondence between each of the block patterns and the accumulated value of the evaluation values is generated, and each generated feature amount Based on the block evaluation table, the block pattern of the pixel block to be used in the block matching is determined for each feature amount .

  A program according to the present invention is for causing a computer to execute the motion vector detection method described above. A computer-readable medium according to the present invention records the above-described program.

  In the present invention, a plurality of pixel data located in the vicinity of the target pixel position is extracted from the image signal of the first frame, and a feature amount related to the target pixel position is obtained based on the plurality of pixel data. In this case, the feature amount only needs to represent features of a plurality of pixel data. For example, a multi-bit code obtained by encoding a plurality of pixel data into 1-bit data is used as the feature amount. Further, for example, the feature amount may be obtained from coefficient data obtained by performing orthogonal transformation such as DCT (Discrete Cosine Transform) on a pixel block composed of a plurality of pixel data.

  In addition, a matching block table indicating a correspondence relationship between the feature amount and the block pattern of the pixel block to be used for block matching is prepared. Here, the block pattern corresponding to a certain feature amount can detect a motion vector with high accuracy by using the pixel block to be used in block matching for the target pixel position where the feature amount is obtained as the block pattern. , Obtained by learning in advance.

  From this matching block table, block pattern information corresponding to the feature amount related to the target pixel position described above is acquired. The first pixel block (reference block) corresponding to the target pixel position of the first frame (target frame) in the block pattern indicated by the acquired block pattern information, and the block indicated by the acquired block pattern information A matching operation is performed between each of the plurality of second pixel blocks (reference blocks) of the second frame (reference frame) of the pattern.

  This matching calculation only needs to be able to obtain a value indicating the degree of correlation between the first pixel block and the second pixel block as a result. For example, the matching operation obtains an absolute difference value between corresponding pixel data between the pixel data constituting the first pixel block and the pixel data constituting the second pixel block, and the obtained absolute difference value It is an operation for obtaining the sum of. For example, the matching operation may be an operation for obtaining a sum of squared differences.

  A motion vector corresponding to the target pixel position of the first frame is obtained based on the matching calculation result corresponding to the plurality of second pixel blocks. In this case, a difference vector between the center pixel position of the second block having the best matching level with the first pixel block and the target pixel value is obtained as a motion vector.

  As described above, in the present invention, when the motion vector corresponding to the target pixel position of the first frame is detected by performing block matching processing, the pixel of the block pattern corresponding to the feature amount related to the target pixel position Blocks are used, and block matching processing using pixel blocks having an appropriate block pattern can be performed, and a motion vector corresponding to the target pixel position can be detected with high accuracy.

  For example, a plurality of second pixel blocks of the second frame that are subjected to matching calculation with the first pixel block corresponding to the target pixel position of the first frame are, for example, the first pixel block corresponding to the target pixel position of the first frame. This is a pixel block in the search area of the second frame. In this case, a motion vector can be detected in the search area of the second frame.

  Also, for example, for each pixel position of the second frame, a feature amount of the pixel position is obtained based on a plurality of pixel data located in the vicinity of the pixel position, and the feature amount and each pixel position of the second frame Is generated, and a plurality of pixel positions of the second frame corresponding to the feature amount related to the target pixel position described above are acquired from the table, and the target pixel position of the first frame is acquired. The plurality of second pixel blocks in the second frame that are subjected to a matching operation with the first pixel block corresponding to the pixel block are pixel blocks corresponding to the plurality of pixel positions in the second frame. In this case, the number of second pixel blocks to be subjected to the matching calculation can be reduced, so that the calculation amount can be greatly reduced.

  The matching block table generation device according to the present invention detects a motion vector corresponding to a target pixel position of a predetermined frame by performing a block matching process using a pixel block of a block pattern corresponding to a feature amount of the target pixel position. An apparatus for generating a matching block table indicating a correspondence relationship between a feature quantity and a block pattern, which is used when storing, a storage means, a passing pixel table generation means, a motion compensation image generation means, and a block evaluation table generation Means and a matching block table generating means.

The storage means is a first frame among three temporally continuous frames, the second frame that is the previous frame of the first frame, and the subsequent frame of the first frame. The image signal with the third frame is stored. The passing pixel table generating means uses the image signal of the second frame and the image signal of the third frame stored in the storage means, and corresponds to each pixel position of the second frame for each block pattern. was pixel position of the third frame, to detect performs block matching processing using the pixel block of the block pattern, and each pixel position of the first frame, the pixel position of the second frame and the third frame A passing pixel table showing the correspondence relationship is generated.

The motion compensated image generating means uses the second frame image signal and the third frame image signal stored in the storage means, and the passing pixel table generated by the passing pixel table generating means, and each block pattern every, corresponding to each pixel position of the first frame, obtains the pixel position of the second frame and the third frame from the passage pixel table, the first based on the pixel value of the pixel position obtained A pixel value of the frame is obtained, and a motion compensated image signal corresponding to the image signal of the first frame is generated.

The block evaluation table generation means uses the first frame image signal stored in the storage means and the motion compensation image signal generated by the motion compensation image generation means, and uses the pixel position and block pattern of the first frame. For each combination, a feature amount is obtained based on a plurality of pixel data located in the vicinity of the pixel position, and obtained from each of the image signal and the motion compensated image signal of the first frame corresponding to the pixel position. An evaluation value indicating the degree of correlation of the pixel block is calculated, and the calculated evaluation value is added to the value of the address corresponding to the block pattern and the calculated feature value, so that a block is obtained for each feature value. A block evaluation table indicating the correspondence between each pattern and the accumulated value of the evaluation values is generated.

  The matching block table generation unit determines a block pattern of a pixel block to be used for block matching for each feature amount based on the block evaluation table of each feature amount generated by the block evaluation table generation unit, and a matching block A table is generated.

  Further, the matching block table generation method according to the present invention performs a block matching process using a motion vector corresponding to a target pixel position of a predetermined frame using a pixel block of a block pattern corresponding to a feature amount of the target pixel position. A method for generating a matching block table indicating a correspondence relationship between a feature quantity and a block pattern, which is used when detecting, a first step of generating the above-described passing pixel table, and the above-described motion compensation image signal , A third step of generating the block evaluation table described above, and a fourth step of generating the matching block table described above.

  A program according to the present invention is for causing a computer to execute the above-described matching block table generation method. A computer-readable medium according to the present invention records the above-described program.

  In the present invention, a matching block table showing the correspondence between the feature quantity and the block pattern is generated. In this case, the block pattern corresponding to a certain feature amount can detect a motion vector with high accuracy by using the pixel block to be used in block matching for the target pixel position where the feature amount is obtained as the block pattern. Sought by learning.

  According to this invention, when the motion vector corresponding to the target pixel position of the first frame is detected by performing the block matching process, the pixel block having the block pattern corresponding to the feature amount related to the target pixel value is used. Thus, block matching processing using pixel blocks having an appropriate block pattern can be performed, and motion vectors can be detected with high accuracy.

  Further, according to the present invention, a plurality of pixel positions of the second frame having the same feature amount as the feature amount related to the target pixel position of the first frame are obtained, and the second pixel position corresponding to the target pixel position of the first frame is acquired. A plurality of second pixel blocks in a second frame that are matched with one pixel block are set as pixel blocks corresponding to the obtained plurality of pixel positions. Since the number of pixel blocks can be reduced, the amount of calculation can be greatly reduced.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration of a motion vector detection device 100 as an embodiment. The motion vector detection apparatus 100 detects a motion vector according to a block matching method.

  The motion vector detection apparatus 100 includes an input terminal 101 to which an image signal is input, a frame memory 102 that stores an image signal of a target frame Fc as a first frame, and an image of a reference frame Fr as a second frame. A frame memory 103 for accumulating signals.

  When an image signal of a frame from the input terminal 101 is supplied to the frame memory 102 and written, the image signal of the previous frame stored in the frame memory 102 is read out, supplied to the frame memory 103, and written. It is. In this case, when the image signal of the target frame Fc is an image signal of the # n-1 frame, the image signal of the reference frame Fr is an image signal of the # n-2 frame one frame before.

In addition, the motion vector detection device 100 includes a class code generation unit 104. The class code generation unit 104 sequentially sets each pixel position of the target frame Fc as a target pixel position, and generates a class code CL _n−1 as a feature amount related to the target pixel position. In this case, the class code generation unit 104 extracts a plurality of pixel data located in the vicinity of the target pixel position from the image signal of the target frame Fc stored in the frame memory 102, and each of the plurality of pixel data is extracted. The class code CL _n-1 is obtained by encoding into 1-bit data.

Here, as shown in FIG. 2, for example, the plurality of pieces of pixel data located in the vicinity of the target position are nine pixels P1 to P9 centered on the target pixel, and a 9-bit code is set as the class code CLn _-1. can get. This class code CL _n-1 is obtained, for example, by 1-bit ADRC (Adaptive Dynamic Range Coding).

In 1-bit ADRC, a maximum value MAX and a minimum value are detected from a plurality of pixel data, and further a dynamic range DR = MAX-MIN is detected. For each of a plurality of pixel data, the minimum value MIN is subtracted from the pixel data. The subtracted value is divided by DR / 2. Thereby, each of the plurality of pixel data is encoded into 1-bit data, and a bit string obtained by arranging the data in a predetermined order is set as a class code CL _n−1 .

In addition, the motion vector detection device 100 is generated by the class code generation unit 104 and the image signal of the target frame Fc stored in the frame memory 102, the image signal of the reference frame Fr stored in the frame memory 103, and the like. The motion vector calculation unit 105 that obtains a motion vector corresponding to the target pixel position of the target frame Fc using the class code CL _n-1 as the feature amount related to the target pixel position of the target frame Fc, and the motion And an output terminal 106 for outputting a motion vector obtained by the vector calculation unit 105.

  The operation of the motion vector detection device 100 shown in FIG. 1 will be described.

  An image signal input to the input terminal 101 is supplied to the frame memory 102 and written therein. As a result, the image signal of the # n−1 frame is stored in the frame memory 102 as the image signal of the target frame Fc. At this time, the image signal of the previous frame stored in the frame memory 102 is read out and supplied to the frame memory 103 for writing. As a result, the image signal of # n-2 frame is stored in the frame memory 103 as the image signal of the reference frame Fr.

In the class code generation unit 104, each pixel position of the target frame Fc is sequentially set as the target pixel position, and a plurality of pieces of pixel data positioned in the vicinity of the target pixel value based on the image signal of the target frame Fc stored in the frame memory 102 Are extracted (see FIG. 2), and a class code CL _n−1 as a feature amount related to the target pixel position is generated using the plurality of pixel data. The class code CL _n−1 generated by the class code generation unit 104 is supplied to the motion vector calculation unit 105.

In the motion vector calculation unit 105, the image signal of the target frame Fc stored in the frame memory 102, the image signal of the reference frame Fr stored in the frame memory 103, and further generated by the class code generation unit 104, A motion vector corresponding to the target pixel position of the target frame Fc is obtained from the class code CL _n−1 as the feature amount related to the target pixel position of the target frame Fc. The motion vector obtained by the motion vector calculation unit 105 is derived to the output terminal 106.

Next, details of the motion vector calculation unit 105 will be described. FIG. 3 shows the configuration of the motion vector calculation unit 105.
The motion vector calculation unit 105 includes an ME memory unit 121. The ME memory unit 121 stores a table indicating the correspondence between the class code and each pixel position of the reference frame Fr. This table shows each pixel of the # n-2 frame generated by the class code generation unit 104 in a state where the image signal of the # n-2 frame as the image signal of the reference frame Fr is accumulated in the frame memory 102. It is generated using the class code CL _n-2 as the feature quantity related to the position.

FIG. 4 shows the structure of the table stored in the ME memory unit 121. This table is composed of (a + 1) × (b + 1) cells indicated by feature amount addresses 0 to a and flag addresses 0 to b. Hereinafter, for example, the cell having the feature address “1” and the flag address “2” is described as a cell (1, 2). The feature amount address corresponds to a class code CL _n-2 as a feature amount relating to each pixel position of the # n-2 frame generated by the class code generation unit 104. For example, when the class code is 9 bits, the maximum value of the feature amount address a = 2 ⁹ −1.

  In the cells after the flag address “1” of the feature amount address 0, the coordinates of the pixel position of the # n-2 frame in which the class code “000000000” is generated by the class code generation unit 104 are stored in raster order. The flag (0, 0) of the feature value address “0” has already stored the coordinates of the pixel position where the class code “000000000” has been generated. The flag of the feature value address “0” The number of cells after the address “1”, that is, the number of used cells is stored. For example, when the class code “000000000” is generated for each pixel position in the # n-2 frame, the cell (0, 1), the cell (0, 2), the cell (0, 2) are generated. The coordinates of each of the three pixel positions are stored in 3), and 3 which is the number of cells in which the coordinates are stored is stored in the cell (0, 0).

  In the cells after the flag address “1” of the feature amount address “1”, the coordinates of the pixel position of the # n-2 frame in which the class code “000000001” is generated by the class code generation unit 104 are stored in raster order. The cell (1, 0) of the flag address “0” of the feature amount address “1” stores the coordinates of the pixel position where the class code “000000001” is generated. The flag address of the feature amount address “0” The number of cells after “1” is stored. For example, if the same class code “000000001” is generated for 10 pixel positions among the pixel positions of the # n-2 frame, 10 is assigned to the cell (1, 1),. The coordinates of each pixel position are stored, and 10 which is the number of cells storing the coordinates is stored in the cell (1, 0). The same applies to the cells after the feature amount address “2”, and the description thereof will be omitted.

In addition, the ME memory unit 121 is generated by the class code generation unit 104 based on a table (see FIG. 4) indicating the correspondence between the class code stored as described above and each pixel position of the reference frame Fr. The information on the pixel position of the reference frame Fr corresponding to the class code CL _n-1 related to the target pixel position of the target frame Fc is output as motion vector candidate position information. In this case, the ME memory unit 121 refers to the part of the feature amount address corresponding to the class code CL _n−1 , and when the number stored in the cell of the flag address “0” is m, the flag address “1”. The coordinates stored in “m” are output as motion vector candidate position information. In this case, the ME memory unit 121 outputs m as motion vector candidate total number information.

Further, the motion vector calculation unit 105 has a matching block table memory 122. The memory 122 stores a matching block table indicating a correspondence relationship between a class code as a feature amount and a block pattern of a pixel block to be used for block matching, and is generated by the class code generation unit 104. Block pattern information corresponding to the class code CL _n-1 related to the target pixel position of the frame Fc is output. The memory 122 constitutes a matching block table storage unit and a block pattern information acquisition unit. Each block pattern has a different block shape (square, cross, etc.) and the number of pixels.

  FIG. 5 shows an example of the matching block table. In this example, a block pattern number b is stored as block pattern information. FIG. 6 shows another example of the matching block table. In this example, a pixel position code is stored as block pattern information. This pixel position code indicates a relative position with respect to the center pixel position. Here, the block pattern corresponding to a certain class code can detect a motion vector with high accuracy by using the pixel pattern to be used in block matching for the target pixel position where the class code is obtained as the block pattern. , Obtained by learning in advance. The generation of the matching block table will be described later.

  Further, the motion vector calculation unit 105 outputs the motion vector candidate position information and the total candidate number information corresponding to the target pixel position of the target frame Fc output from the ME memory unit 121, and the target frame output from the memory 122. Block pattern information of the pixel block to be used in block matching corresponding to the target pixel position of Fc, the image signal of the target frame Fc stored in the frame memory 102 (see FIG. 1), and the frame memory 103 (FIG. 1). The motion vector calculation unit 123 obtains a motion vector corresponding to the target pixel position of the target frame Fc using the image signal of the reference frame Fr stored in the reference). The motion vector calculation unit 123 constitutes a matching calculation unit and a motion vector acquisition unit.

  That is, the motion vector calculation unit 123 includes a reference block as a first pixel block corresponding to the target pixel position of the target frame Fc in the block pattern indicated by the block pattern information supplied from the memory 122, and the ME memory unit 121. A matching calculation is performed between each of the reference blocks as the plurality of second pixel blocks corresponding to the plurality of pixel positions of the reference frame Fr identified by the candidate position information of the motion vector supplied from. In addition, the motion vector calculation unit 123 obtains a motion vector corresponding to the target pixel position of the target frame Fc based on matching calculation results corresponding to a plurality of reference blocks. In this case, the motion vector calculation unit 123 obtains, as a motion vector, a difference vector between the center pixel position of the reference block and the target pixel value that has the best degree of matching with the base block.

The operation of the motion vector calculation unit 105 shown in FIG. 3 will be described.
The class code CL _n−1 as the feature amount related to the target pixel position of the target frame Fc generated by the class code generation unit 104 (see FIG. 1) is supplied to the ME memory unit 121. The ME memory unit 121 stores a table (see FIG. 4) indicating the correspondence between the class code and each pixel position of the reference frame Fr.

The ME memory unit 121 refers to the portion of the feature amount address corresponding to the class code CL _n-1 in the table, and coordinates of a plurality of pixel positions of the reference frame Fr corresponding to the class code CL _n-1. Is output as motion vector candidate position information, and the number of pixel positions is output as motion vector candidate total number information. As described above, the motion vector candidate position information and the total candidate number information output from the ME memory unit 121 are supplied to the motion vector calculation unit 123.

Further, the class code CL _n−1 as the feature amount related to the target pixel position of the target frame Fc generated by the class code generation unit 104 (see FIG. 1) is supplied to the matching block table memory 122. The memory 122 stores a matching block table (see FIGS. 5 and 6) indicating the correspondence between the class code and the block pattern of the pixel block to be used for block matching. From the memory 122, block pattern information corresponding to the class code CL _n-1 related to the target pixel position of the target frame Fc is read from the table and output. As described above, the block pattern information output from the memory 122 is supplied to the motion vector calculation unit 123.

  In the motion vector calculation unit 123, the reference block corresponding to the target pixel position of the target frame Fc in the block pattern indicated by the block pattern information supplied from the memory 122, and the candidate position information of the motion vector supplied from the ME memory unit 121 A matching operation is performed between each of the plurality of reference blocks corresponding to the plurality of pixel positions of the reference frame Fr specified in (1).

  This matching calculation is performed using the image signal of the target frame Fc stored in the frame memory 102 and the image signal of the reference frame Fr stored in the frame memory 103. In the present embodiment, this matching operation calculates an absolute value of the difference between corresponding pixel data between the pixel data constituting the base block and the pixel data constituting the reference block, and a plurality of obtained differences. This is an operation for obtaining the sum of absolute values.

  In addition, the motion vector calculation unit 123 obtains a motion vector corresponding to the target pixel position of the target frame Fc based on matching calculation results corresponding to a plurality of reference blocks. In this case, a difference vector between the center pixel position of the reference block and the target pixel value that has the best degree of matching with the base block is obtained as a motion vector.

Next, details of the motion vector calculation unit 123 will be described. FIG. 7 shows the configuration of the motion vector calculation unit 123.
The motion vector calculation unit 123 receives a candidate number initialization unit 130 that initializes a candidate number n to 1, and initialization of the candidate number n in the candidate number initialization unit 130, thereby obtaining the minimum value of the sum of absolute differences A minimum value initialization unit 131 that initializes min is included. The minimum value initialization unit 131 acquires the number of pixels in the block from the block pattern information, and initializes the minimum value min of the difference absolute value sum to the maximum value. In this case, when the pixel value of one pixel is 8 bits, the minimum value min is initialized to (2 ⁸ × number of pixels in the block).

  Further, the motion vector calculation unit 123 receives the initialization of the candidate number n in the candidate number initialization unit 130 or the determination of the next candidate number n in the next candidate number determination unit 138 described later, and the difference absolute value that is an evaluation value An evaluation value initialization unit 132 that initializes the sum sum to 0, and the difference absolute value sum that is an evaluation value corresponding to the candidate number n in response to the initialization of the difference absolute value sum sum in the evaluation value initialization unit 132 and an evaluation value calculation unit 133 for calculating sum.

  The evaluation value calculation unit 133 sets a reference block corresponding to the target pixel position of the target frame Fc based on the block pattern indicated by the block pattern information supplied from the memory 122 (see FIG. 3), and also sets the reference frame Fr. A reference block corresponding to the candidate position of candidate number n is set. Then, the evaluation value calculation unit 133 obtains an absolute difference value between corresponding pixel data between the pixel data constituting the base block and the pixel data constituting the reference block, and further obtained plural absolute difference values. Is obtained as a sum of absolute values.

  The motion vector calculation unit 123 has an evaluation value determination unit 134. This evaluation value determination unit 134 is a difference absolute value sum sum, which is an evaluation value obtained by the evaluation value calculation unit 133, and a minimum value resetting unit 135 initialized by the minimum value initialization unit 131 or described later. Compared with the reset minimum value min, it is determined whether min> sum, and the determination result is output.

  The motion vector calculation unit 123 includes a minimum value resetting unit 135 and a motion vector number setting unit 136. When the determination result of the evaluation value determination unit 134 indicates that min> sum, the minimum value resetting unit 135 resets the minimum value min to the difference absolute value sum sum obtained by the evaluation value calculation unit 133. . When the determination result of the evaluation value determination unit 134 indicates that min> sum, the motion vector number setting unit 136 sets the motion vector number to the candidate number n at that time.

  In addition, the motion vector calculation unit 123 includes a selected candidate number number determination unit 137. The determination unit 137 determines whether or not the candidate number n is the same as the total number of candidates each time the evaluation value determination unit 134 determines the candidate number n, and the candidate number n is the same as the total number of candidates. In some cases, the motion vector number finally set by the motion vector number setting unit 136 is output.

  In addition, the motion vector calculation unit 123 includes a next candidate number determination unit 138 that receives the determination of the candidate number determination unit 137 and determines the next candidate number n when the candidate number n is not the same as the total number of candidates. In this case, the next candidate number determination unit 138 increments the candidate number n to determine the next candidate number n.

  Further, the motion vector calculation unit 123 includes a motion vector number / motion vector conversion unit 139 that converts the motion vector number output from the determination unit 137 into a motion vector. The conversion unit 139 obtains a difference vector between the reference frame candidate position corresponding to the motion vector number and the target pixel position of the target frame as a motion vector corresponding to the target pixel position of the target frame Fc.

  Next, the operation of the motion vector calculation unit 123 shown in FIG. 7 will be described. The candidate position information output from the ME memory unit 121 (see FIG. 3) is supplied to the evaluation value calculation unit 133. Further, the total candidate number information output from the ME memory unit 121 (see FIG. 3) is supplied to the selected candidate number determination unit 137. Further, the block pattern information output from the matching block table 122 (see FIG. 3) is supplied to the minimum value initialization unit 131 and the evaluation value calculation unit 133.

Candidate number initialization unit 130 initializes candidate number n to 1. In response to the initialization of the candidate number n, the minimum value initialization unit 131 initializes the minimum value min of the sum of absolute differences to the maximum value (2 ⁸ × number of pixels in the block). The minimum value min obtained by the minimum value initialization unit 131 is supplied to the evaluation value determination unit 134.

  In addition, the evaluation value initialization unit 132 initializes the difference absolute value sum that is an evaluation value to 0 in response to the initialization of the candidate number n. Then, the evaluation value calculation unit 133 receives the initialization of the difference absolute value sum sum, and calculates the difference absolute value sum sum which is an evaluation value corresponding to the candidate number 1.

  In this case, the evaluation value calculation unit 133 sets a reference block corresponding to the target pixel position of the target frame Fc based on the block pattern indicated by the block pattern information, and corresponds to the candidate position of candidate number 1 of the reference frame Fr. The reference block is set. Then, in this evaluation value calculation unit 133, the difference absolute value between corresponding pixel data between the pixel data constituting the base block and the pixel data constituting the reference block is obtained, and a plurality of obtained difference absolute values are obtained. By obtaining the sum of the values, a difference absolute value sum sum that is an evaluation value of candidate number 1 is obtained. The difference absolute value sum sum obtained by the evaluation value calculation unit 133 is supplied to the evaluation value determination unit 134.

  In the evaluation value calculation unit 133, the difference absolute value sum sum obtained by the evaluation value calculation unit 133 is compared with the minimum value min initialized by the minimum value initialization unit 131, and whether or not min> sum is satisfied. Determined. The determination result of the evaluation value determination unit 134 is supplied to the minimum value resetting unit 135 and the motion vector number setting unit 136. In the minimum value resetting unit 135, when the determination result indicates that min> sum, the minimum value min is reset to the sum of absolute differences sum calculated by the evaluation value calculating unit 133. As described above, when the minimum value min is reset by the minimum value resetting unit 135, the minimum value min that is subsequently used by the evaluation value determining unit 134 is the minimum value initialized by the minimum value initializing unit 131. The reset minimum value min is used instead of the value min. Also, in the motion vector number setting unit 136, when the determination result indicates that min> sum, candidate number 1 is set as the motion vector number.

  In the selected candidate number determination unit 137, the evaluation value determination unit 134 determines whether or not candidate number 1 is the same as the total number of candidates after the determination related to candidate number 1 is performed. When it is equal to the total number, the motion vector number finally set by the motion vector number setting unit 136 is output. When the candidate number n is not the same as the total number of candidates, the next candidate number determination unit 138 determines the next candidate number 2, and the candidate number 2 is supplied to the evaluation value initialization unit 132.

  Thereafter, with respect to candidate number 2, the same operation as in the case of candidate number 1 described above is performed. Thereafter, the same operation is repeated until the candidate number n becomes the same value as the total number of candidates. Thereby, in the motion vector number setting part 136, the candidate number with the smallest value is finally set as the motion vector number among the difference absolute value sums corresponding to the candidate numbers corresponding to the total number of candidates. Then, the motion vector number is supplied from the determination unit 137 to the motion vector number / motion vector conversion unit 139. Then, the conversion unit 139 obtains a difference vector between the candidate position of the reference frame Fr corresponding to the motion vector number and the target pixel position of the target frame Fc as a motion vector corresponding to the target pixel position of the target frame Fc.

Next, processing for calculating a motion vector corresponding to the target pixel position in the motion vector calculation unit 105 shown in FIG. 3 will be described with reference to the flowchart shown in FIG.
First, in step ST11, the process is started, and in step ST12, candidate total number information and candidate position information corresponding to the class code CL _n-1 related to the target pixel position of the target block Fc are acquired. In step ST12, block pattern information corresponding to the class code CL _n-1 is acquired.

Thereafter, in step ST14, motion vector information corresponding to the target pixel position of the target block Fc is acquired. That is, in step ST15, the variable min for storing the minimum value of the sum of absolute differences is initialized to the maximum value (2 ⁸ × number of pixels in the block). In step ST16, the candidate number n is initialized to 1.

Next, in step ST17, a variable sum for storing the calculation result of the sum of absolute differences is initialized to zero. In step ST18, the reference block of the block pattern indicated by the block pattern information acquired in step ST13 is set at the target pixel position of the target frame Fc, and the candidate position of the identification number n of the reference frame Fr is set in step ST13. A reference block of the block pattern indicated by the acquired block pattern information is set. Further, in this step ST18, an absolute difference value between corresponding pixel data between the pixel data constituting the base block and the pixel data constituting the reference block is obtained, and the sum of the obtained absolute difference values (absolute difference) is obtained. (Sum of values) is obtained and assigned to the variable sum. Here, the calculation of the sum of absolute differences is performed according to equation (2).
Σi = | Fc (i) −Frn (i) | (2)

  However, Fc (i) is the pixel value of the pixel in the standard block, and Frn (i) is the pixel value of the pixel in the reference block of candidate number n. Further, Σi means a sum operation when i is incremented by 1 from 1 to L (= number of pixels in the block). In step S19, the variable sum that is the calculation result of step S18 is compared with the variable min, and it is determined whether or not the variable sum is smaller than the variable min. If it is determined that the variable sum is smaller than the variable min, the process proceeds to step ST20.

  In step ST20, variable min is replaced with variable sum, and candidate number n is stored as a motion vector number. After the process of step ST20, the process proceeds to step ST21. If it is determined in step ST19 that the variable sum is not smaller than the variable min, the process of step ST20 is skipped and the process proceeds to step ST21.

  In step ST21, it is determined whether or not the candidate number n is the same as the total number of candidates. If the candidate number n is not the same as the total number of candidates, the candidate number n is incremented in step ST22, and then the process returns to step ST17 to perform processing for the next candidate number n. On the other hand, when the candidate number n is the same as the total number of candidates, this means that the processing for each candidate number for the total number of candidates has been completed, and in step ST23, the candidate corresponding to the motion vector number that is finally stored The difference vector between the candidate position of the reference frame Fr and the target pixel position of the target frame Fc is calculated as a motion vector corresponding to the target pixel position of the target frame Fc. And a process is complete | finished by step ST24.

As described above, in the motion vector detection device 100 shown in FIG. 1, when the motion vector corresponding to the target pixel position of the target frame Fc is detected by performing the block matching process, the feature amount related to the target pixel position is detected. The pixel block of the block pattern corresponding to the class code CL _n-1 is used, and the block matching processing by the pixel block of the appropriate block pattern can be performed, and the motion vector corresponding to the target pixel position is accurately Can be detected well.

Further, in the motion vector detection device 100 shown in FIG. 1, for each pixel position of the reference frame Fr, a class code CL _n− as a feature quantity of the pixel position based on a plurality of pixel data corresponding to the pixel position. ₂ is generated, and a table (see FIG. 4) showing the correspondence between the class code CL _n−2 and each pixel position of the reference frame Fr is generated. The class code CL related to the target pixel position of the target frame Fc is generated from this table. A plurality of pixel positions of the reference frame Fr corresponding to _n−1 are acquired, and a plurality of reference blocks of the reference frame Fr that are subjected to matching calculation with the reference block corresponding to the target pixel position of the target frame Fc This is a pixel block corresponding to a plurality of pixel positions. The number of reference blocks to be matched can be reduced, and the amount of calculation can be reduced. It is possible to reduce the width.

Next, generation of a matching block table will be described. FIG. 9 shows the configuration of the matching block table generation device 200.
The table generating apparatus 200 includes an input terminal 201 to which an image signal is input, and frame memories 202 to 204 that store image signals of successive first to third frames.

  When an image signal of a frame from the input terminal 101 is supplied to the frame memory 204 and written, the image signal of the previous frame stored in the frame memory 204 is read out and supplied to the frame memory 203 for writing. It is. When an image signal of a certain frame is supplied from the frame memory 204 to the frame memory 203 and written, the image signal of the previous frame stored in the frame memory 203 is read out and supplied to the frame memory 202. Written.

  In this case, the image signal of # n-2 frame as the first frame is accumulated in the frame memory 202, the image signal of # n-1 frame as the second frame is accumulated in the frame memory 203, and the frame memory 204 In this state, the image signal of the #n frame as the third frame is accumulated.

  The table generation apparatus 200 includes a passing pixel table generation unit 205. The passing pixel table generation unit 205 uses the # n-2 frame image signal stored in the frame memory 202 and the #n frame image signal stored in the frame memory 204, for each block pattern. The pixel position of #n frame corresponding to each pixel position of n-2 frame is detected by performing block matching processing using the pixel block of the block pattern, and each pixel position of # n-1 frame and #n A passing pixel table (see FIG. 10) indicating the correspondence between the pixel positions of the -2 frame and the #n frame is generated.

FIG. 11 shows the configuration of the passing pixel table generation unit 205. The passing pixel table generation unit 205 includes a block pattern setting unit 211 that sets a block pattern, and a target pixel position setting unit 212 that sets a target pixel position (x _n−2 , y _n−2 ) of # n-2 frame. And have.

The passing pixel table generation unit 205 includes a block matching unit 213. The block matching unit 213 sets a reference block corresponding to the block pattern set by the block pattern setting unit 211 corresponding to the target pixel position (x _n-2 , y _n-2 ) in the # n-2 frame. Also, a plurality of reference blocks corresponding to the block pattern set by the block pattern setting unit 211 are set in the search area corresponding to the above noted pixel position (x _n-2 , y _n-2 ) of the #n frame. To do. Then, the block matching unit 213 sequentially obtains the sum of absolute differences between the base block and each of the plurality of reference blocks.

Further, the passing pixel table generation unit 205 determines the pixel position (x _n , y _n ) of the #n frame corresponding to the target pixel position (x _n-2 , y _n-2 ) of the # n-2 frame. A corresponding pixel position determination unit 214 is provided. The corresponding pixel position determination unit 214 determines the center pixel position of the reference block corresponding to the minimum difference absolute value sum among the difference absolute value sums corresponding to each of the plurality of reference blocks obtained by the block matching unit 213. The pixel position (x _n , y _n ) of #n frame is determined.

The passing pixel table generation unit 205 has a table writing unit 215. The table write unit 215, the set pixel of interest position target pixel position setting unit _{212 (x n-2, y} n-2) and determined in the corresponding pixel position determination unit 214 is the pixel position (x _n, y _n ) To determine the corresponding pixel position P in # n−1 (see FIG. 12). In this case, with respect to the horizontal direction, an addition average value of x _n−2 and x _n is obtained, and the pixel position x _n−1 1 obtained by rounding down the addition average value and the round-up process to the addition average value The pixel position x _n−1 2 obtained by performing the above is obtained. Similarly, with respect to the vertical direction, an average value of y _n−2 and y _n is obtained, and a pixel position y _n−1 1 obtained by performing round-down processing on the average value and rounding-up processing to the average value The pixel position y _n-1 2 obtained by performing the above is obtained. Thus, as the corresponding pixel position P in # n−1, (x _n−1 1, y _n−1 1), (x _n−1 1, y _n−1 2), (x _n−1 2, y _n−1 1) and (x _n−1 2, y _n−1 2) are obtained.

Further, the table writing unit 215 has addresses (x _n−1 1, y _n−1 1) and (x _n ) of the area corresponding to the block pattern set by the block pattern setting unit 211 in the passing pixel table memory 205M. ₋₁ 1, y _n−1 2), (x _n−1 2, y _n−1 1), (x _n−1 2, y _n−1 2), the target pixel position (# n−2 frame) x _n-2 , y _n-2 ) and the corresponding pixel position (x _n , y _n ) of the #n frame are written.

  The passing pixel table generation unit 205 includes a selected pixel number determination unit 216 and a selected block pattern number determination unit 217. The determination unit 216 determines whether or not the processing for all the pixel positions in the # n-2 frame has been completed upon receiving the processing end of the table writing unit 215, and determines that the processing for all the pixel positions has not ended. In this case, the target pixel position setting unit 212 is urged to set the next target pixel position.

  When the determination unit 216 determines that the process for all pixel positions has been completed, the determination unit 217 determines whether the process for all block patterns has been completed, and determines that the process for all block patterns has not been completed. If so, the block pattern setting unit 211 is prompted to set the next block pattern.

Next, the operation of the passing pixel table generation unit 205 shown in FIG. 11 will be described.
The block pattern setting unit 211 sets the first block pattern among a plurality of block patterns. Then, the target pixel position setting unit 212 sets the first target pixel position (x _n−2 , y _n−2 ) of the # n−2 frame.

In the block matching unit 213, the reference block corresponding to the target pixel position (x _n-2 , y _n-2 ) in the # n-2 frame and corresponding to the block pattern set in the block pattern setting unit 211 is set, Also, a plurality of reference blocks corresponding to the block pattern set by the block pattern setting unit 211 are set in the search area corresponding to the above noted pixel position (x _n-2 , y _n-2 ) of the #n frame. The The block matching unit 213 sequentially obtains the sum of absolute differences between the base block and each of the plurality of reference blocks.

In this manner, the sum of absolute differences corresponding to the plurality of reference blocks obtained by the block matching unit 213 is supplied to the corresponding pixel position determination unit 214. The corresponding pixel position determination unit 214 detects the minimum difference absolute value sum among the difference absolute value sums corresponding to the plurality of reference blocks, and the center pixel position of the reference block corresponding to the minimum difference absolute value. Is determined as the pixel position (x _n , y _n ) of the #n frame.

In the table writing unit 215, the target pixel position (x _n−2 , y _n−2 ) set by the target pixel position setting unit 212 and the pixel position (x _n , y _n ) determined by the corresponding pixel value determination unit 214. , The corresponding pixel position P at # n−1 is obtained (see FIG. 12). In this case, with respect to the horizontal direction, a pixel position x _n−1 1 obtained by rounding down the addition average value of x _n−2 and x _{n and} a pixel position obtained by rounding up the addition average value x _n-1 2 is obtained. Similarly, with respect to the vertical direction, a pixel position y _n−1 1 obtained by rounding down the addition average value of y _n−2 and y _{n and} a pixel position obtained by rounding up the addition average value y _n-1 2 is obtained. That is, in the table writing unit 215, (x _n−1 1, y _n−1 1), (x _n−1 1, y _n−1 2), (x _n ) are used as the corresponding pixel position P in # n−1. ₋₁ 2, y _n−1 1) and (x _n−1 2, y _n−1 2) are obtained.

In the table writing unit 215, addresses (x _n−1 1, y _n−1 1), (x) of the area corresponding to the block pattern set by the block pattern setting unit 211 in the passing pixel table memory 205M. _(n-1 1, yn _-1 2), ( _xn-1 2, yn _-1 1), ( _xn-1 2, yn _-1 2), the target pixel position of the # n-2 frame (X _n-2 , y _n-2 ) and the corresponding pixel position (x _n , y _n ) of the #n frame are written.

  Upon completion of the processing of the table writing unit 215, the selected pixel number determination unit 216 determines whether or not the processing for all pixel positions in the # n-2 frame has been completed. When the determination unit 216 determines that the processing for all pixel positions has not been completed, the determination unit 216 prompts the target pixel position setting unit 212 to set the next target pixel position.

Therefore, the target pixel position (x _n-2 , y _n-2 ) of the # n-2 frame described above is processed until all pixel positions of the # n-2 frame are set and processed by the target pixel position setting unit 212. The processing corresponding to is repeated. Thereby, each pixel position of the # n−1 frame corresponding to the block pattern and the # n−2 frame corresponding to the block pattern are set in the area corresponding to the block pattern set by the block pattern setting unit 211 in the passing pixel table memory 205M. And a passing pixel table (see FIG. 10) showing the correspondence relationship with the pixel positions of #n frames.

  When the selected pixel number determination unit 216 determines that the processing for all pixel positions in the # n-2 frame has been completed, the selected block pattern number determination unit 217 determines whether the processing for all block patterns has been completed. Is determined. When the determination unit 217 determines that the processing for all block patterns has not been completed, the determination unit 217 prompts the block pattern setting unit 211 to set the next block pattern.

  Therefore, the process of generating the passing pixel table (see FIG. 10) corresponding to the set block pattern is repeated until all the block patterns are set and processed by the block pattern setting unit 211. Thereby, a passing pixel table corresponding to each block pattern set by the block pattern setting unit 211 is generated in the passing pixel table 205M.

Processing for generating a passing pixel table in the passing pixel table generating unit 205 shown in FIG. 11 will be described with reference to a flowchart shown in FIG.
First, in step ST31, the process is started, and in step ST32, the first block pattern is set. In step ST33, the first target pixel position (x _n-2 , y _n-2 ) of the # n-2 frame is set.

Next, in step ST34, a reference block including the target pixel position (x _n-2 , y _n-2 ) corresponding to the block pattern set in step ST32 is set in the # n-2 frame. In step ST35, a plurality of reference blocks corresponding to the block pattern set in step ST32 are included in the search area corresponding to the above-described target pixel position (x _n-2 , y _n-2 ) of the #n frame. Set. In step ST35, block matching processing is performed between the base block and each of the plurality of reference blocks, and the pixel position corresponding to the target pixel position (x _n-2 , y _n-2 ) in the #n frame. (X _n , y _n ) is acquired. In this case, the pixel position at the center of the reference block that minimizes the sum of absolute differences as the matching calculation result is acquired as the pixel position (x _n , y _n ).

Next, used in step ST36, the set noticed pixel position in step _{ST33 (x n-2, y} n-2) and the obtained pixel positions in step ST35 (x _n, y _n) a, # n-1 A corresponding pixel position P in the frame is obtained (see FIG. 12). In this case, with respect to the horizontal direction, a pixel position x _n−1 1 obtained by rounding down the addition average value of x _n−2 and x _{n and} a pixel position obtained by rounding up the addition average value x _n-1 2 is obtained. Similarly, with respect to the vertical direction, a pixel position y _n−1 1 obtained by rounding down the addition average value of y _n−2 and y _{n and} a pixel position obtained by rounding up the addition average value y _n-1 2 is obtained. That is, as the corresponding pixel position P in # n-1, (x _n-1 1, y _n-1 1), (x _n-1 1, y _n-1 2), (x _n-1 2, y _{n −1} 1) and (x _n−1 2, y _n−1 2) are acquired.

In step ST37, the addresses (x _n−1 1, y _n−1 1), (x _n−1 1, y) of the area corresponding to the block pattern set in step ST32 in the passing pixel table memory 205M. _n−1 2), (x _n−1 2, y _n−1 1), (x _n−1 2, y _n−1 2), the target pixel position (x _n−2 , y _n−2 ) and the corresponding pixel position (x _n , y _n ) of the #n frame are written.

Next, in step ST38, it is determined whether or not the processing for all pixel positions in the # n-2 frame has been completed. If it is determined that the processing for all pixel positions has not been completed, the process returns to step ST33 to set the next target pixel position (x _n-2 , y _n-2 ) of the # n-2 frame. Repeat the same process. Thereby, a passing pixel table (see FIG. 10) corresponding to the block pattern is generated in an area corresponding to the block pattern set in step ST32 in the passing pixel table memory 205M.

  If it is determined in step ST38 that the processing for all pixel positions has been completed, the process proceeds to step ST39. In step ST39, it is determined whether or not the processing for all the block patterns has been completed. When it is determined that the processing for all the block patterns has not been completed, the process returns to step ST32, the next block pattern is set, and the same processing as described above is repeated. Thus, a passing pixel table corresponding to each block pattern set in step ST32 is generated in the passing pixel table memory 205M.

  When it is determined in step ST39 that the processing for all the block patterns has been completed, the process proceeds to step ST40 and the processing is terminated.

Returning to FIG. 9, the table generation apparatus 200 also includes an MC image generation unit (motion compensation image generation unit) 206. The MC image generation unit 206 uses the # n-2 frame image signal and the #n frame image signal, and the passing pixel table generated by the passing pixel table generation unit 205, for each block pattern. The pixel positions of the # n-2 frame and the #n frame are obtained from the passing pixel table corresponding to the pixel positions of the -1 frame, and the pixels of the # n-1 frame are obtained based on the pixel values of the obtained pixel positions. A value is obtained, and a motion compensated image signal corresponding to the image signal of # n-1 frame is generated.

FIG. 14 shows the configuration of the MC image generation unit 206. The MC image generation unit 206 includes a block pattern setting unit 221 that sets a block pattern, a target pixel position setting unit 222 that sets a target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame, have.

The MC image generation unit 206 includes a pixel value calculation unit 223 that calculates a pixel value of the target pixel position (x _n−1 , y _n−1 ) set by the target pixel position setting unit 222. The pixel value calculation unit 223 refers to the passing pixel table (see FIG. 10) corresponding to the block pattern set by the block pattern setting unit 221 generated in the passing pixel table memory 205M of the passing pixel table generation unit 205. Thus, the pixel positions of the # n-2 frame and the #n frame corresponding to the target pixel position (x _n-1 , y _n-1 ) of the # n-1 frame are obtained.

Further, the pixel value calculation unit 223 adds and averages the obtained pixel values at the pixel positions of the # n-2 frame and the #n frame, and calculates the target pixel position (x _n−1 , y of the # n−1 frame). _n-1 ) pixel values are calculated. For example, as shown in FIG. 15, when the pixel positions of the # n-2 frame and the #n frame corresponding to the target pixel position P of the # n-1 frame are Q1, Q2, R1, and R2, the target pixel position P Is obtained by averaging the pixel values of the pixel positions Q1, Q2, R1, and R2.

The MC image generation unit 206 has a writing unit 224. The writing unit 224 has a target pixel position (x _n−1 , y _n) set by the target pixel position setting unit 222 in an area corresponding to the block pattern set by the block pattern setting unit 221 in the MC image memory 206M. ₋₁ )), the pixel value of the target pixel position (x _n−1 , y _n−1 ) calculated by the pixel value calculation unit 223 is written.

  The MC image generation unit 206 includes a selected pixel number determination unit 225 and a selected block pattern number determination unit 226. When the determination unit 225 determines that the processing for all pixel positions in the # n-1 frame has been completed upon receiving the processing end of the writing unit 224, and determines that the processing for all pixel positions has not ended. Prompts the target pixel position setting unit 222 to set the next target pixel position.

  When the determination unit 225 determines that the process for all pixel positions has been completed, the determination unit 226 determines whether the process for all block patterns has been completed, and determines that the process for all block patterns has not been completed. If so, the block pattern setting unit 221 is urged to set the next block pattern.

Next, the operation of the passing pixel table generation unit 206 shown in FIG. 14 will be described.
The block pattern setting unit 221 sets the first block pattern among a plurality of block patterns. Then, the target pixel position setting unit 222 sets the first target pixel position (x _n−2 , y _n−2 ) of the # n−1 frame.

Then, the pixel value calculation unit 223 refers to the passing pixel table (see FIG. 10) corresponding to the block pattern set by the block pattern setting unit 221 and refers to the target pixel position (x _n−1 ) of the # n−1 frame. , Y _n-1 ), the pixel positions of the # n-2 frame and the #n frame are obtained, and the pixel values of the obtained pixel positions are added and averaged to obtain the target pixel position of the # n-1 frame. A pixel value of (x _n−1 , y _n−1 ) is calculated. In this way, the pixel value of the target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame obtained by the pixel value calculation unit 223 is supplied to the writing unit 224.

In the writing unit 224, the target pixel position (x _n−1 , y _n−) set by the target pixel position setting unit 222 in the area corresponding to the block pattern set by the block pattern setting unit 221 in the MC image memory 206M. ₁ ), the pixel value at the target pixel position (x _n−1 , y _n−1 ) calculated by the pixel value calculation unit 223 is written.

  Upon completion of the processing of the writing unit 224, the selected pixel number determination unit 225 determines whether or not the processing for all pixel positions in the # n−1 frame has been completed. When the determination unit 225 determines that the processing for all pixel positions has not been completed, the determination unit 225 prompts the target pixel position setting unit 222 to set the next target pixel position.

Therefore, the target pixel position (x _n−2 , y _n−2 ) of the above-described # n−1 frame is processed until processing is performed after all the pixel positions of the # n−1 frame are set by the target pixel position setting unit 222. The processing corresponding to is repeated. Accordingly, a motion compensated image signal corresponding to the block pattern is generated in an area corresponding to the block pattern set by the block pattern setting unit 221 in the MC image memory 205M.

  When the selected pixel number determination unit 225 determines that the processing for all the pixel positions of the # n−1 frame is completed, the selected block pattern number determination unit 226 determines whether the processing for all the block patterns is completed. Is determined. When the determination unit 226 determines that the processing for all the block patterns has not been completed, the determination unit 226 prompts the block pattern setting unit 221 to set the next block pattern.

  Therefore, the process for generating the motion compensated image signal corresponding to the set block pattern is repeated until all the block patterns are set and processed by the block pattern setting unit 221. Thereby, a motion compensation image signal corresponding to each block pattern set by the block pattern setting unit 221 is generated in the MC image memory 206M.

Processing for generating a motion compensated image signal in the MC image generation unit 206 shown in FIG. 14 will be described with reference to the flowchart shown in FIG.
First, processing is started in step ST41, and the first block pattern is set in step ST42. In step ST43, the first target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame is set.

Next, in step ST44, with reference to the passage pixel table corresponding to the block pattern set in step ST42 (see FIG. 10), set # n-1 frame the position of the pixel of interest in step ST43 (x _{n- 1} , y _n-1 ), the pixel positions of the # n-2 frame and the #n frame are obtained. Further, in step ST44, the pixel values at the obtained pixel positions are averaged, and the pixel value at the target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame is calculated.

Next, in step ST45, in the MC image memory 206M, the region corresponding to the block pattern set in step ST42 corresponds to the target pixel position (x _n−1 , y _n−1 ) set in step ST43. The pixel value of the target pixel position (x _n−1 , y _n−1 ) calculated in step ST44 is written in the address.

Next, in step ST46, it is determined whether or not the processing for all pixel positions in the # n-1 frame has been completed. When it is determined that the processing for all pixel positions has not been completed, the process returns to step ST43 to set the next target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame. Repeat the same process. Thereby, a motion compensated image signal corresponding to the block pattern is generated in an area corresponding to the block pattern set in step ST42 in the MC image memory 206M.

  If it is determined in step ST46 that the processing for all pixel positions has been completed, the process proceeds to step ST47. In step ST47, it is determined whether or not the processing for all the block patterns has been completed. When it is determined that the processing for all the block patterns has not been completed, the process returns to step ST42, the next block pattern is set, and the same processing as described above is repeated. Thereby, a motion compensation image signal corresponding to each block pattern set in step ST42 is generated in the MC image memory 206M.

  When it is determined in step ST47 that the processing for all the block patterns has been completed, the process proceeds to step ST48 and the processing is terminated.

Returning to FIG. 9, the table generation apparatus 200 also includes a block evaluation table generation unit 207. The block evaluation table generation unit 207 generates a block evaluation table using the # n−1 frame image signal and the motion compensated image signal generated by the MC image generation unit 206. That is, for each combination of the pixel position and block pattern of the # n-1 frame, the block evaluation table generation unit 207 uses the plurality of pixel data of the # n-1 frame corresponding to the pixel position as a feature amount. While obtaining the class code CL _n−1 , an evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position from each of the image signal of the # n−1 frame and the motion compensated image signal is calculated. By adding the calculated evaluation value to the address value corresponding to the block pattern and the obtained class code CL _n-1 , for each class code, each of the block patterns and the accumulated value of the evaluation values A block evaluation table indicating the correspondence is generated.

FIG. 17 shows the configuration of the block evaluation table generation unit 207. The block evaluation table generation unit 207 includes a target pixel position setting unit 231 that sets a target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame, and a block pattern setting unit 232 that sets a block pattern. And have.

The block evaluation table generation unit 207 has a class code generation pixel block acquisition unit 233. The pixel block acquisition unit 233 acquires a pixel block S for class code generation corresponding to the target pixel position (x _n−1 , y _n−1 ) from the image signal of # n−1 frame (FIG. 18). reference). In this case, the pixel block acquisition unit 233 extracts, as a pixel block S, a plurality of pieces of pixel data located in the vicinity of the target pixel position (x _n−1 , y _n−1 ) from the # n−1 frame image signal. . Here, the plurality of pieces of pixel data are similar to the plurality of pieces of pixel data extracted in order to generate the class code in the class code generation unit 104 of the motion vector detection device 100 (see FIG. 1) described above. There are nine pixels P1 to P9 centered on the pixel of (x _n−1 , y _n−1 ) (see FIG. 2).

The block evaluation table generation unit 207 includes a class code generation unit 234 that generates a class code CL _n-1 from the pixel block S acquired by the pixel block acquisition unit 233. In the class code generation unit 234, each of a plurality of pieces of pixel data constituting the pixel block S is set to 1 as in the case where the class code generation unit 104 of the motion vector detection device 100 described above generates the class code CL _n- 1. It is encoded into bit data to obtain a class code CL _n-1 that is a 9-bit code.

Further, the block evaluation table generation unit 207 obtains an evaluation pixel block T (see FIG. 18) corresponding to the target pixel position (x _n−1 , y _n−1 ) from the image signal of # n−1 frame. And an evaluation pixel block acquisition unit 235. In this case, the pixel block acquisition unit 235 extracts a plurality of pixel data located in the vicinity of the target pixel position (x _n−1 , y _n−1 ) as the pixel block T from the image signal of the # n−1 frame. . When the pixel block T is the same as the pixel block S described above, the pixel block acquisition unit 233 described above can also be used as the pixel block acquisition unit 235.

The block evaluation table generation unit 207 also calculates the target pixel position (x _n−1 , y) from the motion compensation image signal corresponding to the block pattern set by the block pattern setting unit 232 stored in the MC image memory 206M. _n-1 ) includes an evaluation pixel block acquisition unit 236 that acquires the evaluation pixel block K (see FIG. 18). In this case, the pixel block acquisition unit 236 extracts a plurality of pieces of pixel data located in the vicinity of the target pixel position (x _n−1 , y _n−1 ) as a pixel block K from the motion compensated image signal.

  Further, the block evaluation table generation unit 207 includes an evaluation value calculation unit 237 that calculates an evaluation value based on the pixel blocks T and K acquired by the evaluation pixel block acquisition units 235 and 236, respectively. The evaluation value calculation unit 237 obtains an absolute difference value between corresponding pixel data between the plurality of pixel data constituting the pixel block T and the plurality of pixel data constituting the pixel block K, and further calculates the plurality of obtained plural By calculating the sum of the absolute differences, the absolute difference value sum as an evaluation value is obtained.

Further, the block evaluation table generation unit 207 has a table writing unit 238. The table writing unit 238 adds the evaluation value calculated by the evaluation value calculation unit 237 to the value stored in the address (b, CL _n-1 ) of the block evaluation table memory 207M. Here, b indicates a block pattern number set by the block pattern setting unit 232, and CL _n−1 is a class code generated by the class code generation unit 234.

  The block evaluation table generation unit 207 includes a selected block pattern number determination unit 239 and a selected pixel number determination unit 240. The determination unit 239 determines whether or not the processing for all the block patterns has been completed in response to the completion of the processing of the table writing unit 238. When determining that the processing for all the block patterns has not been completed, the determination unit 239 The unit 232 is prompted to set the next block pattern.

  When the determination unit 239 determines that the process for all the block patterns has been completed, the determination unit 240 determines whether or not the process for all pixel positions in the # n-1 frame has been completed, and the process for all pixel positions is completed. If it is determined that the target pixel position is not set, the target pixel position setting unit 231 is prompted to set the next target pixel position.

Next, the operation of the block evaluation table generation unit 207 shown in FIG. 17 will be described.
The target pixel position setting unit 231 sets the first target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame. The block pattern setting unit 232 sets the first block pattern among the plurality of block patterns.

The class code generation pixel block acquisition unit 233 acquires a class code generation pixel block S corresponding to the target pixel position (x _n−1 , y _n−1 ) from the # n−1 frame image signal. (See FIG. 18). The pixel block S is supplied to the class code generation unit 234. In the class code generation unit 234, each of a plurality of pixel data constituting the pixel block S is encoded into 1-bit data, and a class code CL _n-1 that is a 9-bit code is obtained. This class code CL _n-1 is supplied to the table writing unit 238 as address information. The block pattern number b is supplied from the block pattern setting unit 232 to the table writing unit 238 as address information.

The evaluation pixel block acquisition unit 235 acquires an evaluation pixel block T corresponding to the target pixel position (x _n−1 , y _n−1 ) from the image signal of # n−1 frame (see FIG. 18). ). Similarly, the evaluation pixel block acquisition unit 236 calculates the target pixel position (x _n−1) from the motion compensation image signal corresponding to the block pattern set by the block pattern setting unit 232 accumulated in the MC image memory 206M. , Y _n-1 ), the evaluation pixel block K is acquired (see FIG. 18). These pixel blocks T and K are supplied to the evaluation value calculation unit 237, respectively.

In the evaluation value calculation unit 237, an absolute difference value between corresponding pixel data between a plurality of pixel data constituting the pixel block T and a plurality of pixel data constituting the pixel block K is obtained, and the plurality of obtained plural By calculating the sum of absolute difference values, the absolute difference value sum as an evaluation value is determined. The evaluation value of the block pattern set by the block pattern setting unit 232 calculated by the evaluation value calculation unit 237 is supplied to the table writing unit 238. The table writing unit 238 adds the evaluation value calculated by the evaluation value calculation unit 237 to the value stored at the address (b, CL _n-1 ) of the block evaluation table memory 207M.

  Upon completion of the processing of the table writing unit 238, the selected block pattern number determination unit 238 determines whether or not the processing for all the block patterns has been completed. When the determination unit 238 determines that the processing for all the block patterns has not been completed, the determination unit 238 prompts the block pattern setting unit 232 to set the next block pattern.

Therefore, until all block patterns are set by the block pattern setting unit 232 and processing is performed, the evaluation value corresponding to the block pattern described above is calculated by the evaluation value calculating unit 237, and the table writing unit 238 stores the block evaluation table memory. The process of adding the evaluation value to the value stored at the address (b, CL _n-1 ) of 207M is repeatedly performed.

As a result, each block pattern stored in the area corresponding to the class code CL _n−1 as the feature amount related to the target pixel position (x _n−1 , y _n−1 ) in the block evaluation table memory 207M. The corresponding evaluation value is sequentially added to the value.

When the selected block pattern number determination unit 239 determines that the processing for all the block patterns has been completed, the selected pixel number determination unit 240 determines whether the processing for all the pixels in the # n−1 frame has been completed. Is determined. When the determination unit 240 determines that the processing for all the pixels has not been completed, the determination unit 240 sets the next target pixel position (x _n−1 , y _n−1 ) in the target pixel position setting unit 231. Encourage you to.

For this reason, the target pixel position ( _xn−1 , y _n−1 ) is set as the target pixel position (x _n−1 , y _n−1 ) in the target pixel position setting unit 231 until processing is performed. _n-1 , y _n-1 ) is repeatedly performed. As a result, a block evaluation table indicating the correspondence between each of the block patterns and the accumulated value of the evaluation values is generated in the block evaluation table memory 207M for each class code.

Next, processing for generating a block evaluation table in the block evaluation table generation unit 207 shown in FIG. 17 will be described with reference to a flowchart shown in FIG.
First, in step ST51, the process is started. In step ST52, the first target pixel position (x _n−1 , y _n−1 ) of the # n−1 frame is set. In step ST53, a pixel block S for class code generation corresponding to the target pixel position (x _n−1 , y _n−1 ) set in step ST52 is acquired from the image signal of # n−1 frame. (See FIG. 18). Further, in step ST54, each of a plurality of pixel data constituting the pixel block S is encoded into 1-bit data to generate a class code CL _n-1 that is a 9-bit code.

Next, in step ST55, an evaluation pixel block T corresponding to the target pixel position (x _n−1 , y _n−1 ) set in step ST52 is acquired from the image signal of # n−1 frame ( (See FIG. 18).

Next, in step ST56, an initial block pattern is set. This block pattern is specified by a block pattern number b, and initially b = 1. In step ST57, the target pixel position (x _n−1 , y _n) set in step ST52 is calculated from the motion compensated image signal corresponding to the block pattern set in step ST56 stored in the MC image memory 206M. _-1 ) to obtain an evaluation pixel block K (see FIG. 18).

Next, in step ST58, pixel data corresponding to each other between the plurality of pixel data constituting the pixel block T acquired in step ST55 and the plurality of pixel data constituting the pixel block K acquired in step ST57. Is obtained, and the sum of a plurality of obtained difference absolute values is obtained to calculate the sum of absolute differences as an evaluation value. In step ST59, the evaluation value calculated in step ST58 is added to the value stored in the address (b, CL _n-1 ) of the block evaluation table memory 207M.

Next, in step ST60, it is determined whether or not the processing for all block patterns has been completed. When it is determined that the processing for all the block patterns has not been completed, the process returns to step ST56, the next block pattern is set, and the same processing as described above is repeated. As a result, each block pattern stored in the area corresponding to the class code CL _n−1 as the feature amount related to the target pixel position (x _n−1 , y _n−1 ) in the block evaluation table memory 207M. The corresponding evaluation value is sequentially added to the value.

If it is determined in step ST60 that the processing for all block patterns has been completed, the process proceeds to step ST61. In this step ST61, it is determined whether or not the processing for all pixel positions in the # n-1 frame has been completed. If it is determined that the processing for all the pixel positions has not been completed, the process returns to step ST52 to set the next target pixel position (x _n−1 , y _n−1 ). As a result, a block evaluation table indicating the correspondence between each of the block patterns and the accumulated value of the evaluation values is generated in the block evaluation table memory 207M for each class code.

  When it is determined in step ST61 that the processing for all the pixels has been completed, the process proceeds to step ST62 and the processing is terminated.

  Returning to FIG. 9, the table generation apparatus 200 also includes a matching block table generation unit 208 that generates a matching block table (see FIGS. 5 and 6). This table generation unit 208 is used for block matching for each class code based on the block evaluation table of each class code generated by the table generation unit 207 and stored in the block evaluation table memory 207M. A block pattern of a power pixel block is determined, and a matching block table is generated. Then, the table generation unit 208 stores the matching block table generated in this way in the matching block table memory 208.

  In the present embodiment, as described above, since the table generation unit 207 obtains the sum of absolute differences as the evaluation value, the table generation unit 208 has a block in which the cumulative value of the evaluation values is minimized for each class code. The pattern is determined as a block pattern of a pixel block to be used in block matching.

  For example, FIG. 20A shows the contents of a block evaluation table indicating the correspondence between each of the block patterns and the accumulated value of the evaluation values regarding the class code CLa. Regarding this class code CLa, the block pattern of the block pattern number ba having the smallest accumulated value is determined as the block pattern of the pixel block to be used in block matching. Also, for example, FIG. 20B shows the contents of a block evaluation table indicating the correspondence between each block pattern and the accumulated value of the evaluation values regarding the class code CLb. Regarding this class code CLb, the block pattern of block pattern number 1 having the smallest accumulated value is determined as the block pattern of the pixel block to be used in block matching. 20A and 20B, bmax indicates the maximum value of the block pattern number b.

Next, the operation of the matching block table generation device 200 shown in FIG. 9 will be described.
An image signal is input from the input terminal 201, and the frame memories 202 to 204 are in a state in which the image signals of the continuous first to third frames are accumulated. In this case, the image signal of # n-2 frame as the first frame is accumulated in the frame memory 202, the image signal of # n-1 frame as the second frame is accumulated in the frame memory 203, and the frame memory 204 In this state, the image signal of the #n frame as the third frame is accumulated.

  The passing pixel table generation unit 205 uses the # n-2 frame image signal stored in the frame memory 202 and the #n frame image signal stored in the frame memory 204 to generate a passing pixel table, It is stored in the passing pixel table memory 205M. In this case, for each block pattern, by detecting the pixel position of the #n frame corresponding to each pixel position of the # n-2 frame by performing block matching processing using the pixel block of the block pattern, A passing pixel table (see FIG. 2) indicating the correspondence between the pixel positions of the # n-1 frame and the pixel positions of the # n-2 frame and the #n frame is generated.

  The MC image generation unit 206 uses the # n-2 frame image signal and the #n frame image signal, and the pass pixel table corresponding to each block pattern generated by the pass pixel table generation unit 205, and compensates for it. An image signal for use is generated and stored in the MC image memory 206M. In this case, for each block pattern, the pixel positions of the # n-2 frame and the #n frame are obtained from the passing pixel table corresponding to the pixel positions of the # n-1 frame, and the pixel at the obtained pixel position is obtained. By obtaining the pixel value of the # n-1 frame based on the value, a motion compensated image signal corresponding to the image signal of the # n-1 frame is generated.

In the block evaluation table generation unit 207, the block evaluation table is generated using the image signal of the # n-1 frame and the motion compensation image signal corresponding to each block pattern generated by the MC image generation unit 206. It is stored in the block evaluation table memory 207M. In this case, for each combination of the pixel position and block pattern of the # n-1 frame, a class code CL _n-1 as a feature quantity is obtained based on a plurality of pixel data corresponding to the pixel position, and # n- An evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position is calculated from each of the image signal and the motion compensated image signal of one frame, and the calculated evaluation value is obtained as the block pattern and the calculated value. By adding to the value of the address corresponding to the class code CL _n-2 , a block evaluation table indicating the correspondence between each block pattern and the accumulated evaluation value is generated for each class code.

  The matching block table generation unit 208 should be used for block matching for each class code based on the block evaluation table of each class code generated by the table generation unit 207 and stored in the block evaluation table memory 207M. The block pattern of the pixel block is determined, a matching block table is generated, and stored in the matching block table memory 208M.

  As described above, the matching block table generating apparatus 200 shown in FIG. 9 generates a matching block table indicating the correspondence between the class code as the feature quantity and the block pattern. In this case, a block pattern corresponding to a certain class code can detect a motion vector with high accuracy by using a pixel block to be used in block matching for the target pixel position where the class code is obtained as the block pattern. Sought by learning.

  In the above-described embodiment, a case has been described in which a class code obtained by encoding a plurality of pixel data into 1-bit data is obtained as a feature quantity. However, the feature quantity is not limited to this. . The feature amount only needs to represent features of a plurality of pixel data. For example, the feature amount may be obtained from coefficient data obtained by performing orthogonal transformation such as DCT (Discrete Cosine Transform) on a pixel block composed of a plurality of pixel data.

  In the above-described embodiment, the matching calculation is a calculation for obtaining the sum of absolute differences. However, the present invention is not limited to this. This matching calculation only needs to be able to obtain a value indicating the degree of correlation between the first pixel block and the second pixel block as a result. For example, the matching operation may be an operation for obtaining a difference square sum.

  In the above-described embodiment, each process is described as being performed by hardware. However, for example, a computer 300 having a hardware configuration as illustrated in FIG. 21, for example, may be processed by software. is there.

  The computer 300 includes a CPU (Central Processing Unit) 331. An input / output interface 335 is connected to the CPU 31 via a bus 334. A ROM (Read Only Memory) 332 and a RAM (Random Access Memory) 333 are connected to the bus 334.

  The input / output interface 335 includes an input unit 336 including an input device such as a keyboard and a mouse for a user to input an operation command, a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal) for displaying an operation screen and a screen showing a processing result. Communication unit via a network represented by the Internet, including an output unit 337 including a display), a storage unit 338 including a hard disk drive for storing programs and various data, a modem, a LAN (Local Area Network) adapter, and the like. Is connected to the communication unit 339. A drive 340 for reading / writing data from / to a recording medium such as the magnetic disk 341, the optical disk 342, the magneto-optical disk 343, and the semiconductor memory 344 is connected.

  Programs for causing the CPU 331 to execute the above-described series of processes include a magnetic disk 341 (including a flexible disk), an optical disk 342 (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), and magneto-optical. It is supplied to the computer 300 in a state stored in the disk 343 (including MD (Mini Disc)) or the semiconductor memory 344, read by the drive 340, and installed in a hard disk drive built in the storage unit 338. The program installed in the storage unit 338 is loaded from the storage unit 338 to the RAM 333 and executed by a command from the CPU 331 corresponding to a command from the user input to the input unit 336.

  Note that the step of describing the program recorded in the recording medium includes not only the processing performed in time series according to the described order but also the processing executed in parallel or individually even if not necessarily processed in time series. Is also included. Further, the image signal to be processed is provided by, for example, a recording medium, or provided from a network via the communication unit 318, and is stored and processed in a hard disk drive built in the storage unit 338.

Further, in the above-described embodiment, for each pixel position of the reference frame Fr, the class code CL _n-2 is obtained as a feature quantity of the pixel position based on a plurality of pixel data located in the vicinity of the pixel position. Then, a table (see FIG. 4) showing the correspondence between the class code CL _n-2 and each pixel position of the reference frame is generated, and the class code CL _n-1 related to the target pixel position of the target frame Fc is generated from this table. A plurality of pixel positions of the corresponding reference frame Fr are acquired, and a plurality of reference blocks of the reference frame Fr that are subjected to a matching operation with a reference block corresponding to the target pixel position of the target frame Fc As a pixel block corresponding to the above, one that greatly reduces the amount of calculation is shown.

  However, according to the present invention, a plurality of reference blocks of the reference frame Fr that are subjected to a matching operation with the reference block corresponding to the target pixel position of the target frame Fc are within the search area of the reference frame Fr corresponding to the target pixel position of the target frame Fc. The pixel block (see FIG. 23) can be similarly applied, and the motion vector detection accuracy can be improved.

  A motion vector can be accurately detected by block matching processing using a pixel block having a block pattern corresponding to the feature amount related to the target pixel position. For example, a motion vector is used to perform motion compensation in a motion compensated prediction encoding apparatus. Applicable for detecting purposes.

It is a block diagram which shows the structure of the motion vector detection apparatus as embodiment. It is a figure which shows the some pixel example for obtaining a class code. It is a block diagram which shows the structure of a motion vector calculation part. It is a figure which shows the structure of the table which shows the correspondence of a class code and each pixel position of a reference frame. It is a figure which shows an example of a matching block table. It is a figure which shows the other example of a matching block table. It is a block diagram which shows the structure of a motion vector calculating part. It is a flowchart which shows a motion vector calculation process. It is a block diagram which shows the structure of a matching block table production | generation apparatus. It is a figure which shows a passage pixel table. It is a block diagram which shows the structure of a passage pixel table production | generation part. It is a figure for demonstrating the production | generation process of a passing pixel table. It is a flowchart which shows the production | generation process of a passing pixel table. It is a block diagram which shows the structure of MC image generation part. It is a figure for demonstrating MC image generation processing. It is a flowchart which shows the production | generation process of MC image. It is a block diagram which shows the structure of a block evaluation table production | generation part. It is a figure for demonstrating the production | generation process of a block evaluation table. It is a flowchart which shows the production | generation process of a block evaluation table. It is a figure which shows the example of the content of a block evaluation table. It is a figure which shows the hardware constitutions of a computer. It is a block diagram which shows the structure of the conventional motion vector detection apparatus. It is a figure which shows the correspondence of the object frame at the time of detecting a motion vector, and a reference frame. It is a flowchart which shows the conventional motion vector detection process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Motion vector detection apparatus, 101 ... Input terminal, 102, 103 ... Frame memory, 103 ... Class code generation part, 105 ... Motion vector calculation part, 106 ... Output terminal, DESCRIPTION OF SYMBOLS 121 ... ME memory part, 122 ... Matching block table memory, 123 ... Motion vector calculating part, 200 ... Matching block table production | generation apparatus, 201 ... Input terminal, 202-204 ... Frame Memory, 205... Passing pixel table generation unit, 206... MC image generation unit, 207... Block evaluation table generation unit, 208.

Claims (12)

A motion vector detection apparatus that detects a motion vector by performing block matching processing using image signals of first to third frames,
Feature quantity acquisition means for extracting a plurality of pixel data located in the vicinity of the target pixel position from the image signal of the first frame, and obtaining a feature quantity related to the target pixel position based on the plurality of pixel data;
The pixel block related to the target pixel position acquired from the image signal of the first frame, the image signal of the second frame that is the previous frame of the first frame, and the frame that is the subsequent frame of the first frame A block determined based on an evaluation value indicating the degree of correlation obtained from the pixel block corresponding to the pixel position of interest obtained from the motion compensated image signal obtained for each block pattern using the image signal of the third frame Storage means for storing a matching block table indicating a correspondence relationship between the pattern and the feature amount related to the target pixel position;
Block pattern information acquisition means for obtaining block pattern information capable of identifying the positions of pixels constituting the block pattern corresponding to the feature quantity obtained by the feature quantity acquisition means from the matching block table stored in the storage means;
The first pixel block corresponding to the target pixel position of the first frame in the block pattern indicated by the block pattern information obtained by the block pattern information obtaining unit, and the block pattern obtained by the block pattern information obtaining unit A matching calculation means for performing a matching calculation between each of the plurality of second pixel blocks of the second frame of the block pattern indicated by the information;
Motion vector acquisition means for obtaining a motion vector corresponding to a target pixel position of the first frame based on a matching calculation result corresponding to the plurality of second pixel blocks obtained by the matching calculation means ;
The matching block table is
Using the image signal of the first frame and the motion compensated image signal, for each combination of the pixel position and the block pattern of the first frame, the feature is based on a plurality of pixel data located in the vicinity of the pixel position And calculating an evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position from each of the image signal of the first frame and the motion compensated image signal. A block evaluation table showing the correspondence between each of the block patterns and the accumulated value of the evaluation values for each feature amount by adding the value to the address value corresponding to the block pattern and the obtained feature amount Is generated,
Based on the generated block evaluation table for each feature amount, the block pattern of the pixel block to be used in the block matching is determined for each feature amount.
A motion vector detection device that is generated . For each pixel position of the second frame, a feature amount of the pixel position is obtained based on a plurality of pixel data located in the vicinity of the pixel position, and the feature amount and each pixel position of the second frame are Table generating means for generating a table indicating the correspondence relationship between
A pixel position acquisition unit that obtains a plurality of pixel positions of the second frame corresponding to the feature amount obtained by the feature amount acquisition unit from the table generated by the table generation unit;
The motion vector detection device according to claim 1, wherein the plurality of second pixel blocks in the second frame are pixel blocks corresponding to the plurality of pixel positions obtained by the pixel position acquisition unit, respectively.   2. The motion vector detection according to claim 1, wherein the plurality of second pixel blocks of the second frame are pixel blocks in a search area of the second frame corresponding to a target pixel position of the first frame. apparatus.   The motion vector detection device according to claim 1, wherein the feature amount acquisition unit uses, as the feature amount, a code obtained by encoding each of the plurality of pixel data into 1-bit data.   The matching calculation performed by the matching calculation means calculates a difference absolute value between corresponding pixel data between the pixel data constituting the first pixel block and the pixel data constituting the second pixel block, The motion vector detection apparatus according to claim 1, wherein the motion vector detection apparatus is an operation for obtaining a sum of the obtained absolute differences. A motion vector detection method for detecting a motion vector by performing block matching processing using image signals of first to third frames,
A first step of extracting a plurality of pixel data located in the vicinity of the target pixel position from the image signal of the first frame, and obtaining a feature amount related to the target pixel position based on the plurality of pixel data;
The pixel block related to the target pixel position acquired from the image signal of the first frame, the image signal of the second frame that is the previous frame of the first frame, and the frame that is the subsequent frame of the first frame A block determined based on an evaluation value indicating the degree of correlation obtained from the pixel block corresponding to the pixel position of interest obtained from the motion compensated image signal obtained for each block pattern using the image signal of the third frame Block pattern information that can identify the positions of the pixels constituting the block pattern corresponding to the feature amount obtained in the first step from the matching block table indicating the correspondence between the pattern and the feature amount related to the target pixel position. A second step of obtaining
The first pixel block corresponding to the target pixel position of the first frame in the block pattern indicated by the block pattern information obtained in the second step, and the block pattern information obtained in the second step A third step of performing a matching operation between each of the plurality of second pixel blocks of the second frame in the block pattern shown;
And a fourth step of obtaining a motion vector corresponding to the target pixel position of the first frame based on the matching calculation result corresponding to the plurality of second pixel blocks obtained in the third step. ,
The matching block table is
Using the image signal of the first frame and the motion compensated image signal, for each combination of the pixel position and the block pattern of the first frame, the feature is based on a plurality of pixel data located in the vicinity of the pixel position And calculating an evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position from each of the image signal of the first frame and the motion compensated image signal. A block evaluation table showing the correspondence between each of the block patterns and the accumulated value of the evaluation values for each feature amount by adding the value to the address value corresponding to the block pattern and the obtained feature amount Is generated,
Based on the generated block evaluation table for each feature amount, the block pattern of the pixel block to be used in the block matching is determined for each feature amount.
A motion vector detection method that is generated . In order to detect a motion vector by performing block matching processing using the image signals of the first to third frames,
A first step of extracting a plurality of pixel data located in the vicinity of the target pixel position from the image signal of the first frame, and obtaining a feature amount related to the target pixel position based on the plurality of pixel data;
The pixel block related to the target pixel position acquired from the image signal of the first frame, the image signal of the second frame that is the previous frame of the first frame, and the frame that is the subsequent frame of the first frame A block determined based on an evaluation value indicating the degree of correlation obtained from the pixel block corresponding to the pixel position of interest obtained from the motion compensated image signal obtained for each block pattern using the image signal of the third frame Block pattern information that can identify the positions of the pixels constituting the block pattern corresponding to the feature amount obtained in the first step from the matching block table indicating the correspondence between the pattern and the feature amount related to the target pixel position. A second step of obtaining
The first pixel block corresponding to the target pixel position of the first frame in the block pattern indicated by the block pattern information obtained in the second step, and the block pattern information obtained in the second step A third step of performing a matching operation between each of the plurality of second pixel blocks of the second frame in the block pattern shown;
And a fourth step of obtaining a motion vector corresponding to the target pixel position of the first frame based on the matching calculation result corresponding to the plurality of second pixel blocks obtained in the third step. ,
The matching block table is
Using the image signal of the first frame and the motion compensated image signal, for each combination of the pixel position and the block pattern of the first frame, the feature is based on a plurality of pixel data located in the vicinity of the pixel position And calculating an evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position from each of the image signal of the first frame and the motion compensated image signal. A block evaluation table showing the correspondence between each of the block patterns and the accumulated value of the evaluation values for each feature amount by adding the value to the address value corresponding to the block pattern and the obtained feature amount Is generated,
Based on the generated block evaluation table for each feature amount, the block pattern of the pixel block to be used in the block matching is determined for each feature amount.
A computer-readable medium having recorded thereon a program for causing a computer to execute the generated motion vector detection method. In order to detect a motion vector by performing block matching processing using the image signals of the first to third frames,
A first step of extracting a plurality of pixel data located in the vicinity of the target pixel position from the image signal of the first frame, and obtaining a feature amount related to the target pixel position based on the plurality of pixel data;
The pixel block related to the target pixel position acquired from the image signal of the first frame, the image signal of the second frame that is the previous frame of the first frame, and the frame that is the subsequent frame of the first frame A block determined based on an evaluation value indicating the degree of correlation obtained from the pixel block corresponding to the pixel position of interest obtained from the motion compensated image signal obtained for each block pattern using the image signal of the third frame Block pattern information that can identify the positions of the pixels constituting the block pattern corresponding to the feature amount obtained in the first step from the matching block table indicating the correspondence between the pattern and the feature amount related to the target pixel position. A second step of obtaining
The first pixel block corresponding to the target pixel position of the first frame in the block pattern indicated by the block pattern information obtained in the second step, and the block pattern information obtained in the second step A third step of performing a matching operation between each of the plurality of second pixel blocks of the second frame in the block pattern shown;
And a fourth step of obtaining a motion vector corresponding to the target pixel position of the first frame based on the matching calculation result corresponding to the plurality of second pixel blocks obtained in the third step. ,
The matching block table is
Using the image signal of the first frame and the motion compensated image signal, for each combination of the pixel position and the block pattern of the first frame, the feature is based on a plurality of pixel data located in the vicinity of the pixel position And calculating an evaluation value indicating the degree of correlation of the pixel block obtained corresponding to the pixel position from each of the image signal of the first frame and the motion compensated image signal. A block evaluation table showing the correspondence between each of the block patterns and the accumulated value of the evaluation values for each feature amount by adding the value to the address value corresponding to the block pattern and the obtained feature amount Is generated,
Based on the generated block evaluation table for each feature amount, the block pattern of the pixel block to be used in the block matching is determined for each feature amount.
A program for causing a computer to execute the generated motion vector detection method. Feature quantity and block pattern used when detecting a motion vector corresponding to a target pixel position of a predetermined frame by performing block matching processing using a pixel block of a block pattern corresponding to the feature quantity of the target pixel position An apparatus for generating a matching block table showing a correspondence relationship between
Of the three consecutive frames, the first frame, the second frame that is the previous frame of the first frame, and the third frame that is the subsequent frame of the first frame; Storage means for storing the image signal of
Using the second frame image signal and the third frame image signal stored in the storage means, the third frame corresponding to each pixel position of the second frame for each block pattern. The pixel position of the frame is detected by performing block matching processing using the pixel block of the block pattern, and the pixel position of the first frame, the pixel position of the second frame, and the third frame Passing pixel table generating means for generating a passing pixel table showing the correspondence relationship of
Using the image signal of the second frame and the image signal of the third frame stored in the storage means, and the passing pixel table generated by the passing pixel table generating means, for each block pattern, Corresponding to each pixel position of the first frame, pixel positions of the second frame and the third frame are obtained from the passing pixel table, and the first frame is calculated based on the pixel value of the obtained pixel position. Motion compensation image generation means for obtaining a pixel value of the frame and generating a motion compensation image signal corresponding to the image signal of the first frame;
For each combination of pixel position and block pattern of the first frame, using the image signal of the first frame stored in the storage unit and the motion compensated image signal generated by the motion compensated image generation unit In addition, a feature amount is obtained based on a plurality of pixel data located in the vicinity of the pixel position, and a pixel obtained corresponding to the pixel position from each of the image signal of the first frame and the motion compensation image signal By calculating an evaluation value indicating the degree of correlation between the blocks, and adding the calculated evaluation value to the address value corresponding to the block pattern and the obtained feature amount, a block is obtained for each feature amount. A block evaluation table generating means for generating a block evaluation table indicating the correspondence between each of the patterns and the cumulative value of the evaluation values;
Based on the block evaluation table of each feature quantity generated by the block evaluation table generating means, a block pattern of a pixel block to be used in the block matching is determined for each feature quantity, and the matching block table is generated. A matching block table generating device comprising matching block table generating means. Feature quantity and block pattern used when detecting a motion vector corresponding to a target pixel position of a predetermined frame by performing block matching processing using a pixel block of a block pattern corresponding to the feature quantity of the target pixel position A matching block table showing a correspondence relationship between
Of the three consecutive frames, the first frame, the second frame that is the previous frame of the first frame, and the third frame that is the subsequent frame of the first frame; Prepare the image signal of
Using the image signal of the second frame and the image signal of the third frame, for each block pattern, the pixel position of the third frame corresponding to each pixel position of the second frame A passing pixel table that is detected by performing a block matching process using a pixel block of a pattern and indicates the correspondence between the pixel positions of the first frame and the pixel positions of the second frame and the third frame A first step of generating
Using the image signal of the second frame and the image signal of the third frame, and the passing pixel table generated in the first step, each pixel position of the first frame for each block pattern Corresponding to the passing pixel table, the pixel position of the second frame and the third frame are obtained, the pixel value of the first frame is obtained based on the pixel value of the obtained pixel position, A second step of generating a motion compensated image signal corresponding to the image signal of one frame;
Using the image signal of the first frame and the motion-compensated image signal generated in the second step, each pixel position and block pattern combination of the first frame is positioned in the vicinity of the pixel position. An evaluation value indicating a degree of correlation of pixel blocks obtained from the image signal of the first frame and the motion compensation image signal in correspondence with the pixel position, while obtaining a feature amount based on a plurality of pixel data And adding the calculated evaluation value to the address value corresponding to the block pattern and the calculated feature value, for each feature value, the accumulated value of each of the block patterns and the evaluation value. A third step of generating a block evaluation table indicating a correspondence relationship with
Based on the block evaluation table for each feature value generated in the third step, a block pattern of a pixel block to be used in the block matching is determined for each feature value, and the matching block table is generated. 4. A matching block table generation method comprising four steps. Feature quantity and block pattern used when detecting a motion vector corresponding to a target pixel position of a predetermined frame by performing block matching processing using a pixel block of a block pattern corresponding to the feature quantity of the target pixel position In order to generate a matching block table showing the correspondence relationship with
Of the three consecutive frames in time, the first frame, the second frame that is the previous frame of the first frame, and the third frame that is the subsequent frame of the first frame Prepare the image signal,
Using the image signal of the second frame and the image signal of the third frame, for each block pattern, the pixel position of the third frame corresponding to each pixel position of the second frame A passing pixel table that is detected by performing a block matching process using a pixel block of a pattern and indicates the correspondence between the pixel positions of the first frame and the pixel positions of the second frame and the third frame A first step of generating
Using the image signal of the second frame and the image signal of the third frame, and the passing pixel table generated in the first step, each pixel position of the first frame for each block pattern Corresponding to the passing pixel table, the pixel position of the second frame and the third frame are obtained, the pixel value of the first frame is obtained based on the pixel value of the obtained pixel position, A second step of generating a motion compensated image signal corresponding to the image signal of one frame;
Using the image signal of the first frame and the motion-compensated image signal generated in the second step, each pixel position and block pattern combination of the first frame is positioned in the vicinity of the pixel position. An evaluation value indicating a degree of correlation of pixel blocks obtained from the image signal of the first frame and the motion compensation image signal in correspondence with the pixel position, while obtaining a feature amount based on a plurality of pixel data And adding the calculated evaluation value to the address value corresponding to the block pattern and the calculated feature value, for each feature value, the accumulated value of each of the block patterns and the evaluation value. A third step of generating a block evaluation table indicating a correspondence relationship with
Based on the block evaluation table for each feature value generated in the third step, a block pattern of a pixel block to be used in the block matching is determined for each feature value, and the matching block table is generated. A computer-readable medium having recorded thereon a program for causing a computer to execute the matching block table generation method comprising the steps of 4. Feature quantity and block pattern used when detecting a motion vector corresponding to a target pixel position of a predetermined frame by performing block matching processing using a pixel block of a block pattern corresponding to the feature quantity of the target pixel position In order to generate a matching block table showing the correspondence relationship with
Of the three consecutive frames, the first frame, the second frame that is the previous frame of the first frame, and the third frame that is the subsequent frame of the first frame; Prepare an image signal of
Using the image signal of the second frame and the image signal of the third frame, for each block pattern, the pixel position of the third frame corresponding to each pixel position of the second frame A passing pixel table that is detected by performing a block matching process using a pixel block of a pattern and indicates the correspondence between the pixel positions of the first frame and the pixel positions of the second frame and the third frame A first step of generating
Using the image signal of the second frame and the image signal of the third frame, and the passing pixel table generated in the first step, each pixel position of the first frame for each block pattern Corresponding to the passing pixel table, the pixel position of the second frame and the third frame are obtained, the pixel value of the first frame is obtained based on the pixel value of the obtained pixel position, A second step of generating a motion compensated image signal corresponding to the image signal of one frame;
Using the image signal of the first frame and the motion-compensated image signal generated in the second step, each pixel position and block pattern combination of the first frame is positioned in the vicinity of the pixel position. An evaluation value indicating a degree of correlation of pixel blocks obtained from the image signal of the first frame and the motion compensation image signal in correspondence with the pixel position, while obtaining a feature amount based on a plurality of pixel data And adding the calculated evaluation value to the address value corresponding to the block pattern and the calculated feature value, for each feature value, the accumulated value of each of the block patterns and the evaluation value. A third step of generating a block evaluation table indicating a correspondence relationship with
Based on the block evaluation table for each feature value generated in the third step, a block pattern of a pixel block to be used in the block matching is determined for each feature value, and the matching block table is generated. A program for causing a computer to execute a matching block table generation method comprising:

Images