JP4626158B2 - Motion vector detection apparatus, motion vector detection method, and computer program - Google Patents

Description

  The present invention relates to a motion vector detection device, a motion vector detection method, and a computer program. More specifically, the present invention relates to a motion vector detection apparatus, a motion vector detection method, and a computer program that execute a motion vector detection process from moving image data.

  With the recent enhancement of information processing devices and communication terminals, high-speed communication infrastructure, and the spread of high-density recording media such as DVDs and Blu-ray discs, Storage and reproduction of moving image data using a high-density recording medium has been actively performed. Along with this situation, there is a demand for improved efficiency and high speed in data processing for moving image data, for example, encoding processing.

  Motion-compensated image coding in high-efficiency coding of moving image data, moving object detection processing, speed detection processing, etc. in a visual sensor of a traffic monitoring system or autonomous vehicle, movement of each object included in the image data Therefore, a process for detecting the direction and magnitude (speed) of the image, that is, a motion vector detection process is required.

  For example, as an example of motion compensation type image encoding processing, an MPEG (Moving Picture Coding Experts Group) method, which is an international standard method for high-efficiency video encoding, has been proposed. This MPEG method is a DCT (Discrete). This is a method of performing coding that combines (Coscine Transform) and motion compensated prediction coding. In motion-compensated predictive coding, a correlation between image signal levels in a current frame constituting moving image data and a continuous frame of the immediately preceding frame is detected, and a motion vector is obtained based on the detected correlation. Efficient encoding is achieved by performing a motion image correction process based on the motion vector.

As one of motion vector detection methods, a block matching method is known. The outline of the block matching method will be described with reference to FIG. For example, a current frame [F _t ] 20 at time (t) and a previous frame [F _t-1 ] 10 at time (t−1) shown in the figure are extracted. . One screen of the frame image is divided into small areas (hereinafter referred to as blocks) composed of a plurality of pixels, m pixels × n lines.

The current frame [F _t ] 20 is set as a reference frame, and the check block By21 of the reference frame is moved within a predetermined search area 22, and has the smallest pixel value difference from the reference block Bx11 of the previous frame [F _t-1 ] 10. That is, the test block having the best pixel value (the most correlated) is detected. It is estimated that the reference block Bx11 of the previous frame [F _t−1 ] 10 has moved to the position of the highly correlated test block detected from the current frame [F _t ] 20. Based on the vector indicating the estimated motion, a motion vector of each pixel is obtained. As described above, the block matching method is a method for obtaining a motion vector by performing a correlation determination (matching determination) process between frames in a predetermined block (m × n) unit.

  In the block matching method, a motion vector is obtained for each block. As an evaluation value representing the correlation of each block, that is, the degree of coincidence, for example, values of pixels at the same spatial position between a plurality of pixels in the reference block Bx and a plurality of pixels in the inspection block By The frame difference is obtained by subtracting the frame difference, and the sum of the absolute values of the calculated frame differences is added. Alternatively, the sum of squares of frame differences can be used.

  However, since the block matching method described above is a full search for comparing all data in the search area, the number of comparisons required for detection is very large, and there is a drawback that it takes time for motion detection.

  In addition, when a moving part and a stationary part are included in a block, it cannot be said that the movement detected in units of blocks corresponds to the movement of individual pixels in the block. Such a problem can be adjusted by setting the block size. For example, if the block is enlarged, the problem of multiple motions in the block is likely to occur in addition to an increase in the amount of calculation. On the other hand, when the block size is reduced so that a plurality of motions are not included in the block, the matching determination area is reduced, which causes a problem that the accuracy of motion detection is lowered. That is, when block matching is performed, there is a high possibility that many inspection blocks similar to the reference block, that is, many inspection blocks having a high correlation with the reference block appear. This is because those not caused by motion are included, and the accuracy of motion detection decreases. For example, when a character telop moves in a horizontal or vertical direction, the influence of a repetitive pattern tends to appear. In the case of kanji character patterns, the same character often has the same pattern when divided into small parts. Therefore, there is a problem that it is difficult to obtain an accurate movement when a plurality of movements are mixed in a block.

  The applicant of the present patent application proposes a motion vector detection method and a detection apparatus that can detect a motion vector for each pixel without increasing the amount of calculation and prevent erroneous detection in Patent Document 1, for example. ing.

The point of the motion vector detection process disclosed in Patent Document 1 is that a motion vector is not determined by calculating an evaluation value for each pixel or block, but as a first step process, a plurality of pixels are added to one of the frames. Set multiple blocks, set representative points of each block, examine the correlation between each representative point and each pixel in the search area set in the other frame, calculate the evaluation value based on the correlation information, and evaluate An evaluation value table as correlation information based on values is formed, and a plurality of candidate vectors are extracted from the evaluation value table. Next, as a process of the second step, a candidate vector that seems to be the best for each pixel is selected from the extracted candidate vectors and associated with them, and determined as a motion vector for each pixel. in this way,
Evaluation value table generation processing,
Candidate vector selection processing based on the evaluation value table,
Process for associating an optimal one of a plurality of candidate vectors as a motion vector corresponding to each pixel This is a method for obtaining a motion vector for each pixel by the above process. This method is hereinafter referred to as a candidate vector method.

  The advantage of the motion vector detection processing by the candidate vector method is that the calculation amount can be reduced by extracting a limited number of candidate vectors based on the evaluation value table. In addition, even in a boundary portion of a subject where erroneous detection of a motion vector is likely to occur, it may be possible to determine the best motion vector corresponding to each pixel from candidate vectors narrowed down in advance. Conventionally, a method has been adopted in which a motion vector of each pixel is calculated as a pixel difference between frames as an evaluation value, and a full search process is performed to obtain an evaluation value for all pixels in the frame. Since it is possible to determine the best motion vector corresponding to each pixel from the candidate vectors that have been narrowed down in advance, the probability that the same evaluation value will occur is reduced compared to the full search process, and erroneous detection is prevented. Is prevented.

However, block matching processing is performed when determining the final motion vector corresponding to each pixel based on this candidate vector. The block matching process is a process for setting a pixel in the vicinity of the target pixel in the previous frame as a block and detecting the overall correlation of a plurality of pixels included in the block. In order to perform accurate motion vector determination processing by applying block matching, it is necessary to increase the block size and perform accurate correlation determination. When the block size is increased, the amount of calculation of evaluation values such as the sum of absolute differences executed as the calculation for correlation calculation increases, the efficiency decreases, and the memory for holding pixel values also increases. This causes a problem that the hardware scale becomes large.
JP 2001-61152 A

  The present invention has been made in view of the above problems. For example, in motion vector detection processing based on a candidate vector method, block matching is applied when determining a motion vector corresponding to each pixel from a plurality of candidate vectors. It is an object of the present invention to provide a motion vector detection device, a motion vector detection method, and a computer program that enable accurate motion vector determination.

The first aspect of the present invention is:
A motion vector detection device for detecting a motion vector from moving image data;
An evaluation value table forming unit that generates an evaluation value table based on pixel value correlation information between different frames on the time axis;
A candidate vector extraction unit that detects a candidate vector of a motion vector corresponding to a frame constituent pixel of moving image data based on the evaluation value table;
A motion vector determination unit that determines a motion vector corresponding to each pixel from the candidate vector,
The motion vector determination unit
The feature pixel is extracted from the region near the pixel of interest associated motion vector, based on the correlation determination process based on the feature pixel, Ri configuration der to determine the motion vector corresponding to the target pixel,
The difference between the pixel value of the pixel of interest associated with the motion vector and the pixel value of the designated pixel indicated by the candidate vector is calculated, and the difference is compared with a predetermined threshold value. The following correlation determination process for only the selection candidate vector selected by the provisional determination process that executes the process of selecting the candidate vector indicating the designated pixel as a highly correlated candidate vector:
A feature pixel is extracted from the neighborhood region of the target pixel to which the motion vector is associated, the relative positional relationship between the target pixel and the feature pixel, and the relative relationship between the instruction pixel of the selection candidate vector and the feature pixel in the vicinity of the instruction pixel. A motion vector detection apparatus is characterized in that a motion vector corresponding to a pixel of interest is determined based on a correlation determination process based on a positional relationship .

Furthermore, in one embodiment of the motion vector detection device of the present invention, the motion vector determination unit extracts, as a feature pixel, a pixel having a maximum pixel value difference absolute value from an adjacent pixel from a neighboring region of the pixel of interest associated with the motion vector. It is characterized by doing.

Furthermore, in an embodiment of the motion vector detection device of the present invention, the motion vector determination unit extracts a plurality of feature pixels from a neighborhood region of the pixel of interest associated with the motion vector.

Furthermore, in one embodiment of the motion vector detection device of the present invention, the motion vector determination unit calculates a maximum pixel and a minimum of two pixels having a pixel value difference from the pixel of interest from a neighboring region of the pixel of interest associated with the motion vector. and extracting as a feature pixel.

Furthermore, the second aspect of the present invention provides
A motion vector detection method for detecting a motion vector from moving image data,
An evaluation value table forming step for generating an evaluation value table based on pixel value correlation information between different frames on the time axis;
A candidate vector extraction step of detecting a candidate vector of a motion vector corresponding to a frame constituent pixel of moving image data based on the evaluation value table;
A motion vector determining step for determining a motion vector corresponding to each pixel from the candidate vector,
The motion vector determination step includes:
The feature pixel is extracted from the region near the pixel of interest associated motion vector, based on the correlation determination process based on the feature pixel, Ri step der to determine the motion vector corresponding to the target pixel,
The difference between the pixel value of the pixel of interest associated with the motion vector and the pixel value of the designated pixel indicated by the candidate vector is calculated, and the difference is compared with a predetermined threshold value. The following correlation determination process for only the selection candidate vector selected by the provisional determination process that executes the process of selecting the candidate vector indicating the designated pixel as a highly correlated candidate vector:
A feature pixel is extracted from the neighborhood region of the target pixel to which the motion vector is associated, the relative positional relationship between the target pixel and the feature pixel, and the relative relationship between the instruction pixel of the selection candidate vector and the feature pixel in the vicinity of the instruction pixel. A motion vector detection method is characterized in that a motion vector corresponding to a pixel of interest is determined based on a correlation determination process based on a positional relationship .

Furthermore, the third aspect of the present invention provides
In the motion vector detection device, a computer program for executing processing for detecting a motion vector from moving image data,
An evaluation value table forming step for generating an evaluation value table based on pixel value correlation information between different frames on the time axis;
A candidate vector extraction step of detecting a candidate vector of a motion vector corresponding to a frame constituent pixel of moving image data based on the evaluation value table;
A program for executing a motion vector determination step for determining a motion vector corresponding to each pixel from the candidate vector ,
The motion vector determination step includes:
The feature pixel is extracted from the region near the pixel of interest associated motion vector, based on the correlation determination process based on the feature pixel, Ri step der to make determining the motion vector corresponding to the target pixel,
The difference between the pixel value of the pixel of interest associated with the motion vector and the pixel value of the designated pixel indicated by the candidate vector is calculated, and the difference is compared with a predetermined threshold value. The following correlation determination process for only the selection candidate vector selected by the provisional determination process for executing the process of selecting the candidate vector indicating the designated pixel as a highly correlated candidate vector:
A feature pixel is extracted from the vicinity region of the target pixel to which the motion vector is associated, the relative positional relationship between the target pixel and the feature pixel, and the relative relationship between the instruction pixel of the selection candidate vector and the feature pixel in the vicinity of the instruction pixel. A computer program is characterized in that a motion vector corresponding to a pixel of interest is determined based on a correlation determination process based on a positional relationship .

  The computer program of the present invention is, for example, a storage medium or communication medium provided in a computer-readable format to a computer system capable of executing various program codes, such as a CD, FD, or MO. It is a computer program that can be provided by a recording medium or a communication medium such as a network. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of the present invention, in the motion vector detection process from moving image data, in the process of selecting and determining the motion vector corresponding to each pixel from the plurality of candidate vectors, the feature pixel is detected from the vicinity region of the target pixel to which the motion vector is associated. Since it is configured to extract and perform correlation determination based on the position information of the feature pixel or pixel value information and perform motion vector determination processing, it is not necessary to apply block matching processing, and for correlation calculation It is possible to reduce both the evaluation value calculation such as the difference absolute value sum operation as the calculation, improve the processing efficiency, and also reduce the memory for holding the pixel value, thereby reducing the hardware scale. Realized.

Hereinafter, a motion vector detection device, a motion vector detection method, and a computer program according to the present invention will be described in detail with reference to the drawings. The description will be made in the order of the following items.
1. 1. Generation of evaluation value table by representative point matching, overview of candidate vector method 2. Overall configuration of motion vector detection device and motion vector detection processing procedure Details of motion vector decision processing based on feature pixels

[1. Generation of evaluation value table by representative point matching, overview of candidate vector method]
Hereinafter, in the motion vector detection process described below, a representative point matching method is applied. The representative point matching method is disclosed in Japanese Patent No. 2083999, which was previously filed by the applicant of this patent and has been patented. In other words, the motion vector detection process described below is a process example using the candidate vector method (disclosed in Japanese Patent Application Laid-Open No. 2001-61152) described in the background section and using the representative point matching method. is there.

  In the following description, an example in which one frame constituting moving image data is set as one screen and a motion vector detection process in a frame is performed by a mutual verification process between the respective screens (frames) will be described. The present invention is not limited to such a processing example, and can be applied to, for example, a case where one field divided into one frame is treated as one screen and motion vector detection processing is performed in units of one field.

  The processing example described below will be described mainly as a processing example for moving image data based on a television signal. However, the present invention is also applicable to various moving image data other than a television signal. . Also, when a video signal is a processing target, either an interlace signal or a non-interlace signal may be used.

  With reference to FIGS. 2 to 4, an evaluation value table creation process based on the representative point matching method applied in the motion vector detection process according to the present invention, a candidate vector extraction process based on the evaluation value table, and each of the extracted candidate vector based processes An outline of the motion vector setting process corresponding to the pixel will be described.

For example, a current frame [F _t ] 80 at time (t) and a previous frame [F _t-1 ] 70 at time (t−1) shown in FIG. .

For example, the current frame [F _t ] 80 is a reference frame, the previous frame [F _t−1 ] 70 is divided into a plurality of blocks of m pixels × n lines, and a representative point Ry representing each block is set. The representative points of each block are, for example,
a. The pixel value at the center of the block,
b. The average pixel value of all the pixels in the block,
c. An intermediate value of the pixel values of all the pixels in the block,
Etc., pixel values representing blocks are associated with each other.

In the representative point matching method, a predetermined search area 81 is set in the current frame [F _t ] 80 corresponding to the representative point Ry 71 of the block set in the previous frame [F _t−1 ] 70, and the set search area 81 is set. The pixel value of each pixel included in the pixel is compared with the pixel value of the representative point Ry71. The search area 81 is set as a pixel area of p pixels × q lines, for example.

That is, the representative point pixel value of any one of the above a to c and the pixel value of each pixel in the search area 81 are compared and verified to calculate an evaluation value (for example, frame difference or determination result). The evaluation value is calculated for each shift (each pixel position) in the search area 81. A search area is set in the current frame [F _t ] 80 corresponding to each representative point of the block set in the previous frame [F _t−1 ] 70, and the representative point pixel value and each pixel in the corresponding search area are set. Evaluation values based on the comparison of the pixel values are acquired and integrated for all representative points in one screen. Therefore, an evaluation value table having the same size as the search area is formed.

The search area corresponding to each representative point may be set so that a part thereof overlaps with an adjacent search area as shown in FIG. In the example shown in FIG. 3A, the search area is set to an area of p × q (pixels or lines). For example, the representative point 71a of the block set in the previous frame [F _t−1 ] 70 is set. The corresponding search area 81a and the search area 81b corresponding to the representative point 71b of the block set in the previous frame [F _t-1 ] 70 are set as an overlapping area.

  In this way, the search area is set in association with each representative point, and each representative point is compared with the pixels in the search area set in correspondence with the representative point to obtain a comparison value. Based on this, for example, the higher the degree of correlation (the higher the degree of matching of pixel values), the higher the evaluation value is set, and the evaluation values corresponding to the constituent pixels of each search area are set.

The evaluation values in each search area are integrated as shown in FIG. 3B, and as a result, an evaluation value table 90 as shown in FIG. 3C is generated. The evaluation value table is set to the representative points Ry1 to Ryn of each block set to, for example, n blocks set in the previous frame [F _t-1 ] 70 and the current frame [F _t ] 80 which is a reference frame. An evaluation value based on a comparison with each pixel in the search area corresponding to each representative point Ry1 to n, for example, an integrated value of a threshold determination result of an absolute difference value, and an evaluation value having the same size as the search area Formed as a table.

  In the evaluation value table 90, a peak (extreme value) occurs when the correlation between the pixel value and the representative point at each shift position (i, j) in the search area is high. The peak appearing in the evaluation value table corresponds to the movement of the display object on the screen of the moving image data.

  For example, when the entire screen (frame) moves in the same manner, the evaluation value table having the same size as the search area (p × q) has 1 in the position corresponding to the end point of the vector having the movement direction and distance. Two peaks appear. If there are two objects with different movements in the screen (frame), two vectors having different movement directions and distances in the evaluation value table having the same size as the search area (p × q). Two peaks appear at two positions corresponding to the end point of. When there is a stationary part, a peak corresponding to the stationary part also appears.

Based on such peaks appearing in the evaluation value table, motion vector candidates (candidate vectors) in the previous frame [F _t−1 ] 70 and the current frame [F _t ] 80 which is a reference frame are obtained.

  After extracting a plurality of candidate vectors based on the peak appearing in the evaluation value table, for each pixel of the frame, select the most suitable candidate vector from the extracted candidate vectors, and use it as a motion vector corresponding to each pixel. Set.

  A motion vector setting process corresponding to each pixel to be executed based on the extraction candidate vector will be described with reference to FIG.

In FIG. 4A, the center pixel 91 represents one pixel of the previous frame [F _t-1 ]. This pixel has, for example, a luminance value (α). Also, it is assumed that a plurality of candidate vectors have already been extracted based on the peaks that have appeared in the evaluation value table, and these candidate vectors are candidate vectors A, B, and C shown in the figure. It is determined that one pixel 91 of the previous frame [F _t−1 ] moves according to any of these candidate vectors and is displayed at a position corresponding to the pixel of the current frame [F _t ].

In FIG. 4A, a pixel a95, a pixel b96, and a pixel c97 are pixel positions estimated as movement destinations from one pixel 91 in the previous frame [F _t−1 ] based on candidate vectors A, B, and C. The pixels of the current frame [F _t ] are shown. The correlation between the pixel value of the block including these three pixels and the pixel value of the block including the pixel 91 is determined by the block matching process, the pair having the highest correspondence is selected, and the candidate set in the selected pair Let the vector be the motion vector of the pixel 91.

The reason for applying the block matching process is that the following problem occurs when the correspondence of only one pixel is inspected.
(1) For n candidate vectors in a reference frame and n candidate vectors in a reference frame, there are a plurality of pixels having the same or similar correlation with the pixels ahead of the motion vector, and which motion vector is the correct motion vector Alternatively, it is difficult to make a determination only with a pixel difference absolute value (hereinafter referred to as an MC residual) representing a high degree of correlation.
(2) If the motion vector is correct, the MC residual with the pixel ahead of the motion vector is considered to be the smallest, but in actuality, the image data is different from the pixel behind the correct motion vector due to the influence of noise or the like. The MC residual is not necessarily the smallest among the candidate vectors.

  Because of these problems, it is not possible to make a judgment based on the MC residual of only one pixel. Therefore, using a plurality of pixels having a block size, the pixels in the block centering on the pixel of interest and the candidate vector destination block Check the correlation of the pixels.

A specific block matching process will be described with reference to FIG. As shown in FIG. 4B, the sum of absolute differences (SAD) is calculated according to the following equation as an index value for correlation of a plurality of pixels included in the block indicated by the candidate vector.

Here, F _t-1 (x, y) is the luminance level of the previous frame of interest, F _t (x + v _x , y + v _y ) is the luminance level of the motion vector destination in the reference current frame, and M × N is used for evaluation. Block size.

For example, a method of using a candidate vector (v _x , v _y ) having the smallest difference absolute value sum (SAD) calculated by the above formula as a motion vector of the pixel of interest is applied. However, in motion vector allocation using blocks such as block matching, it is necessary to increase the block size in order to improve motion vector allocation performance, but to calculate evaluation values such as the sum of absolute differences Causes a problem that the amount of calculation increases, and thus the hardware scale increases.

  The motion vector detection processing configuration based on representative point matching that solves these problems is described in [3. Details of motion vector determination processing based on feature pixels] will be described in detail.

  Note that, as described above, the representative point matching is limited because it is possible to set a representative point representing each block, calculate an evaluation value for only the set representative point, and set a candidate vector. By calculating the evaluation value of only a few representative points, the amount of calculation required for calculating the evaluation value can be reduced, and high-speed processing is possible.

[2. Overall configuration of motion vector detection apparatus and motion vector detection processing procedure]
FIG. 5 shows the configuration of an embodiment of the motion vector detection apparatus of the present invention for executing the motion vector detection process, and the processing sequence of the motion vector detection process is shown in the flowchart of FIG.

  As shown in FIG. 5, the motion vector detection apparatus includes an evaluation value table forming unit 101, a candidate vector extraction unit 102, a motion vector determination unit 103, and a control unit (controller) 104. The evaluation value table forming unit 101 inputs image data to be subjected to motion vector detection processing via an input terminal, and generates an evaluation value table. The input image is image data obtained by raster scanning, for example. The image data includes, for example, a luminance signal in a component signal of a digital color video signal.

  The input image data is supplied to the evaluation value table forming unit 101, and an evaluation value table having the same size as the search area is formed based on the representative point matching method described above. This is the process of step S101 in the flow of FIG.

  The candidate vector extraction unit 102 extracts a plurality of motion vectors as candidate vectors in one screen from the evaluation value table generated by the evaluation value table forming unit 101. That is, as described above, a plurality of candidate vectors are extracted based on the peaks that appear in the evaluation value table. This is the process of step S102 in the flow of FIG.

  The motion vector determination unit 103 determines, for each pixel in all frames, the correlation between pixels associated with the candidate vector by block matching or the like for each of the plurality of candidate vectors extracted by the candidate vector extraction unit 102. Then, a candidate vector that connects the blocks having the highest correlation is set as a motion vector corresponding to the pixel. This is the process of step S103 in the flow of FIG. This process is the process described above with reference to FIG.

  The motion vector determination unit 103 performs an optimal vector selection process from candidate vectors for all pixels included in one screen (frame), determines a motion vector corresponding to each pixel, and outputs the determined motion vector To do. As described above, block matching can be applied when determining a motion vector corresponding to each pixel from candidate vectors. However, in the present invention, block matching is not applied, and a motion vector based on feature pixel position information is used. Execute the determination process. Details of this configuration will be described later.

  The control unit 104 performs processing timing control in the evaluation value table forming unit 101, candidate vector extraction unit 102, motion vector determination unit 103, storage of intermediate data in a memory, output processing control, and the like.

  The above is the outline of the evaluation value table generation executed in the motion vector detection device based on representative point matching and the motion vector detection processing to which the candidate vector method is applied.

[3. Details of motion vector determination processing based on feature pixels]
As described with reference to FIG. 5, the motion vector detection device to which representative point matching is applied includes the evaluation value table forming unit 101, the candidate vector extracting unit 102, and the motion vector determining unit 103, and the evaluation value table forming unit 101, an evaluation value table is generated based on the input image data, a candidate vector extraction unit 102 extracts a plurality of candidate vectors from the evaluation value table, and a motion vector determination unit 103 further extracts each of the plurality of candidate vectors from the plurality of candidate vectors. A process of determining a motion vector corresponding to a pixel is performed.

  However, as described above, in order to improve the determination accuracy of the motion vector in the motion vector determination unit 103, it is necessary to increase the applied block size in block matching. When the block size is increased, it is necessary to store a large number of pixel values included in the block in the memory, and in the calculation process of the sum of absolute differences described above, an arithmetic process based on a large number of pixel values included in the block Must be performed. As a result, it is necessary to increase the hardware scale, and there is a problem that the processing efficiency is lowered.

  Therefore, in the present invention, the motion vector determination process based on the feature pixel position information is executed without applying block matching to the motion vector determination process executed by the motion vector determination unit 103. Details of the processing of the motion vector determination unit will be described below.

Hereinafter, regarding the motion vector determination processing configuration based on the feature pixel position information,
Two processing configurations of (1) motion vector determination processing by the two-point matching method and (2) motion vector determination processing by the three-point matching method will be sequentially described.

(1) Motion vector determination processing by the two-point matching method First, motion vector determination processing by the two-point matching method will be described. FIG. 7 shows a detailed configuration of the motion vector determination unit of the motion vector detection device according to the present embodiment. The motion vector determination unit 200 illustrated in FIG. 7 corresponds to the motion vector determination unit 103 of the motion vector detection device illustrated in FIG.

  The motion vector determination unit 200 illustrated in FIG. 7 receives a plurality of candidate vector information determined based on the above-described evaluation value table from the candidate vector extraction unit 102 illustrated in FIG. 5 and determines a motion vector to be associated with each pixel. Execute the process.

  As illustrated in FIG. 7, the motion vector determination unit 200 according to the present embodiment includes a pixel correlation determination unit 210, a temporary determination unit 221, a gate 222, an additional information calculation unit 250, and a feature pixel position correlation determination unit 230. The pixel correlation determination unit 210 includes a current frame memory 211, a previous frame memory 212, a pixel value difference calculation unit 213, and an absolute value calculation unit 214. The feature pixel position correlation determination unit 230 includes a current frame additional information memory 231, a previous frame additional information memory 232, and a correlation determination unit 233. Details of the additional information calculation unit 250 will be described with reference to FIG.

  The pixel correlation determination unit 210 receives an image signal. This input image is, for example, image data obtained by raster scanning. The image data includes, for example, a luminance signal in a component signal of a digital color video signal.

  The image data is input for each frame data, and is first stored in the current frame memory 211 and then stored in the previous frame memory 212. Accordingly, the pixel correlation determination unit 210 holds two consecutive frame data in the memory.

  Furthermore, the pixel correlation determination unit 210 receives candidate vector information from the candidate vector extraction unit 102 illustrated in FIG. 5, and each pixel of the previous frame, that is, a pixel (target pixel) for which a motion vector is to be determined based on the candidate vector information. For each time, the pixel position of the current frame indicated by the plurality of candidate vectors is specified, and the difference between the pixel of interest of the previous frame and the pixel of the current frame indicated by the plurality of candidate vectors is calculated in the pixel value difference calculation unit 213, The absolute value calculation unit 214 calculates the difference absolute value and outputs it to the temporary determination unit 221.

  A series of these processes will be described with reference to FIG. FIG. 8 shows input frame (current frame) image data 300 and past frame (previous frame) image data 310. These frame data are data stored in the current frame memory 211 and the previous frame memory 212.

  The pixel correlation determination unit 210 receives candidate vector information from the candidate vector extraction unit 102 illustrated in FIG. 5, and these are candidate vectors a to g illustrated in FIG. 8. These are candidate vectors obtained by peak detection in the evaluation value table described above.

  The pixel correlation determination unit 210 specifies the pixel position of the current frame indicated by the plurality of candidate vectors for each pixel of the previous frame, that is, for each pixel (target pixel) whose motion vector should be determined based on the candidate vector information. For example, for the pixel of interest 311 of the past frame (previous frame) image data 310 of FIG. 8, the pixel positions A to G of the current frame indicated by the plurality of candidate vectors are specified.

  The pixel value difference calculation unit 213 calculates a difference between the pixel of interest 311 of the past frame (previous frame) image data 310 and the pixel values of the pixel positions A to G of the current frame indicated by the candidate vector. The absolute value calculation unit 214 calculates these difference absolute values, that is, pixel difference absolute values (MC residuals), and outputs them to the temporary determination unit 221.

  The provisional determination unit 221 inputs an absolute difference value (MC residual) between the pixel of interest and each pixel determined by a plurality of candidate vectors. For example, when there are n candidate vectors, n differential absolute values (MC residuals) [di] are input. However, i = 1 to n.

  The provisional determination unit 221 narrows down the motion vector corresponding to the pixel of interest 311 from the n differential absolute values (MC residuals) [di]. Specifically, a small MC residual is selected from n ((difference absolute value (MC residual) [di].

Assuming that the minimum value of n MC residuals is dmin and a predetermined threshold is TH,
di-dmin ≦ TH
A candidate vector is selected that indicates either a pixel that satisfies dmin or a pixel that satisfies dmin.

  In the example of FIG. 8, for example, if A, C, and E are selected as pixels satisfying the above-mentioned criteria among the pixels A to E, the selection candidate vector information is output from the temporary determination unit 221 to the gate 222. From the candidate vector information input from the candidate vector extraction unit 102 illustrated in FIG. 5, only the selected candidate vector information is input to the feature pixel position correlation determination unit 230.

  The feature pixel position correlation determination unit 230 executes a process of determining a motion vector corresponding to one pixel from the candidate vectors selected by the temporary determination unit 221. The feature pixel position correlation determination unit 230 performs a pixel value correlation determination process based on the feature pixel position information of the neighboring pixels of the target pixel input from the additional information calculation unit 250, and uses the candidate vector selected by the temporary determination unit 221. Processing for determining one motion vector is performed.

  With reference to FIG. 9, the detailed structure of the additional information calculating part 250 is demonstrated. For example, the additional information calculation unit 250 extracts, from the pixels in the vicinity of the target pixel, the pixel having the largest pixel value difference absolute value with the adjacent pixel as the feature pixel.

  9 includes an adjacent pixel difference absolute value calculation unit 251 including a register 252, a pixel value difference calculation unit 253, and an absolute value calculation unit 254, a register 255, a maximum difference value detection unit 256, and a maximum difference. A pixel position detection unit 257 and a register 258 are included.

  The adjacent pixel difference absolute value calculation unit 251 inputs an image signal to be subjected to motion vector detection processing and stores image frame data in the register 252. The pixel value difference calculation unit 253 sequentially calculates the difference between adjacent pixels of the image data stored in the register 252, and the absolute value calculation unit 254 sequentially calculates the absolute value of the difference between adjacent pixels to detect the maximum difference value. To the unit 256.

  Specific processing will be described with reference to FIG. The additional information calculation unit 250 sequentially calculates the absolute difference value of adjacent pixels of the image data stored in the register 252 so that, for example, the pixel having the maximum difference absolute value with the adjacent pixel in the pixel in the vicinity region of the target pixel. Is identified. The pixel in the vicinity region of the target pixel is a pixel region set in advance, such as a range of −8 to +7 pixels before and after the target pixel in the horizontal direction.

  In the example of FIG. 10, the pixel 282 having the maximum difference absolute value with the adjacent pixel is specified in the vicinity region of the target pixel 281.

  The maximum difference value detection unit 256 shown in FIG. 9 performs comparison processing while sequentially storing the difference absolute values of adjacent pixels that are sequentially input in the register 255, and the maximum difference value is maximum for every 16 pixels in a predetermined region, that is, −8 to +7. A pixel difference value having a pixel value difference absolute value is detected. That is, as shown in FIG. 11, pixel difference values MAXa, MAXb, and MAXc having the maximum pixel value difference absolute value for every 16 pixels are sequentially detected.

  Pixel information having the maximum pixel difference absolute value for each predetermined region (for example, every 16 pixels) detected by the maximum difference value detection unit 256 is input to the maximum difference pixel position detection unit 257, and for each predetermined region (for example, every 16 pixels). Is detected and stored in the register 258.

  For example, in the example illustrated in FIG. 10, the pixel position of the pixel 282 having the maximum absolute difference from the adjacent pixel in the vicinity region of the target pixel 281 is a pixel of [+4] when the target pixel is set to pixel position: 0. Position.

  The maximum difference pixel position information for each predetermined area (for example, every 16 pixels) stored in the register 258 is sequentially output to the current frame additional information storage memory 231.

  Returning to FIG. 7, the description of the process of the motion vector determination unit 200 will be continued. The feature pixel position information from the additional information calculation unit 250 described with reference to FIG. 9 is stored in the current frame additional information memory 231 of the feature pixel position correlation determination unit 230 shown in FIG. The current frame additional information memory 231 stores maximum difference pixel position information for each predetermined area (for example, every 16 pixels) extracted corresponding to one frame. The information in the current frame additional information memory 231 is moved and stored in the previous frame additional information memory 232 for each frame processing step.

  The current frame additional information memory 231 further receives an output from the temporary determination unit 221, that is, selection candidate vector information selected by the temporary determination unit 221.

  The correlation determination unit 233 executes a process of determining a unique motion vector corresponding to the target pixel from the selection candidate vectors selected by the temporary determination unit 221 based on the feature pixels extracted in the vicinity region of the target pixel. .

  The motion vector determination process in the correlation determination unit 233 will be described with reference to FIG. FIG. 12 shows input frame (current frame) image data 301 and past frame (previous frame) image data 310 similar to those described above with reference to FIG.

The input frame (current frame) image data 301 shows candidate vectors a, c, and e that have been selected by the candidate vector selection process by the provisional determination unit described above with reference to FIG.

  The correlation determination unit 233 in the feature pixel position correlation determination unit 230 converts the target pixel selected from the past frame (previous frame) image data 310, that is, the feature pixel position data in the vicinity region of the target pixel 311 as a pixel to which a motion vector is associated. Based on this, one motion vector is determined from the candidate vectors a, c, e.

  In the example illustrated in FIG. 12, the additional information calculation unit 250 indicates that the characteristic pixel position in the vicinity region of the target pixel 311 is the pixel position [−3] when the pixel position of the target pixel 311 is [0]. It is determined by the input information. In FIG. 12, the neighboring area is shown as an 8-pixel area of −4 to +3 of the target pixel.

  The correlation determination unit 233 determines the correlation between the feature pixel position in the vicinity region of the target pixel 311 and the feature pixel position in the vicinity region of the corresponding pixel positions A, C, and E indicated by the selection candidate vectors a, c, and e. In the example shown in the figure, in the vicinity region 301 of the corresponding pixel position A indicated by the selection candidate vector a, when the pixel position A is set to [0], a feature pixel appears at the position [−3]. This feature pixel is a feature pixel selected for each pixel area in a predetermined area unit in the additional information calculation unit described above, and is the maximum pixel of the adjacent pixel difference absolute value in the area set as the vicinity area of the pixel position A. is there.

  In the neighborhood region 302 of the corresponding pixel position C indicated by the selection candidate vector c, when the pixel position C is set to [0], the feature pixel appears at the position [−2], and the selection candidate vector e is indicated. In the vicinity region 303 of the corresponding pixel position E, the characteristic pixel appears at the position [+2] when the pixel position E is set to [0].

  The feature pixel in the vicinity region of the target pixel 311 appears at the position [−3] when the target pixel position is [0]. This feature pixel position corresponds to the feature pixel position [−3] in the vicinity region 301 of the pixel A indicated by the candidate vector a. It does not correspond to the characteristic pixel positions [−2] and [+2] in the neighboring areas 302 and 303 of the pixels C and E indicated by the other two candidate vectors c and e.

  As a result, the candidate vector a is selected and determined as the motion vector corresponding to the pixel of interest 311.

  The correlation determination unit 233 sequentially sets such processing as the pixel of interest for the constituent pixels of the frame, and executes the correlation determination processing of the feature pixel positions in the neighboring region for each of the set target pixels, as described above. A motion vector corresponding to each pixel is selected from the selection candidate vectors and determined.

  As described above, the motion vector determination unit illustrated in FIG. 7 identifies the feature pixel position based on the pixel value in the vicinity area of the target pixel, and the relative position corresponding to the target pixel position of the feature pixel in the vicinity area of the target pixel. And the candidate pixel selected by the tentative determination unit 221 with the feature pixel position in the vicinity region of the pixel indicated by the candidate vector, and the candidate vector indicating the pixel having the highest correlated feature pixel position is associated with the target pixel. The motion vector to be determined is determined.

  The above motion vector determination processing sequence will be described with reference to the flowchart shown in FIG.

  Processing of each step in the flow shown in FIG. 13 will be described. The flow shown in FIG. 13 is a motion vector determination process for one target pixel. When determining a motion vector for all the constituent pixels of the frame image, the constituent pixels of the frame image are sequentially set as the target pixel. The flow shown in FIG. 13 is executed for each pixel of interest.

  First, the process executed in step S201 is a process of calculating a difference absolute value between the pixel of interest and a pixel indicated by n candidate vectors, that is, an MC residual dn. This process is executed in the pixel correlation determination unit 210 of the motion vector determination unit 200 in FIG.

  Steps S202 to S204 are processes in the temporary determination unit 221. In step S202, the temporary determination unit 221 receives n differential absolute values (MC residuals) [di], and detects a minimum value of n MC residuals: dmin.

In step S203, i = 0 (0 ≦ i <n) is initially set. In step S204, a predetermined threshold is set as TH.
di-dmin ≦ TH
It is determined whether or not the above is satisfied.

  The process executed in step S204 is a provisional determination process executed as the selection process from the candidate vector described above with reference to FIG. If it is determined that di-dmin ≦ TH is not satisfied, it is determined that the vector is not a selection candidate vector, and step S207 is performed without performing correlation determination based on the feature pixel position in the vicinity region of the target pixel executed in steps S205 and S206. Proceed to

  In the provisional determination process executed in step S204, if di-dmin ≦ TH is satisfied, the selection candidate vector is selected, and the process proceeds to step S205. The processing in steps S205 and S206 is the processing of the feature pixel position correlation determination unit 230 of the motion vector determination unit 200 shown in FIG.

  In step S205, the correlation of the feature pixel position is determined. As described above with reference to FIG. 12, this processing is performed by correlating the position of the feature pixel in the vicinity area of the target pixel and the feature pixel position in the vicinity area of the pixel selected as the selection candidate vector, that is, This is a process for determining whether or not there is a correspondence.

  In step S206, a process of selecting a higher correlation is executed, and in step S207, it is determined whether or not all candidate vectors have been verified based on the value of i. Proceeding to S209, the value of i is updated (incremented by 1), and the processes after step S204 are repeatedly executed.

  When the verification for all candidate vectors is completed, the process proceeds to step S208, and at that time, the position of the feature pixel in the neighborhood region of the pixel of interest calculated in step S205 having the highest correlation, that is, the selection candidate is selected. A candidate vector corresponding to a highly correlated feature pixel position in the vicinity region of the pixel selected as the vector is determined as a motion vector corresponding to the target pixel as a processing target.

  As described above, in the motion vector determination process by the two-point matching method in the present embodiment, the target pixel and the feature pixel position information in the vicinity thereof are used without performing the block matching process in the motion vector determination process from the candidate vector. It is a configuration. Therefore, it is necessary to perform correlation calculation calculation based on a large number of pixel values according to the size of the block applied in the block matching process, and an efficient process is possible.

  When the input pixel data is input in raster scan order, the detection processing of the pixel position where the absolute difference value is maximum in the vicinity region limited to the horizontal direction with respect to the target pixel, or the adjacent pixel difference absolute value is maximum. The pixel position detection process, that is, the additional information calculation process in the additional information calculation unit 250 can be executed in parallel with the evaluation value table forming process executed in the evaluation value table forming unit 101 in FIG. At the end of the table formation process, additional information can be calculated, and a new process execution time is not set, and a motion vector detection process without reducing the processing speed is possible.

  In the above-described embodiment, as the vicinity region of the pixel of interest, the pixel position of interest is [0], and 16 pixels of −8 to +7 and 8 pixels of −4 to +3 are described as examples. It can be set as an arbitrary area. Considering that the input pixels are in raster scan order, if neighboring regions are set only in the right side in the horizontal direction, adjacent pixel comparison processing based on the input pixel values in raster scan order becomes possible. The memory area can be reduced, and the hardware scale can be further reduced.

In the above-described embodiment, a neighboring region of a pixel is set, and the additional information calculation unit 250 calculates a pixel position having the maximum adjacent pixel difference absolute value in the neighboring region as feature pixel information, and this feature pixel information is correlated. The configuration applied to
(A) The position of the pixel having the maximum adjacent pixel difference absolute value is applied. However, in addition to the configuration in which only the pixel position information having the maximum adjacent pixel difference absolute value is applied to the correlation determination, for example, the following information A configuration in which is applied as correlation determination information is also possible.
(B) Position and pixel value of the pixel having the maximum adjacent pixel difference absolute value

  FIG. 14 shows a configuration example of the additional information calculation unit 250 when the pixel position and the pixel value having the maximum adjacent pixel difference absolute value are applied as the correlation determination information.

  The configuration of the additional information calculation unit 250 illustrated in FIG. 14 is substantially the same as the configuration of the additional information calculation unit 250 illustrated in FIG. 9, and the adjacent pixel difference including the register 252, the pixel value difference calculation unit 253, and the absolute value calculation unit 254. An absolute value calculation unit 251, a register 255, a maximum difference value detection unit 256, a maximum difference pixel position detection unit 257, and a register 258 are included.

  The adjacent pixel difference absolute value calculation unit 251 inputs an image signal to be subjected to motion vector detection processing and stores image frame data in the register 252. The pixel value difference calculation unit 253 sequentially calculates the difference between adjacent pixels of the image data stored in the register 252, and the absolute value calculation unit 254 sequentially calculates the absolute value of the difference between adjacent pixels to detect the maximum difference value. To the unit 256.

  The maximum difference value detection unit 256 performs comparison processing while sequentially storing the difference absolute values of adjacent pixels that are sequentially input in the register 255, and the pixel difference having the maximum pixel value difference absolute value for each predetermined area, for example, every 16 pixels. Detect value.

  Pixel information having a pixel value difference absolute value for each predetermined region (for example, every 16 pixels) detected by the maximum difference value detection unit 256 is input to the maximum difference pixel position detection unit 257, and for each predetermined region (for example, every 16 pixels). Is detected and stored in the register 258.

  The additional information calculation unit 250 described with reference to FIG. 9 is configured to output only the position information stored in the register 258 to the current frame additional information memory 231 and perform a correlation determination process based on the position information. 14 outputs not only the position information indicating the maximum difference pixel position stored in the register 258 but also the pixel value information of the maximum difference pixel position from the register 255 to the current frame additional information memory 231. To do.

  In the characteristic pixel position correlation determination unit 230 shown in FIG. 7, the correlation determination processing is executed by applying the two pieces of data, that is, the two pieces of information of the pixel position information and the pixel value information that have the maximum adjacent pixel difference absolute value. To do.

  For example, in FIG. 12 described above, even when any of the selection candidate vectors has a characteristic pixel at the same position as the pixel position detected in the vicinity region of the target pixel, By verifying the correlation of values, the one with the highest degree of matching can be selected and determined as the motion vector of the pixel of interest.

  Thus, not only position information but also both position information and pixel value level information and pixel value level difference information are used for correlation determination, whereby a more accurate motion vector can be determined.

  Note that the above-described two-point matching uses the assumption that the object moves with an area. However, for example, if the feature pixel is set at a position far from the target pixel, the possibility that the object is the same object decreases. In view of this, it is possible to adopt a configuration in which not only the degree of matching of the feature pixel position but also the process of reflecting the distance to the feature pixel in the correlation is performed. In other words, the feature pixel at a position close to the target pixel is increased in weight at the correlation determination, and the feature pixel at a position far from the target pixel is decreased at the correlation determination.

(2) Motion vector determination processing by the three-point matching method Next, motion vector determination processing by the three-point matching method will be described. FIG. 15 shows a detailed configuration of the motion vector determination unit of the motion vector detection device according to the present embodiment. The motion vector determination unit 400 illustrated in FIG. 15 corresponds to the motion vector determination unit 103 of the motion vector detection device illustrated in FIG.

  A motion vector determination unit 400 shown in FIG. 15 receives a plurality of candidate vector information determined based on the above-described evaluation value table from the candidate vector extraction unit 102 shown in FIG. 5, and determines a motion vector to be associated with each pixel. Execute the process.

  As shown in FIG. 15, the motion vector determination unit 400 according to this embodiment includes a pixel correlation determination unit 410, a temporary determination unit 421, an additional information calculation unit 450, a current frame additional information memory 422, a previous frame additional information memory 423, A neighborhood region information correlation determination unit 430 is included. The pixel correlation determination unit 410 includes a current frame memory 411, a previous frame memory 412, a pixel value difference calculation unit 413, and an absolute value calculation unit 414. The neighborhood area information correlation determination unit 430 includes registers 431 and 432, a gate 433, and a correlation determination unit 434. Details of the additional information calculation unit 450 will be described with reference to FIG.

  The pixel correlation determination unit 410 receives an image signal. This input image is image data obtained by, for example, raster scanning, as described above with reference to FIG. The image data includes, for example, a luminance signal in a component signal of a digital color video signal.

  The image data is input for each frame data, and is first stored in the current frame memory 411 and then stored in the previous frame memory 412. Accordingly, the pixel correlation determination unit 410 holds two consecutive frame data in the memory.

  Furthermore, the pixel correlation determination unit 410 receives candidate vector information from the candidate vector extraction unit 102 illustrated in FIG. 5, and each pixel of the previous frame, that is, a pixel (target pixel) for which a motion vector is to be determined based on the candidate vector information. For each time, the pixel position of the current frame indicated by the plurality of candidate vectors is specified, and the difference between the target pixel of the previous frame and the pixel of the current frame indicated by the plurality of candidate vectors is calculated by the pixel value difference calculation unit 413, The absolute value calculation unit 414 calculates the difference absolute value and outputs it to the temporary determination unit 421. A series of these processes is the same as in the case of the above two-point matching.

  The provisional determination unit 421 inputs an absolute difference value (MC residual) between the pixel of interest and each pixel determined by a plurality of candidate vectors. For example, when there are n candidate vectors, n differential absolute values (MC residuals) [di] are input. However, i = 1 to n.

The provisional determination unit 421 narrows down the motion vector corresponding to the pixel of interest from the n differential absolute values (MC residuals) [di]. Specifically, a small MC residual is selected from n ((difference absolute value (MC residual) [di]. This process is the same as in the previous two-point matching. For example, Assuming that the minimum value of n MC residuals is dmin and a predetermined threshold is TH,
di-dmin ≦ TH
A candidate vector is selected that indicates either a pixel that satisfies dmin or a pixel that satisfies dmin.

  In the example of FIG. 8 described above, for example, if A, C, and E are selected as pixels satisfying the above-mentioned criteria among the pixels A to E, the selection candidate vector information is gated from the temporary determination unit 421. Only the candidate vector information selected from the candidate vector information input to the candidate vector extraction unit 102 shown in FIG. 5 and input from the candidate vector extraction unit 102 shown in FIG. 5 is input to the neighborhood region information correlation determination unit 430.

  In the image correlation determination unit 410 of the present embodiment, the pixel information of two pixels in the vicinity region of the target pixel is further input from the current frame memory 411 of the image correlation determination unit 410 to the register 432 of the vicinity region information correlation determination unit 430. Furthermore, two pixels corresponding to the n instruction pixels indicated by the candidate vectors, that is, n × 2 pixels, are input from the previous frame memory 412 of the image correlation determination unit 410 to the register 431 of the neighborhood region information correlation determination unit 430. .

  Which pixel to output in the vicinity region is determined by the output of the current frame additional information memory 422, the previous frame additional information memory 423, and n pieces of candidate vector information.

  The calculation result of the additional information calculation unit 450 is output to the current frame additional information memory 422 and the previous frame additional information memory 423.

  With reference to FIG. 16, the detailed structure of the additional information calculating part 450 is demonstrated. For example, the additional information calculation unit 450 uses, as feature pixels, a pixel having the largest pixel value difference from the pixel of interest and a pixel having the smallest pixel value difference from the pixel of interest among the pixels in the vicinity of the pixel of interest. Extract.

  16 includes a register 452, a target pixel difference calculation unit 451 including a pixel value difference calculation unit 453, a maximum difference pixel information acquisition unit (MAX) 454, and a minimum difference pixel information acquisition unit ( MIN) 455, a maximum difference pixel position detection unit 456, a register 457, a minimum difference pixel position detection unit 458, and a register 459.

  The target pixel difference calculation unit 451 inputs an image signal to be a motion vector detection processing target, and stores image frame data in the register 452. The pixel value difference calculation unit 453 sequentially calculates the difference between the neighboring regions of the image data stored in the register 452, stores the pixel information having the maximum difference value in the difference maximum value pixel information acquisition unit (MAX) 454, and sets the minimum Pixel information having a difference value is stored in a difference minimum value pixel information acquisition unit (MIN) 455. Note that these pieces of pixel information include pixel position information.

  A specific process will be described with reference to FIG. The additional information calculation unit 450 sequentially calculates the difference between the target pixel and the neighboring pixel value of the target pixel with respect to the image data stored in the register 452, so that, for example, in the pixel in the vicinity region of the target pixel, Pixel information having the maximum difference value and pixel information having the minimum difference value are acquired. The pixel in the vicinity region of the target pixel is, for example, a pixel region set in advance such as a range of −8 to +7 pixels in the horizontal direction of the target pixel or a block set in the two-dimensional region.

  In the example of FIG. 17, the pixel having the maximum difference value with the target pixel in the vicinity region of the target pixel 481 is the pixel 482, that is, the pixel position [−5] when the pixel position of the target pixel 481 is set to [0]. The pixel is selected as a pixel having a maximum difference value from the pixel of interest, that is, the maximum pixel of the spatial gradient, and pixel information is acquired as one feature pixel. The pixel having the minimum difference value from the target pixel is a pixel 483, that is, a pixel having a minimum difference value from the target pixel when the pixel position of the target pixel 481 is [0], That is, pixel information is acquired as one feature pixel selected as the smallest pixel of the spatial gradient.

  The maximum difference pixel information acquisition unit (MAX) 454 illustrated in FIG. 16 determines the maximum difference value for the target pixel for each of 16 pixels of a predetermined region, for example, −8 to +7, based on the difference information of the pixels that are sequentially input. Is detected. The difference minimum value pixel information acquisition unit (MIN) 455 detects a pixel having a minimum difference value with respect to the target pixel for each predetermined region, based on the difference information of the sequentially input pixels.

  Pixel information having the maximum pixel difference value for each predetermined region detected by the maximum difference pixel information acquisition unit (MAX) 454 is input to the maximum difference pixel position detection unit 456, and the maximum for each predetermined region (for example, every 16 pixels). Difference pixel position information is detected and stored in the register 457.

  Pixel information having the minimum pixel difference value for each predetermined region detected by the minimum difference pixel information acquisition unit (MIN) 455 is input to the minimum difference pixel position detection unit 458, and is minimum for each predetermined region (for example, every 16 pixels). Difference pixel position information is detected and stored in the register 459.

  For example, in the example shown in FIG. 17, the pixel position of the pixel 482 having the maximum difference from the target pixel in the vicinity region of the target pixel 481 is the pixel position [−5] when the target pixel is set to pixel position: 0. The pixel position of the pixel 483 that minimizes the difference from the pixel of interest is the pixel position [−1] when the pixel of interest is pixel position: 0.

  The maximum difference pixel position information for each predetermined area (for example, every 16 pixels) stored in the register 457 and the minimum difference pixel position information for each predetermined area (for example, every 16 pixels) stored in the register 459 are stored in the current frame additional information. The data is sequentially output to the memory 422.

  Returning to FIG. 15, the description of the process of the motion vector determination unit 400 will be continued. The feature pixel position information from the additional information calculation unit 450 described with reference to FIG. 16 is input to the current frame additional information memory 422 shown in FIG. The additional information moves to the previous frame additional information memory 423 for each frame process.

  Additional information of the current frame memory input to the current frame additional information memory 422, that is, two pieces of feature pixel position information corresponding to the target pixel, that is, pixel position information having a maximum difference from the target pixel, and a minimum of the target pixel The pixel position information having the difference is output to the current frame memory 411 of the pixel correlation determination unit 410, and the pixel information of the two points is stored in the register 432 of the neighborhood region information determination unit 430 based on the output information. . This pixel information includes pixel position information.

  Further, the additional information of the previous frame memory input to the previous frame additional information memory 423, that is, the pixel position information having the maximum difference from the target pixel and the pixel position information having the minimum difference from the target pixel are correlated with each other. The data is output to the previous frame memory 412 of the determination unit 410. The previous frame memory 412 further receives n candidate vector information from the candidate vector extraction unit, specifies two feature pixels from the neighboring region determined for each pixel indicated by the candidate vector information, and n × The second feature pixel information is stored in the register 431 of the neighborhood area information determination unit 430. This pixel information includes pixel position information.

  The n × 2 feature pixel information stored in the register 431 is output to the correlation determination unit 434 via the gate 433. In the gate 433, only the feature pixel information in the neighborhood region of the pixel corresponding to the selection candidate vector is sent to the correlation determination unit 434 based on the selection candidate vector selected by the temporary determination process in the temporary determination unit 421 from the n candidate vectors. Is output.

  On the other hand, the feature pixel information of the pixel of interest is directly output to the correlation determination unit 434. The correlation determination unit 434 performs correlation determination processing of these feature pixels, and determines a unique motion vector from the selection candidate vectors.

  The motion vector determination process in the correlation determination unit 434 will be described with reference to FIG. FIG. 18 shows input frame (current frame) image data 500 and past frame (previous frame) image data 510 similar to those described above with reference to FIG.

  The input frame (current frame) image data 500 shows candidate vectors a, c, and e that have been selected by the candidate vector selection process by the provisional determination unit described above with reference to FIG.

  The correlation determination unit 433 in the vicinity region information determination unit 430 is based on the feature pixel position data in the vicinity region of the pixel of interest selected from the past frame (previous frame) image data 510, that is, the pixel of interest 511 as a pixel associated with the motion vector. Thus, one motion vector is determined from the candidate vectors a, c, e.

  In the example shown in FIG. 18, the region near the target pixel 511 is a two-dimensional block region. The setting of this neighborhood region is arbitrary, and may be set in a one-dimensional region in the horizontal direction as in the above-described two-point matching.

  When the pixel position of the target pixel 511 is centered, a spatial gradient maximum pixel 512 that is a pixel position that becomes the maximum difference pixel with the target pixel and a spatial gradient minimum pixel 513 that is a pixel position that becomes the minimum difference pixel with the target pixel are determined. To do.

  On the other hand, the input frame (current frame) 500 is also set by the pixel area set around the pixel indicated by each selection candidate vector, that is, the pixel area 501 set by the selection candidate vector a and the selection candidate vector c. In the pixel area 502 and the pixel area 503 set by the selection candidate vector e, the spatial gradient maximum pixel that is the pixel position that becomes the maximum difference pixel from the center pixel and the pixel position that becomes the minimum difference pixel from the center pixel The spatial gradient minimum pixel is determined.

  The correlation determination unit 433 determines the correlation between the feature pixel position in the vicinity region of the pixel of interest 511 and the feature pixel position in the vicinity region of the corresponding pixel position indicated by the selection candidate vectors a, c, and e.

  In the example shown in the figure, the neighboring area 501 indicated by the selection candidate vector a is a position corresponding to the feature pixel position in the vicinity area of the pixel of interest 511, and the other two selection candidate vectors c and e indicate. The neighboring areas 502 and 503 are not at positions corresponding to the characteristic pixel positions in the neighboring area of the pixel of interest 511.

  As a result, the candidate vector a is selected and determined as the motion vector corresponding to the target pixel 511.

  The correlation determination unit 433 sequentially sets such processing as the pixel of interest for the constituent pixels of the frame, executes the correlation determination processing of the feature pixel positions in the vicinity region for each of the set target pixels, and performs frame configuration. A motion vector corresponding to each pixel is selected from the selection candidate vectors and determined.

  As described above, the motion vector determination unit 400 illustrated in FIG. 15 identifies two feature pixel positions based on the pixel values in the vicinity area of the target pixel, and corresponds to the target pixel positions of the feature pixels in the vicinity area of the target pixel. The correlation between the relative position to be detected and the feature pixel position in the vicinity region of the pixel indicated by the candidate vector selected by the temporary determination unit 221 is determined, and the candidate vector indicating the pixel having the highest correlation feature pixel position is focused on It is determined as a motion vector corresponding to the pixel.

  The above motion vector determination processing sequence will be described with reference to the flowchart shown in FIG. The flow shown in FIG. 19 is a motion vector determination process for one target pixel. When determining a motion vector for all the constituent pixels of the frame image, the constituent pixels of the frame image are sequentially set as the target pixel. The flow shown in FIG. 19 is executed for each pixel of interest.

  First, the process executed in step S301 is a process for calculating the absolute difference value between the pixel of interest and the pixel indicated by the n candidate vectors, that is, the MC residual dn. This process is executed in the pixel correlation determination unit 410 of the motion vector determination unit 400 in FIG.

  Steps S302 to S304 are processes in the temporary determination unit 421. In step S302, the provisional determination unit 421 inputs n differential absolute values (MC residuals) [di], and detects the minimum value of d MC residuals: dmin.

In step S303, i = 0 (0 ≦ i <n) is initially set. In step S304, a predetermined threshold is set as TH.
di-dmin ≦ TH
It is determined whether or not the above is satisfied.

  The process executed in step S304 is a provisional determination process executed as the selection process from the candidate vector described above with reference to FIG. If it is determined that di-dmin ≦ TH is not satisfied, it is determined that the vector is not a selection candidate vector, and the correlation determination based on the feature pixel position in the vicinity region of the target pixel is performed in steps S305, S306, and S307. Proceed to step S308.

  When di-dmin ≦ TH is satisfied in the provisional determination process executed in step S304, the selection candidate vector is selected, and the process proceeds to step S305.

  In step S305, the correlation of the feature pixel position is determined. As described above with reference to FIG. 16, this processing correlates the positions of two feature pixels in the vicinity region of the target pixel and the two feature pixel positions in the vicinity region of the pixel selected as the selection candidate vector. That is, it is a process for determining whether or not there is a correspondence between positions.

  In step S306, a process of selecting a higher correlation is executed, and in step S307, it is determined whether or not all candidate vectors have been verified based on the value of i. Proceeding to S310, the value of i is updated (incremented by 1), and the processing after step S304 is repeatedly executed.

  When the verification for all candidate vectors is completed, the process proceeds to step S308, at which time the position of the two feature pixels in the neighborhood of the pixel of interest calculated in step S305 having the highest correlation, that is, selected in step S305, A candidate vector corresponding to a highly correlated feature pixel position in the neighborhood region of the pixel selected as the selection candidate vector is determined as a motion vector corresponding to the target pixel as a processing target.

  As described above, in the motion vector determination process using the three-point matching method in the present embodiment, the pixel of interest and the information on the position of two feature pixels in the vicinity thereof are not performed in the process of determining the motion vector from the candidate vector without performing the block matching process. It is the structure using. Therefore, it is necessary to perform correlation calculation calculation based on a large number of pixel values according to the size of the block applied in the block matching process, and an efficient process is possible.

  In the above embodiment, the correlation determination is performed based on the position information by applying the two feature pixel positions of the current frame and the previous frame, but the correlation determination process is performed based on the pixel value at the feature pixel position corresponding to the target pixel. It is good also as a structure. In other words, the block matching is performed using only a total of three points using the pixel value of the pixel of interest and two feature pixels in the vicinity region. This processing sequence will be described with reference to FIG.

  FIG. 20 shows input frame (current frame) image data 600 and past frame (previous frame) image data 610, as in FIG. 18 described above.

  The input frame (current frame) image data 600 shows candidate vectors a, c, and e that have been selected by the candidate vector selection process by the provisional determination unit described above with reference to FIG.

  The correlation determination unit 433 in the vicinity region information determination unit 430 is based on the feature pixel position data in the vicinity region of the pixel of interest selected from the past frame (previous frame) image data 510, that is, the pixel of interest 511 as a pixel associated with the motion vector. Thus, one motion vector is determined from the candidate vectors a, c, e.

  In the example shown in FIG. 20, the vicinity region of the pixel of interest 611 is a two-dimensional block region. The setting of this neighborhood region is arbitrary, and may be set in a one-dimensional region in the horizontal direction as in the above-described two-point matching.

  When the pixel position of the pixel of interest 611 is centered, the spatial gradient maximum pixel 612 that is the pixel position that is the pixel of interest and the maximum difference pixel, and the spatial gradient minimum pixel 613 that is the pixel position of the pixel of interest and the minimum difference pixel are determined. To do.

  On the other hand, in the input frame (current frame) 600, the pixel area set around the pixel indicated by each selection candidate vector, that is, the pixel area 601 set by the selection candidate vector a, and the selection candidate vector c are set. In each of the pixel area 602 and the pixel area 603 set by the selection candidate vector e, pixels corresponding to the positions of the spatial gradient maximum pixel 612 and the spatial gradient minimum pixel 613 for the target pixel 611 detected in the past frame are picked up. Then, it is output to the neighborhood region information correlation determination unit 430 shown in FIG.

  As shown in FIG. 20, from each pixel region 601, 602, 603 indicated by each selection candidate vector a, c, e, the spatial gradient maximum pixel 612 for the target pixel 611 detected in the past frame, and the spatial gradient minimum A pixel corresponding to each position of the pixel 613 is picked up and a correlation is determined.

  That is, the neighborhood value information correlation determination unit 430 illustrated in FIG. 15 performs pixel value correlation determination between these pixels. That is, a correlation determination is performed between the pixel at each position of the spatial gradient maximum pixel 612 and the spatial gradient minimum pixel 613 with respect to the pixel of interest 611 and the pixel values at corresponding positions of the pixel regions 601, 602, and 603. For the center pixel, correlation determination is performed in the temporary determination unit 421. As a result, a matching processing result using only three points is obtained, and it is determined that the highest correlation is obtained by the three-point matching processing. A selection candidate vector corresponding to the data is determined as a motion vector corresponding to the pixel of interest.

  FIG. 21 shows a motion vector determination processing flow according to the above method. In the processing flow shown in FIG. 21, the processing in steps S401 to S404 is the same as the processing in steps S301 to S304 in the processing flow described above with reference to FIG. Process.

  In step S405, block matching with three pixels is executed. That is, as described with reference to FIG. 20, each pixel region indicated by the selection candidate vector corresponds to each position of the spatial gradient maximum pixel and the spatial gradient minimum pixel with respect to the target pixel detected in the past frame. Pixels are picked up and correlation is determined. For the pixel correlation information corresponding to the center pixel, that is, the target pixel position, the determination result of the temporary determination unit is referred to.

  Correlation determination is executed by matching these three points, and a vector showing the highest correlation is determined as a motion vector corresponding to the target pixel. The processes in steps S406 to S408 and S410 are the same as steps S306 to S308 and S310 in the process flow described above with reference to FIG.

  As described above, in the motion vector determination process from the candidate vector to which the three-point matching method is applied, the correlation determination using all the pixels in the vicinity (block) of the target pixel is not performed, and the two pixels in the vicinity of the target pixel and the vicinity thereof are determined. This is a configuration using characteristic pixel position information and pixel value information. Therefore, it is not necessary to perform correlation calculation based on a large number of pixel values according to the size of the block applied in the block matching process, and an efficient process is possible.

  The two processes described above, that is, the motion vector determination process based on the correlation determination using the correlation information of the characteristic pixel position described with reference to FIG. 19 and the correlation information of the pixel value described with reference to FIG. A process using a motion vector determination process based on the correlation determination that is used is also possible.

  FIG. 22 shows a flow for explaining a processing sequence in which correlation determination using correlation information of characteristic pixel positions and motion vector determination processing by correlation determination using correlation information of pixel values are used together.

  In the processing flow shown in FIG. 22, the processing in steps S501 to S504 is the same as the processing in steps S301 to S304 in the processing flow described above with reference to FIG. Process.

  In step S505, the correlation of the feature pixel position is determined. As described above with reference to FIG. 16, this processing correlates the positions of two feature pixels in the vicinity region of the target pixel and the two feature pixel positions in the vicinity region of the pixel selected as the selection candidate vector. That is, it is a process for determining whether or not there is a correspondence between positions.

  Next, in step S506, block matching with three pixels is executed. That is, as described with reference to FIG. 20, each pixel region indicated by the selection candidate vector corresponds to each position of the spatial gradient maximum pixel and the spatial gradient minimum pixel with respect to the target pixel detected in the past frame. Pixels are picked up and correlation is determined. For the pixel correlation information corresponding to the center pixel, that is, the target pixel position, the determination result of the temporary determination unit is referred to.

  In step S507, a vector showing the highest correlation is selected in consideration of both the correlation determination in step S505 and the correlation determination in step S506. The processing in steps S508 to S510 is the same as the processing in steps S307 to S308 and S310 in the processing flow described above with reference to FIG.

  Thus, even in the motion vector determination processing from the candidate vector to which the correlation between the pixel position and the three-point matching method is applied, it is not necessary to perform the correlation determination using all the pixels in the vicinity of the target pixel (block). The configuration uses only a very small amount of pixel information, and it is necessary to perform a correlation calculation calculation based on a large number of pixel values according to the size of a block to be applied in the block matching processing, thereby enabling efficient processing.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims section described at the beginning should be considered.

  The series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. The computer can receive the program transferred in this manner and install it on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of the present invention, in a motion vector detection process from moving image data, a pixel of interest to which a motion vector is associated in a process of selecting and determining a motion vector corresponding to each pixel from a plurality of candidate vectors. It is necessary to apply block matching processing because the feature pixel is extracted from the neighborhood region of the image, the correlation is determined based on the position information of the feature pixel or the pixel value information, and the motion vector is determined. In addition, it is possible to reduce both evaluation value calculations such as the absolute difference sum calculation as the calculation for correlation calculation, improving the processing efficiency and reducing the memory for holding pixel values. Miniaturization of hardware scale is realized. Therefore, by applying the present invention to an image processing apparatus or the like that executes moving image data encoding processing or the like, more accurate motion vector detection processing can be performed.

It is a figure explaining the outline | summary of a block matching method. Explains the outline of the evaluation value table creation process by the representative point matching method applied in the motion vector detection process, the candidate vector extraction process based on the evaluation value table, and the motion vector setting process corresponding to each pixel based on the extracted candidate vector (No. 1) Explains the outline of the evaluation value table creation process by the representative point matching method applied in the motion vector detection process, the candidate vector extraction process based on the evaluation value table, and the motion vector setting process corresponding to each pixel based on the extracted candidate vector (No. 2) Explains the outline of the evaluation value table creation process by the representative point matching method applied in the motion vector detection process, the candidate vector extraction process based on the evaluation value table, and the motion vector setting process corresponding to each pixel based on the extracted candidate vector (No. 3) It is a figure which shows one Example structure of the motion vector detection apparatus of this invention which performs a motion vector detection process. It is a flowchart explaining the process sequence of the motion vector detection apparatus of this invention which performs a motion vector detection process. It is a figure which shows the detailed structure of the motion vector determination part of the motion vector detection apparatus of this invention. It is a figure explaining the process in the temporary determination part of a motion vector determination part. It is a figure which shows the detailed structure of the additional information calculating part of the motion vector determination part of this invention. It is a figure explaining the characteristic pixel detected in an additional information calculating part. It is a figure explaining the feature pixel detection process aspect detected in an additional information calculating part. It is a figure explaining the detail of the process performed in the motion vector determination part of the motion vector detection apparatus of this invention. It is a flowchart figure explaining the sequence of the motion vector determination process of the motion vector detection apparatus of this invention. It is a figure which shows the structural example of the additional information calculating part of the motion vector determination part of this invention. It is a figure which shows the detailed structure (Example 2) of the motion vector determination part of the motion vector detection apparatus of this invention. It is a figure which shows the structural example of the additional information calculating part of the motion vector determination part of this invention. It is a figure explaining the characteristic pixel detected in an additional information calculating part. It is a figure explaining the detail of the process performed in the motion vector determination part of the motion vector detection apparatus of this invention. It is a flowchart figure explaining the sequence of the motion vector determination process of the motion vector detection apparatus of this invention. It is a figure explaining the detail of the process performed in the motion vector determination part of the motion vector detection apparatus of this invention. It is a flowchart figure explaining the sequence of the motion vector determination process of the motion vector detection apparatus of this invention. It is a flowchart figure explaining the sequence of the motion vector determination process of the motion vector detection apparatus of this invention.

Explanation of symbols

10 Previous frame 11 Inspection block By
12 Search area 20 Current frame 21 Reference block Bx
70 Front frame 71 Representative point Ry
80 Current Frame 81 Search Area 90 Evaluation Value Table 91 Previous Frame Pixel 95, 96, 97 Current Frame Pixel 101 Evaluation Value Table Forming Unit 102 Candidate Vector Extraction Unit 103 Motion Vector Determination Unit 104 Control Unit (Controller)
200 Motion vector determination unit 210 Pixel correlation determination unit 211 Current frame memory 212 Previous frame memory 213 Pixel value difference calculation unit 214 Absolute value calculation unit 221 Temporary determination unit 222 Gate 230 Feature pixel position correlation determination unit 231 Current frame additional information memory 232 Previous Frame additional information memory 233 Correlation determining section 250 Additional information calculating section 251 Adjacent pixel difference absolute value calculating section 252 Register 253 Pixel value difference calculating section 254 Absolute value calculating section 255 Register 256 Maximum difference value detecting section 257 Maximum difference pixel position detecting section 258 Register 281 Pixel of interest 282 Feature pixel (Adjacent pixel difference absolute value maximum pixel)
300 Input frame (current frame) image data 310 Past frame (previous frame) image data 301, 302, 303 Neighborhood region 311 Pixel of interest 400 Motion vector determination unit 410 Pixel correlation determination unit 411 Current frame memory 412 Previous frame memory 413 Pixel value difference Calculation unit 414 Absolute value calculation unit 421 Temporary determination unit 422 Current frame additional information memory 423 Previous frame additional information memory 430 Neighborhood information correlation determination unit 431, 432 Register 433 Gate 434 Correlation determination unit 450 Additional information calculation unit 451 Calculation of target pixel difference Unit 452 Register 453 Pixel value difference calculation unit 454 Difference maximum value pixel information acquisition unit (MAX)
455 Difference minimum value pixel information acquisition unit (MIN)
456 Maximum difference pixel position detection unit 457 Register 458 Minimum difference pixel position detection unit 459 Register 481 Pixel of interest 482 Spatial gradient maximum pixel 483 Spatial gradient minimum pixel 500 Input frame (current frame) image data 510 Past frame (previous frame) image data 501 , 502, 503 Neighborhood region 511 Pixel of interest 512 Spatial gradient maximum pixel 513 Spatial gradient minimum pixel 600 Input frame (current frame) image data 610 Past frame (previous frame) image data 601, 602, 603 Neighborhood region 611 Pixel of interest 612 Spatial gradient Maximum pixel 613 Spatial gradient minimum pixel

Claims (6)

A motion vector detection device for detecting a motion vector from moving image data;
An evaluation value table forming unit that generates an evaluation value table based on pixel value correlation information between different frames on the time axis;
A candidate vector extraction unit that detects a candidate vector of a motion vector corresponding to a frame constituent pixel of moving image data based on the evaluation value table;
A motion vector determination unit that determines a motion vector corresponding to each pixel from the candidate vector,
The motion vector determination unit
The feature pixel is extracted from the region near the pixel of interest associated motion vector, based on the correlation determination process based on the feature pixel, Ri configuration der to determine the motion vector corresponding to the target pixel,
The difference between the pixel value of the pixel of interest associated with the motion vector and the pixel value of the designated pixel indicated by the candidate vector is calculated, and the difference is compared with a predetermined threshold value. The following correlation determination process for only the selection candidate vector selected by the provisional determination process that executes the process of selecting the candidate vector indicating the designated pixel as a highly correlated candidate vector:
A feature pixel is extracted from the neighborhood region of the target pixel to which the motion vector is associated, the relative positional relationship between the target pixel and the feature pixel, and the relative relationship between the instruction pixel of the selection candidate vector and the feature pixel in the vicinity of the instruction pixel. A motion vector detection apparatus that determines a motion vector corresponding to a pixel of interest based on a correlation determination process based on a positional relationship . The motion vector determination unit
Motion vector detection device according to claim 1, characterized in that the pixel value difference absolute value from the neighboring region and the adjacent pixels of the pixel of interest to map the motion vector is extracted as the feature pixel maximum pixel. The motion vector determination unit
Motion vector detection device according to claim 1, characterized in that extracts a plurality of characteristic pixels from the neighboring region of the target pixel to associate the motion vector. The motion vector determination unit
Motion vector detection device according to claim 3, characterized in that extracting a feature pixel maximum pixel and the minimum of the two pixels of the pixel value difference between the target pixel from the neighboring region of the target pixel to associate the motion vector. A motion vector detection method for detecting a motion vector from moving image data,
An evaluation value table forming step for generating an evaluation value table based on pixel value correlation information between different frames on the time axis;
A candidate vector extraction step of detecting a candidate vector of a motion vector corresponding to a frame constituent pixel of moving image data based on the evaluation value table;
A motion vector determining step for determining a motion vector corresponding to each pixel from the candidate vector,
The motion vector determination step includes:
The feature pixel is extracted from the region near the pixel of interest associated motion vector, based on the correlation determination process based on the feature pixel, Ri step der to determine the motion vector corresponding to the target pixel,
The difference between the pixel value of the pixel of interest associated with the motion vector and the pixel value of the designated pixel indicated by the candidate vector is calculated, and the difference is compared with a predetermined threshold value. The following correlation determination process for only the selection candidate vector selected by the provisional determination process that executes the process of selecting the candidate vector indicating the designated pixel as a highly correlated candidate vector:
A feature pixel is extracted from the neighborhood region of the target pixel to which the motion vector is associated, the relative positional relationship between the target pixel and the feature pixel, and the relative relationship between the instruction pixel of the selection candidate vector and the feature pixel in the vicinity of the instruction pixel. A motion vector detection method, comprising: determining a motion vector corresponding to a pixel of interest based on a correlation determination process based on a positional relationship . In the motion vector detection device, a computer program for executing processing for detecting a motion vector from moving image data,
An evaluation value table forming step for generating an evaluation value table based on pixel value correlation information between different frames on the time axis;
A candidate vector extraction step of detecting a candidate vector of a motion vector corresponding to a frame constituent pixel of moving image data based on the evaluation value table;
A program for executing a motion vector determination step for determining a motion vector corresponding to each pixel from the candidate vector ,
The motion vector determination step includes:
The feature pixel is extracted from the region near the pixel of interest associated motion vector, based on the correlation determination process based on the feature pixel, Ri step der to make determining the motion vector corresponding to the target pixel,
The difference between the pixel value of the pixel of interest associated with the motion vector and the pixel value of the designated pixel indicated by the candidate vector is calculated, and the difference is compared with a predetermined threshold value. The following correlation determination process for only the selection candidate vector selected by the provisional determination process for executing the process of selecting the candidate vector indicating the designated pixel as a highly correlated candidate vector:
A feature pixel is extracted from the vicinity region of the target pixel to which the motion vector is associated, the relative positional relationship between the target pixel and the feature pixel, and the relative relationship between the instruction pixel of the selection candidate vector and the feature pixel in the vicinity of the instruction pixel. A computer program for determining a motion vector corresponding to a pixel of interest based on a correlation determination process based on a positional relationship .

Images