JP4804238B2 - Motion vector detection device - Google Patents

Description

  The present invention relates to a motion vector detection device that detects the motion of a screen of a moving image.

  Conventionally, various methods for detecting a motion vector by dividing an image into blocks of a predetermined size and calculating the correlation between the previous frame image and the current frame image for each block have been proposed. The main methods of motion vector detection are the gradient method for obtaining a motion vector from the spatial gradient and temporal gradient of the luminance value, and the luminance value of a part of the current image and the luminance value of a part of the image before a predetermined time. There is a matching method for obtaining a motion vector from the absolute value of the difference between frames.

  Among these, the representative point matching method is widely implemented and can be realized with a relatively small hardware scale. The representative point matching method is used for MUSE encoders and screen shake (camera shake) correction. There is also a block matching method in which an evaluation target is a block instead of a point, and the block matching method is used for motion compensation for the purpose of high-efficiency encoding of an image. The basic concept is the same for both the representative point matching method and the block matching method.

  However, the conventional motion vector detection method based on representative point matching has a problem in that it tends to malfunction due to noise on an entirely dark image or a flat screen with a small density gradient. For this reason, a technique has been proposed in which a density gradient of luminance values near each representative point position is obtained, weighted according to the density gradient value, and cumulatively added (for example, Patent Document 1 below).

Japanese Patent No. 2860702

  However, in the case of Patent Document 1 described above, the position of the representative point is basically fixed, and is the density gradient information of the luminance value near the representative point reliable to calculate the motion vector? It is to determine whether or not. Therefore, for example, if the reliability of the density gradient information of the luminance value is low for all representative points, an accurate motion vector cannot be detected. In other words, the technique of Patent Document 1 focuses on reducing malfunctions and does not actively enable correct detection.

  A so-called repetitive pattern exists as an image for which motion vector detection is difficult for the representative point matching method. This is because the representative point matching method uses the correlation property of the image that the difference in pixel density value generally increases as the distance increases. For example, a predetermined amount is shifted from all representative point positions in a repetitive pattern. If the density data of the selected pixel is the same as the density data of the representative point, there is a problem that it is difficult to determine whether the pixel is moving or stationary.

  The present invention has been made in view of the above-described problems. In addition to a flat image with a small density gradient, which is difficult to detect a motion vector, in the repetitive pattern that is prone to malfunction, the malfunction is reduced and a correct motion vector is obtained. An object of the present invention is to realize a motion vector detection device capable of detection.

The invention according to claim 1 is a motion vector detection device for detecting a motion vector by a representative point matching method from a video signal of a moving image composed of a plurality of temporally continuous frames, wherein the plurality of frames Each of the screens is divided into blocks of a predetermined size, a density gradient calculation unit that calculates a density gradient value as a luminance gradient value in each pixel in each block, and each pixel in each block. The motion vector detection includes a representative point position determination unit, a motion vector detection unit, and a gradient direction correction unit that determine, as a representative point in each block, one pixel whose density gradient value satisfies a predetermined condition. parts, based on the representative points determined by the representative point position determination unit detects a motion vector between the plurality of frames by the representative point matching method, the concentration The distribution calculation unit calculates the density gradient value with respect to a gradient in a first direction that is one direction among a plurality of directions in each pixel in the block, and the representative point position determination unit calculates the first direction. The representative point in the block is determined based on the density gradient value, and the gradient direction correcting unit is based on a correlation between luminance values of the representative point and each pixel in the block including the representative point. Then, the direction from the position of the representative point toward the pixel having the highest correlation after the representative point is detected as the second direction for each block, and after the detection of the second direction, the density gradient calculation unit Calculates the density gradient value with respect to the gradient in the second direction at each pixel in the next block, and after the detection in the second direction, the representative point position determination unit performs the second direction The concentration gradient value of A motion vector detecting apparatus for determining the representative points in the next said block based.

According to a third aspect of the present invention, there is provided a motion vector detection device for detecting a motion vector by a representative point matching method from a video signal of a moving image composed of a plurality of temporally continuous frames, wherein the plurality of frames Each of the screens is divided into blocks of a predetermined size, and the first difference absolute value is a difference absolute value between a luminance value at each pixel in each block and a luminance value at a pixel shifted by a predetermined position from each pixel. A first difference absolute value calculation unit that calculates a value, and a representative point position that determines, as a representative point in each block, one pixel having the maximum first difference absolute value among the pixels in each block A determination unit; a motion vector detection unit; and a pixel shift amount determination unit, wherein the motion vector detection unit is configured to represent a representative point based on the representative point determined by the representative point position determination unit. A motion vector between the plurality of frames detected by etching method, the pixel shift amount determining unit, for each said block, the correlation of luminance values of said representative points and pixels in the block including the representative point Based on the above, the displacement from the representative point of the pixel whose correlation is higher than the representative point is detected as a pixel shift amount, the pixel shift amount is updated as the predetermined position, and after the predetermined position is updated, The first difference absolute value calculation unit calculates the first difference calculation value at each of the pixels in the next block based on the luminance value at the pixel shifted by the predetermined position after the update. After the update, the representative point position determination unit performs the next block based on the first difference calculation value calculated based on the luminance value in the pixel shifted by the predetermined position after the update. A motion vector detecting apparatus for determining the representative points at.

According to the first aspect of the present invention, the density gradient calculation unit that calculates the density gradient value in each pixel in the block, and one pixel that satisfies the predetermined condition for the density gradient value is determined as the representative point in each block. A representative point position determining unit, a motion vector detecting unit, and a gradient direction correcting unit, and the motion vector detecting unit is configured to generate a plurality of frames by a representative point matching method based on the representative points determined by the representative point position determining unit. The motion vector between is detected. Therefore, the position of the representative point can be made different for each block of each frame, and even in a flat image with a small density gradient, which is difficult to detect a motion vector, and a repeated pattern that is prone to malfunction, the malfunction is reduced and correct. A motion vector detection apparatus capable of detecting a motion vector can be realized.
The density gradient calculation unit calculates a density gradient value with respect to a gradient in a first direction, which is one of a plurality of directions, in each pixel in the block. The representative point position determination unit determines a representative point in the block based on the density gradient value in the first direction. Based on the correlation between the representative point and the luminance value of each pixel in the block including the representative point, the gradient direction correcting unit changes the direction from the position of the representative point toward the pixel having the highest correlation after the representative point. It detects as a 2nd direction for every block. After the detection in the second direction, the density gradient calculation unit calculates the density gradient value with respect to the gradient in the second direction at each pixel in the next block. After the detection in the second direction, the representative point position determination unit determines a representative point in the next block based on the density gradient value in the second direction. When a linear component is included in the image, if the representative point is determined based only on the density gradient value in the first direction, there is a possibility that the representative point may be erroneously detected. By detecting the direction and determining the representative point based on the second direction, it is possible to determine a more appropriate representative point. Therefore, a motion vector detection device capable of detecting a correct motion vector with higher reliability can be obtained.

According to the third aspect of the present invention, the first difference absolute value calculation unit that calculates the first difference absolute value between the luminance value at each pixel in the block and the luminance value at the pixel shifted from each pixel by a predetermined position. A representative point position determination unit that determines one pixel having the maximum first difference absolute value among the pixels in the block as a representative point in each block, a motion vector detection unit, a pixel shift amount determination unit, The motion vector detection unit detects a motion vector between a plurality of frames by a representative point matching method based on the representative point determined by the representative point position determination unit. Therefore, the position of the representative point can be made different for each block of each frame, and even in a flat image with a small density gradient, which is difficult to detect a motion vector, and a repeated pattern that is prone to malfunction, the malfunction is reduced and correct. A motion vector detection apparatus capable of detecting a motion vector can be realized. The pixel shift amount determining unit, for each block, and once determined representative points, based on the correlation of luminance values of the pixels in the block that contains the representative point, the correlation of next highest pixel representative point The displacement from the representative point is detected as the pixel shift amount, the pixel shift amount is updated as a predetermined position, and after the predetermined position is updated, the first difference absolute value calculation unit shifts the luminance at the pixel shifted by the updated predetermined position. Based on the value, the first difference calculation value is calculated at each pixel in the next block, and after updating the predetermined position, the representative point position determination unit is based on the luminance value at the pixel shifted by the predetermined position after the update. Based on the calculated first difference value, the representative point in the next block is determined. When a linear component is included in the image, the representative point may be erroneously detected. However, the pixel shift amount determination unit updates the pixel shift amount as a predetermined position, and the first difference absolute value is maximum. By determining one pixel as a representative point, a more appropriate representative point can be determined. Therefore, a motion vector detection device capable of detecting a correct motion vector with higher reliability can be obtained.

<Embodiment 1>
The present embodiment is a motion vector detection device that detects a motion vector from a video signal of a moving image composed of a plurality of temporally continuous frames by a representative point matching method, and includes a density at each pixel in a block. A motion vector detection unit includes a density gradient calculation unit that calculates a gradient value, a representative point position determination unit that determines, as a representative point in each block, one pixel that satisfies a predetermined condition, and a motion vector detection unit. The unit is a motion vector detection device that detects a motion vector between a plurality of frames by a representative point matching method based on the representative point determined by the representative point position determination unit.

  First, the principle of the representative point matching method will be described with reference to FIGS. In the representative point matching method, each screen of a plurality of frames is generally divided into blocks of a predetermined size, and a motion vector is detected for each block. However, here, for the sake of simplicity, a case will be described in which one motion vector is detected in the entire screen, not the motion vector for each block.

As shown in FIG. 1, a screen of a plurality of temporally continuous frames F (t) and F (t−1) (t is time) is represented by K × L pieces of pixels having a × b pixels. Divide into blocks. One representative point R is set for each block. From the luminance value at the representative point of the (k, l) -th block (k and l are horizontal and vertical block numbers in each frame) of the previous frame F (t−1) and the position of the representative point in the block ( The correlation value p ₍ with the luminance value at each pixel of the (k, l) block in the current frame F (t) displaced by i, j) (i and j are the number of horizontal and vertical pixels in the block). _{If k, l)} (i, j) is obtained for each displacement and the correlation value p _{(k, l)} (i, j) is cumulatively added for each displacement over all blocks, the correlation value P for the entire screen is obtained. (I, j) is obtained. The correlation value P (i, j) at this time is expressed as a correlation distribution on a plane having a size of one block.

  In general, the correlation value P (i, j) of an image has an inverted conical shape (conical shape) as shown in FIG. The vectors MV from the representative point R in the block up to the position where the minimum point of the correlation value P (i, j) is projected on the plane of one block are the frames F (t), F (t−1). It becomes a motion vector between. When the image is stationary between the frames F (t) and F (t−1), the minimum point is the origin (0, 0) corresponding to the representative point position of the frame F (t−1). . On the other hand, when the image moves between the frames F (t) and F (t−1), the shape of the mortar-like correlation distribution moves in parallel, so that this P (i, j) is minimized (i , J) can be detected as a motion vector.

  Next, the configuration of the motion vector detection device according to the present embodiment will be described. FIG. 3 is a block diagram showing the motion vector detection device according to the present embodiment. This motion vector detection device is mounted on a mobile phone, for example, for camera shake correction of a camera attached to the mobile phone. The motion vector detection apparatus is supplied with an analog video signal 10 from a camera attached to the mobile phone.

  The motion vector detection apparatus includes an A / D converter 11 that converts the analog video signal 10 into a digital video signal of a moving image composed of a plurality of temporally continuous frames. In addition, the motion vector detection device includes a density gradient calculation unit 12, a representative point position determination unit 13, and a motion vector detection unit.

The density gradient calculation unit 12 divides each screen of a plurality of frames of the digital video signal into blocks of a predetermined size (K × L blocks in FIG. 1), and each pixel in each block (in FIG. 1). In the case of (a × b pixels), a density gradient value that is a luminance gradient value is calculated. In addition, the representative point position determination unit 13 determines, as a representative point (a ₀ , b ₀ ) in each block, one pixel in which the density gradient value satisfies a predetermined condition among the pixels in each block. Here, the predetermined condition indicates that the density gradient value is maximum in each block.

The motion vector detection unit is a functional unit that detects a motion vector between a plurality of frames by the representative point matching method based on the representative points (a ₀ , b ₀ ) determined by the representative point position determination unit 13. Specifically, it is composed of a representative point memory 14, a difference absolute value calculation unit 15, a correlation value memory 16, and a correlation value comparison unit 17.

  Next, the operation of the motion vector detection device according to the present embodiment will be described. First, the analog video signal 10 is converted into a digital video signal by the A / D converter 11 and then input to the density gradient calculation unit 12 and the difference absolute value calculation unit 15. The density gradient calculation unit 12 calculates density gradient values from the calculation target pixel and surrounding pixel data using various known operators for calculating gradient values. This operator includes a horizontal Sobel operator shown in FIG. 4 corresponding to the first derivative operation, a vertical Sobel operator shown in FIG. 5, or a Laplacian operator shown in FIG. 6 corresponding to the second derivative operation. May be used. In addition, various operators used for feature point extraction in computer vision research, such as the Moravec operator expressed by the following equation 1, may be used.

The concentration gradient information calculated by the concentration gradient calculation unit 12 is input to the representative point position determination unit 13. The representative point position determination unit 13 determines, for each block, one pixel having the maximum density gradient value in the block as a representative point. In the prior art, the processing according to the present invention for determining the representative point (a ₀ , b ₀ ) by the representative point position determination unit 13 is not performed, and the representative point is, for example, the center point (a / 2, b / 2).

Information on the position (a ₀ , b ₀ ) of the representative point in each block and its luminance value is stored in the representative point memory 14 for each block. That is, in the representative point memory 14, the value of the representative point of the current frame F (t) is written. Further, the value of the representative point of the previous frame F (t−1) from the current frame is read from the representative point memory 14 by the difference absolute value calculation unit 15.

Then, the difference absolute value calculator 15 represents the luminance value of each pixel in each block of the current frame F (t) (this is ^expressed as d ^t _{(k, l)} (a ₀ + i, b ₀ + j)). , The luminance value of the representative point stored in the representative point memory 14 in each block of the previous frame F (t−1) (this is represented as d ^t−1 _{(k, l)} (a ₀ , b ₀ ). ) Is calculated (this is expressed as p _{(k, l)} (i, j)).

That is, the difference absolute value calculation unit 15 calculates the difference absolute value indicating the correlation between the representative point of the previous frame F (t−1) and the pixel of the current frame separated from the representative point by (i, j). The screen is divided into K × L blocks of size a × b, and pixels (a ₀ , b ₀ ) having a large density gradient from each block are used as representative points, and the previous frame F (t−1). As a correlation value between the representative point of the (k, l) -th block and each pixel in the block of the current frame F (t), the absolute difference p _{(k, l)} (i, j) is It is calculated as follows.

The correlation information of each block calculated by the difference absolute value calculation unit 15 is input to the correlation value memory 16. The correlation value memory 16 calculates the absolute difference for each pixel displacement from the position of the representative point of the previous frame F (t−1) stored in the representative point memory 14 over all of the K × L blocks. The values p _{(k, l)} (i, j) are accumulated and added, and the addition result is stored as a correlation value for each displacement (represented as P (i, j)). That is, in the correlation value memory 16, the absolute value of the difference p _{(k, l)} (i, j) is cumulatively added over all blocks for every displacement of (i, j), and finally one block worth. As a correlation distribution on the plane of the size of, information of correlation value P (i, j) indicating the correlation of the entire screen is obtained. When the correlation value P (i, j) of the entire screen is expressed by an equation,

It becomes.

  The correlation value comparison unit 17 has a displacement (i that minimizes the correlation value (that is, the correlation is maximum)) among the correlation values P (i, j) having a two-dimensional space of a × b stored in the correlation value memory 16. , J) are detected as motion vectors.

  Next, the difference between the case where the position of the representative point is fixed as in the prior art and the case of the present embodiment in which the position of the representative point is changed according to the density gradient value information will be described. For simplicity, it is assumed that a certain block is stationary as one-dimensional data as shown in FIG. When the position of the representative point is determined to be the center (O) position, the output of the absolute difference value obtained based on the representative point of this block is as shown in FIG. In FIG. 8, in addition to the position of the representative point (O), there is a wide area where the difference absolute value is “0” over the range O1.

  On the other hand, assuming that the position (P) where the maximum value of the density gradient is taken in FIG. 7 is a representative point, the output of the absolute difference value obtained based on the representative point of this block is as shown in FIG. In FIG. 9, there is no area where the difference absolute value is “0” except for the position P1 of the representative point (P), and the area where the difference absolute value is “0” is much smaller than in FIG.

  Eventually, since the data of other blocks are also cumulatively added to detect the minimum position of the correlation value, the influence of this one block is weakened. However, for the final motion vector detection, the case of FIG. In the case of FIG. 9 according to the present invention, the data is clearly more reliable. That is, representative point matching is facilitated.

  That is, according to the motion vector detection device according to the present embodiment, the density gradient calculation unit 12 that calculates the density gradient value in each pixel in the block, and one pixel that satisfies the predetermined condition for the density gradient value in each block. A representative point position determination unit 13 for determining a representative point and a motion vector detection unit, and a plurality of motion vector detection units based on the representative points determined by the representative point position determination unit 13 by a representative point matching method. The motion vector between frames is detected. Therefore, the position of the representative point can be made different for each block of each frame, and even in a flat image with a small density gradient, which is difficult to detect a motion vector, and a repeated pattern that is prone to malfunction, the malfunction is reduced and correct. A motion vector detection apparatus capable of detecting a motion vector can be realized.

  Further, according to the motion vector detection device according to the present embodiment, the predetermined condition indicates that the density gradient value is maximum in each block. When the pixel having the maximum density gradient value is determined as the representative point, the representative point matching is facilitated as described above. Therefore, it is possible to detect the motion vector with high accuracy.

  Further, according to the motion vector detection device according to the present embodiment, the motion vector detection unit includes the representative point memory 14, the difference absolute value calculation unit 15, the correlation value memory 16, and the correlation value comparison unit 17. . Therefore, the motion vector detection unit can be configured with a simple device configuration.

<Embodiment 2>
The present embodiment is a modification of the motion vector detection device according to the first embodiment, and the gradient direction detects the direction in which pixels having luminance values having a correlation with the luminance value of the representative point once determined are arranged. A correction unit is further provided. After the direction is detected, the density gradient calculation unit causes the density gradient value to be calculated with respect to the gradient in the detection direction at each pixel in each block, and the representative point position determination unit causes the density gradient value in the detection direction to be detected. The representative points in each block are determined based on the above.

  FIG. 10 is a diagram showing a motion vector detection device according to the present embodiment. The motion vector detection apparatus according to the present embodiment includes an A / D converter 11, a density gradient calculation unit 12, a representative point position determination unit 13, a representative point memory 14, a difference absolute value calculation unit 15, and a correlation according to the first embodiment. The value memory 16 and the correlation value comparison unit 17, the A / D converter 41, the density gradient calculation unit 42, the representative point position determination unit 43, the representative point memory 44, the difference absolute value calculation unit 45, and the correlation having the same functions. A value memory 46 and a correlation value comparison unit 47 are provided.

  The motion vector detection device according to the present embodiment further includes a gradient direction correcting unit that detects a direction in which pixels having luminance values having a correlation with the luminance value of the representative point once determined. Specifically, the gradient direction correction unit includes a difference absolute value calculation unit 48, a correlation value memory 49, a correlation value comparison unit 50, and a gradient direction determination unit 51.

  Next, the operation of the motion vector detection device according to the present embodiment will be described. First, an analog video signal 40 supplied from a camera attached to a mobile phone or the like is converted into a digital video signal by an A / D converter 41, and then is sent to a density gradient calculation unit 42 and difference absolute value calculation units 45 and 48. Entered.

  Unlike the case of the first embodiment, the density gradient calculation unit 42 calculates the density gradient value with respect to the gradient in the first direction, which is one of a plurality of directions, in each pixel in each block. More specifically, for example, one of the operators corresponding to differentiation in eight directions as shown in FIGS. 11 to 18 is selected to calculate the gradient information. 11 shows the upper left derivative, FIG. 12 shows the upper derivative, FIG. 13 shows the upper right derivative, FIG. 14 shows the right derivative, FIG. 15 shows the lower right derivative, and FIG. 16 shows the lower derivative. FIG. 17 shows each operator of the lower left direction differentiation, and FIG. 18 shows the left direction differentiation. Although a 3 × 3 operator is used here, an operator such as 5 × 5 may be used.

  Note that which of the eight-direction operators is used is selected based on information from the gradient direction determining unit 51 as described later. For the first operator selection, for example, an upper-right direction differential operator is used. It only has to be set.

  The concentration gradient information calculated by the concentration gradient calculation unit 42 is input to the representative point position determination unit 43. The representative point position determination unit 43 determines, for each block, one pixel having the maximum density gradient value in the first direction, which is one of the eight directions, as the representative point in the block.

Information on the position (a ₁ , b ₁ ) of the representative point in each block and its luminance value is stored in the representative point memory 44 for each block. That is, in the representative point memory 44, the value of the representative point of the current frame F (t) is written. Also, the value of the representative point of the previous frame F (t−1) from the current frame is read from the representative point memory 44 by the difference absolute value calculation unit 45.

Then, the difference absolute value calculation unit 45 represents the luminance value of each pixel in each block of the current frame F (t) (this is represented as d ^t _{(k, l)} (a ₁ + i, b ₁ + j)). , The luminance value of the representative point stored in the representative point memory 44 in each block of the previous frame F (t−1) (this is represented as d ^t−1 _{(k, l)} (a ₁ , b ₁ ). ) Is calculated (this is expressed as p _{(k, l)} (i, j)).

That is, the difference absolute value calculation unit 45 calculates the difference absolute value indicating the correlation between the representative point of the previous frame F (t−1) and the pixel of the current frame separated from the representative point by (i, j). The screen is divided into K × L blocks of size a × b, and pixels (a ₁ , b ₁ ) having a large density gradient are used as representative points in each block, and the previous frame F (t−1). As a correlation value between the representative point of the (k, l) -th block and each pixel in the block of the current frame F (t), the absolute difference value p _{(k, l)} (i, j) is the following equation 4 It is calculated as follows.

The correlation information of each block calculated by the difference absolute value calculation unit 45 is input to the correlation value memory 46. The correlation value memory 46 calculates the absolute difference for each displacement of each pixel from the position of the representative point of the previous frame F (t−1) stored in the representative point memory 44 over all of the K × L blocks. The values p _{(k, l)} (i, j) are accumulated and added, and the addition result is stored as a correlation value for each displacement (represented as P (i, j)). In other words, in the correlation value memory 46, the value of the absolute difference value p _{(k, l)} (i, j) is cumulatively added over all the blocks for every displacement of (i, j), and finally one block worth. As a correlation distribution on the plane of the size of, information of correlation value P (i, j) indicating the correlation of the entire screen is obtained. When the correlation value P (i, j) of the entire screen is expressed by an equation,

It becomes.

  The correlation value comparison unit 47 is a displacement (i that minimizes the correlation value (ie, the correlation is maximum) among the correlation values P (i, j) having a two-dimensional space of a × b stored in the correlation value memory 46. , J) are detected as motion vectors.

  Next, the operations of the gradient direction correction unit, that is, the difference absolute value calculation unit 48, the correlation value memory 49, the correlation value comparison unit 50, and the gradient direction determination unit 51 that did not appear in the first embodiment will be described. When the representative point position determination unit 43 determines a representative point of a certain intermediate block or the last block in the screen, it is written to the representative point memory 44 for correlation calculation of the next frame F (t + 1), and the difference It is also input to the absolute value calculation unit 48.

Then, the difference absolute value calculation unit 48 represents the luminance value of each pixel in each block of the current frame F (t) (this is ^expressed as d ^t _{(k, l)} (a ₁ + i, b ₁ + j)). The difference from the luminance value of the representative point determined by the representative point position determination unit 43 in each block of the current frame F (t) (this is ^expressed as d ^t _{(k, l)} (a ₁ , b ₁ )) An absolute value (represented as q _{(k, l)} (i, j)) is calculated.

That is, the difference absolute value calculation unit 48 does not represent the representative point of the previous frame F (t−1), but represents a representative point of the current frame F (t) and a pixel separated by (i, j) from the representative point of the current frame. The absolute difference value indicating the correlation with is calculated. The screen is divided into K × L blocks of size a × b, and pixels (a ₁ , b ₁ ) having a large density gradient from each block are used as representative points, and the current frame F (t) As a correlation value between the representative point of the (k, l) block and each pixel in the block of the current frame F (t), the difference absolute value q _{(k, l)} (i, j) is Is calculated.

The correlation information of each block calculated by the difference absolute value calculation unit 48 is input to the correlation value memory 49. The correlation value memory 49 is provided for every displacement of each pixel from the position of the representative point of the current frame F (t) determined by the representative point position determination unit 43 over all or part of the K × L blocks. The absolute difference value q _{(k, l)} (i, j) is accumulated and added, and the addition result is stored as a correlation value for each displacement (this is represented as Q (i, j)).

  The information on the correlation value Q (i, j) stored in the correlation value memory 49 is used for selection of the 8-way operator used in the concentration gradient calculation unit 42 when calculating the concentration gradient in the next adjacent block. . Expressing the in-screen correlation value Q (i, j) up to the middle of this screen or the entire screen by an equation,

It becomes.

  The correlation value comparison unit 50 selects the correlation value next to the correlation value (“0”) of the representative point from the correlation values Q (i, j) having a two-dimensional space of a × b stored in the correlation value memory 49. The displacement (i, j) that is the minimum (that is, the correlation is the maximum after the representative point) is detected as a highly correlated position in the screen until the middle of the screen or on the entire screen.

  Then, the gradient direction determining unit 51 determines the direction from the representative point position toward the displacement where the correlation value is the smallest next to the correlation value of the representative point, and the luminance having a correlation with the luminance value of the representative point for each block. The second direction, which is the direction in which the value pixels are arranged, is determined.

  After the detection in the second direction, the density gradient calculation unit 42 calculates the density gradient value with respect to the gradient in the second direction at each pixel in the next block. In the case of the gradient calculation in the second direction, one of the operators corresponding to the differentiation in the eight directions shown in FIGS. 11 to 18 is selected and used. After the detection in the second direction, the representative point position determination unit 43 determines a representative point in the next block based on the density gradient value in the second direction.

  Next, when the position of the representative point is simply changed according to the density gradient information as in the first embodiment, and the second direction with high correlation in the screen is detected as in the present embodiment, and the second A difference from the case where the position of the representative point is changed according to the concentration gradient in the direction will be described.

In general, changes in the density of walls, columns, desks, and the like that are the subject of an image appear as lines. For example, as shown in FIG. 19, when there is a large density gradient on the line (continuously in the depth direction), if only the intensity of the density gradient is considered without considering the direction, only the position (P) on the line is detected. Resulting in. As a result, the correlation value P (i, j) obtained by accumulatively adding the absolute difference value p _{(k, l)} (i, j) becomes a linearly low value as shown in FIG. In FIG. 19, since there is a luminance peak at the position (Q), the position P2 is desirable as a representative point in FIG. 20, but the correlation value P (i, j) in FIG. Therefore, the position P2 is not necessarily selected as the representative point. Therefore, there is a possibility that the representative point is erroneously detected.

On the other hand, as in the present embodiment, the direction in which the concentration gradient continues in a linear manner in FIG. 19 (that is, the depth direction in FIG. 19, and this direction corresponds to the luminance value of the representative point once determined. (Corresponding to the second direction, which is the direction in which pixels having correlated luminance values are aligned), and the gradient direction correcting unit recognizes the position of (Q) instead of (P) when focusing on the gradient in that direction. Is possible. As a result, the correlation value P (i, j) obtained by accumulatively adding the absolute difference value p _{(k, l)} (i, j) is lowered as shown in P2a as shown in FIG. This produces an effect and improves the accuracy of motion vector detection.

  That is, the motion vector detection device according to the present embodiment further includes a gradient direction correcting unit, and the gradient direction correcting unit includes pixels having luminance values having a correlation with the luminance value of the representative point once determined. The alignment direction is detected as the second direction, and after the detection of the second direction, the density gradient calculation unit 42 calculates the density gradient value with respect to the gradient in the second direction at each pixel in the next block, and After the detection in the two directions, the representative point position determination unit 43 determines a representative point in the next block based on the density gradient value in the second direction. When a linear component is included in the image, if the representative point is determined based only on the density gradient value in the first direction, there is a possibility that the representative point may be erroneously detected. By detecting the direction and determining the representative point based on the second direction, it is possible to determine a more appropriate representative point. Therefore, a motion vector detection device capable of detecting a correct motion vector with higher reliability can be obtained.

  Further, according to the motion vector detection device according to the present embodiment, the gradient direction correction unit includes the difference absolute value calculation unit 48, the correlation value memory 49, the correlation value comparison unit 50, and the gradient direction determination unit 51. Including. Therefore, the gradient direction correcting unit can be configured with a simple device configuration.

<Embodiment 3>
The present embodiment is a modification of the motion vector detection device according to the second embodiment, and employs a difference absolute value calculation unit instead of the density gradient calculation unit 42, and a pixel shift amount instead of the gradient direction correction unit. The decision part is adopted.

  In the second embodiment, the position having a high correlation with the luminance value of the representative point is detected, the gradient information in the direction is fed back as evaluation data, and the next representative point is selected. On the other hand, in the present embodiment, the position of a pixel having a luminance that is highly correlated with the luminance value of the representative point is detected, and the difference from the representative point up to that position is used as a direct shift amount. In the difference absolute value calculation unit adopted in this step, the pixel having the luminance that maximizes the difference absolute value between the luminance value of the pixel to be calculated and the luminance value of the pixel shifted by the shift amount is determined as the representative point.

  FIG. 22 is a diagram showing a motion vector detection device according to the present embodiment. The motion vector detection apparatus according to the present embodiment includes an A / D converter 41, a representative point memory 44, difference absolute value calculation units 45 and 48, correlation value memories 46 and 49, and a correlation value comparison unit 47 according to the second embodiment. , 50, an A / D converter 71 having a similar function, a representative point memory 74, difference absolute value calculation units 75, 78, correlation value memories 76, 79, and correlation value comparison units 77, 80.

  The motion vector detection apparatus according to the present embodiment further includes a difference absolute value calculation unit 72, a representative point position determination unit 73, and a pixel shift unit 81. Note that the representative point memory 74, the difference absolute value calculation unit 75, the correlation value memory 76, and the correlation value comparison unit 77 are based on the representative points determined by the representative point position determination unit 73, and a plurality of frames are represented by the representative point matching method. A motion vector detection unit that is a functional unit for detecting a motion vector between them is configured. Then, the absolute difference calculation unit 78, the correlation value memory 79, the correlation value comparison unit 80, and the pixel shift unit 81 include the pixel position of the luminance value having a correlation with the luminance value of the representative point for each block, and the representative point. A pixel shift amount determination unit configured to detect a difference from the position as a pixel shift amount and update the pixel shift amount as a predetermined position is configured.

  Next, the operation of the motion vector detection device according to the present embodiment will be described. First, an analog video signal 70 supplied from a camera or the like attached to a mobile phone is converted into a digital video signal by an A / D converter 71, and then a difference absolute value calculation unit 72 and difference absolute value calculation units 75 and 78 Input to the pixel shift unit 81.

  The difference absolute value calculation unit 72 divides each screen of a plurality of frames of the digital video signal into blocks of a predetermined size (K × L blocks in FIG. 1), and each pixel (FIG. 1) in each block. The absolute value of the difference between the luminance value at (a × b pixels) and the luminance value at the pixel shifted by a predetermined position from each pixel is calculated. Note that the information on the predetermined position is updated based on the information from the pixel shift unit 81 as will be described later. For the first predetermined position, for example, (a / 2, b / 2) is set. Good.

The representative point position determination unit 73 selects one pixel having the maximum difference absolute value calculated by the difference absolute value calculation unit 72 among the pixels in each block as a representative point (a ₂ , b ₂ ) and decide.

  That is, the difference absolute value calculation unit 72 uses the target pixel of the current frame F (t) and the target pixel of the current frame F (t) as the evaluation function for determining the representative point position from the target pixel of the current frame F (t) by the predetermined position ( First, for example, the absolute value of the difference from the pixel shifted by (a / 2, b / 2)) is calculated, and the representative point position determination unit 73 has the maximum evaluation data calculated by the absolute difference value calculation unit 72. The position is determined as a representative point for each block.

Information on the position (a ₂ , b ₂ ) of the representative point in each block and its luminance value is stored in the representative point memory 74 for each block. That is, in the representative point memory 74, the value of the representative point of the current frame F (t) is written. Further, the value of the representative point of the previous frame F (t−1) from the current frame is read from the representative point memory 74 by the difference absolute value calculation unit 75.

Then, the difference absolute value calculation unit 75 represents the luminance value of each pixel in each block of the current frame F (t) (this is ^expressed as d ^t _{(k, l)} (a ₂ + i, b ₂ + j)). The luminance value of the representative point stored in the representative point memory 74 in each block of the previous frame F (t−1) (this is represented as d ^t−1 _{(k, l)} (a ₂ , b ₂ ). ) Is calculated (this is expressed as p _{(k, l)} (i, j)).

That is, the difference absolute value calculation unit 75 calculates the difference absolute value indicating the correlation between the representative point of the previous frame F (t−1) and the pixel of the current frame separated by (i, j) from the representative point. The screen is divided into K × L blocks each having a size of a × b, and the pixel (a ₂ , b ₂ ) having the maximum difference absolute value calculated by the difference absolute value calculation unit 72 from each block. ) As a representative point, and as a correlation value between the representative point of the (k, l) -th block of the previous frame F (t−1) and each pixel in the same frame of the current frame F (t), the absolute difference value p _{( k, l)} (i, j) is calculated as shown in Equation 8 below.

The correlation information of each block calculated by the difference absolute value calculation unit 75 is input to the correlation value memory 76. The correlation value memory 76 calculates the absolute difference for each displacement of each pixel from the position of the representative point of the previous frame F (t−1) stored in the representative point memory 74 over all of the K × L blocks. The values p _{(k, l)} (i, j) are accumulated and added, and the addition result is stored as a correlation value for each displacement (represented as P (i, j)). That is, in the correlation value memory 76, the value of the absolute difference value p _{(k, l)} (i, j) is cumulatively added over all the blocks for every displacement of (i, j), and finally one block worth. As a correlation distribution on the plane of the size of, information of correlation value P (i, j) indicating the correlation of the entire screen is obtained. When the correlation value P (i, j) of the entire screen is expressed by an equation,

It becomes.

  The correlation value comparison unit 77 has a displacement (i that minimizes the correlation value (that is, the correlation is maximum)) among the correlation values P (i, j) having a two-dimensional space of a × b stored in the correlation value memory 76. , J) are detected as motion vectors.

  Next, the operations of the pixel shift amount determination unit provided in place of the gradient direction correction unit of the second embodiment, that is, the difference absolute value calculation unit 78, the correlation value memory 79, the correlation value comparison unit 80, and the pixel shift unit 81 are described. explain. When the representative point position determination unit 73 determines a representative point of a certain intermediate block or the last block in the screen, the representative point is written in the representative point memory 74 for correlation calculation of the next frame F (t + 1), and the difference The absolute value calculation unit 78 is also input.

Then, the difference absolute value calculation unit 78 represents the luminance value of each pixel in each block of the current frame F (t) (this is ^expressed as d ^t _{(k, l)} (a ₂ + i, b ₂ + j)). The difference from the luminance value of the representative point determined by the representative point position determination unit 73 in each block of the current frame F (t) _(denoted as d ^t _{(k, l)} (a ₂ , b ₂ )) An absolute value (represented as q _{(k, l)} (i, j)) is calculated.

That is, the difference absolute value calculation unit 78 does not represent the representative point of the previous frame F (t−1), but represents the representative point of the current frame F (t) and the pixel separated by (i, j) from the representative point of the current frame. The absolute difference value indicating the correlation with is calculated. The screen is divided into K × L blocks each having a size of a × b, and the pixel (a ₂ , b ₂ ) having the maximum difference absolute value calculated by the difference absolute value calculation unit 72 from each block. ) As a representative point, and as a correlation value between the representative point of the (k, l) -th block of the current frame F (t) and each pixel in the same block of the current frame F (t), the difference absolute value q _{(k, l)} (i, j) is calculated as in the following equation (10).

The correlation information of each block calculated by the difference absolute value calculation unit 78 is input to the correlation value memory 79. The correlation value memory 79 is provided for every displacement of each pixel from the position of the representative point of the current frame F (t) determined by the representative point position determination unit 73 over all or part of the K × L blocks. The absolute difference value q _{(k, l)} (i, j) is accumulated and added, and the addition result is stored as a correlation value for each displacement (this is represented as Q (i, j)).

  The information of the correlation value Q (i, j) stored in the correlation value memory 79 is used for determining a predetermined position used in the difference absolute value calculation in the next adjacent block in the difference absolute value calculation unit 72. The Expressing the in-screen correlation value Q (i, j) up to the middle of this screen or the entire screen by an equation,

It becomes.

  The correlation value comparison unit 80 selects the correlation value next to the correlation value (“0”) of the representative point from the correlation values Q (i, j) having a two-dimensional space of a × b stored in the correlation value memory 79. The displacement (i, j) that is the minimum (that is, the correlation is the maximum after the representative point) is detected as a highly correlated position in the screen until the middle of the screen or on the entire screen.

  Then, for each block, the pixel shift unit 81 detects, as a pixel shift amount, the displacement with the smallest correlation value from the representative point position next to the representative point, and the pixel shift amount is sent to the difference absolute value calculation unit 72. And updated as a predetermined position which is a shift amount of the pixel used. Then, the updated information on the predetermined position is input to the difference absolute value calculation unit 72.

  After the update of the predetermined position, the difference absolute value calculation unit 72 calculates the difference calculation value based on the luminance value at the pixel shifted by the predetermined position after the update, not the pixel shifted by (a / 2, b / 2). Is calculated at each pixel in the next block. Then, the representative point position determination unit 73 determines a representative point in the next block based on the difference calculation value calculated based on the luminance value in the pixel shifted by the updated predetermined position.

  In the second embodiment, a highly correlated position in the screen is detected, and gradient information in the direction is fed back as evaluation data to select the next representative point. In the third embodiment, the position in the screen is selected. The absolute value of the difference with the position itself as a direct shift amount is fed back as evaluation data. For this reason, it is possible to select a representative point with a more ideal correlation value.

  Next, in the monotonous and repetitive pattern in which the detection of the motion vector is very difficult, simulation images in the conventional case where the position of the representative point is fixed and in the case where the method of the third embodiment is used are shown in FIGS. It shows in FIG.

  FIG. 23 shows an input image, which is an 8-bit monochrome image having 320 horizontal pixels and 240 vertical pixels. For this input image, a total of 25 blocks of 5 × 5 are set in units of 48 pixels horizontally and 36 pixels vertically.

  FIG. 24 is a diagram illustrating a conventional case in which representative points are set while being fixed at the position of the white circle (a / 2, b / 2) at the center of each block. FIG. 25 is a diagram showing the case of the present embodiment in which the evaluation data in each block is overwritten and the position of a white circle having the maximum difference absolute value is determined as the representative point.

  The correlation information obtained from the fixed representative point in FIG. 24 is as shown in FIG. 26 with the detection range of the correlation value being 64 pixels in the horizontal direction and 48 pixels in the vertical direction, and the correlation information obtained from the representative point position in FIG. It became like this. FIG. 26 and FIG. 27 represent the correlation information as an 8-bit black and white image, ideally in a mortar shape in which the center point is the smallest (black) and the value increases (whiter) as the distance from the center increases. I just need it. In the case of FIG. 26, there are periodically black positions other than the center, and the shape is far from a mortar shape, but it can be confirmed that it is considerably improved in FIG.

  As described above, according to the present invention, it is possible to detect a motion even with a monotonous repetitive pattern or the like in which it has been difficult to detect a motion vector at a representative point at a conventional fixed position.

  That is, according to the motion vector detection device according to the present embodiment, the difference absolute value calculation that calculates the difference absolute value between the luminance value of each pixel in the block and the luminance value of the pixel shifted from the pixel by a predetermined position. Unit 72, representative point position determination unit 73 for determining, as a representative point in each block, one pixel having the maximum difference absolute value among the pixels in the block, a motion vector detection unit, and a pixel shift amount determination unit The motion vector detection unit detects a motion vector between a plurality of frames by the representative point matching method based on the representative point determined by the representative point position determination unit 73. Therefore, the position of the representative point can be made different for each block of each frame, and even in a flat image with a small density gradient, which is difficult to detect a motion vector, and a repeated pattern that is prone to malfunction, the malfunction is reduced and correct. A motion vector detection apparatus capable of detecting a motion vector can be realized. In addition, the pixel shift amount determination unit detects, as a pixel shift amount, the difference between the pixel position of the luminance value having a correlation with the luminance value of the representative point once determined for each block and the position of the representative point. The pixel shift amount is updated as a predetermined position, and after the predetermined position is updated, the difference absolute value calculation unit 72 calculates the difference calculation value based on the luminance value in the pixel shifted by the updated predetermined position in the next block. After the predetermined position is updated, the representative point position determination unit 73 calculates the next block based on the difference calculation value calculated based on the luminance value at the pixel shifted by the updated predetermined position. A representative point is determined. When a linear component is included in the image, the representative point may be erroneously detected. However, the pixel shift amount determination unit updates the pixel shift amount as a predetermined position, and the difference absolute value is maximized. By determining a pixel as a representative point, a more appropriate representative point can be determined. Therefore, a motion vector detection device capable of detecting a correct motion vector with higher reliability can be obtained.

  Further, according to the motion vector detection device according to the present embodiment, the motion vector detection unit includes the representative point memory 74, the difference absolute value calculation unit 75, the correlation value memory 76, and the correlation value comparison unit 77. . Therefore, the motion vector detection unit can be configured with a simple device configuration.

  Further, according to the motion vector detection device according to the present embodiment, the pixel shift amount determination unit includes the difference absolute value calculation unit 78, the correlation value memory 79, the correlation value comparison unit 80, and the pixel shift unit 81. Including. Therefore, the pixel shift amount determination unit can be configured with a simple device configuration.

  As an application example of the present invention, in addition to conventional applications such as camera shake correction, the present invention can be applied to a motion sensor using a video signal from a camera, and can be used for an input interface such as a game or character input.

It is a figure explaining the principle of a representative point matching method. It is a figure explaining the principle of a representative point matching method. 1 is a block diagram illustrating a motion vector detection device according to Embodiment 1. FIG. It is a figure which shows a horizontal direction Sobel operator. It is a figure which shows a vertical direction Sobel operator. It is a figure which shows a Laplacian operator. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 1. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 1. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 1. FIG. 6 is a block diagram showing a motion vector detection device according to Embodiment 2. FIG. It is a figure which shows the operator of upper left direction differentiation. It is a figure which shows the operator of an upward differentiation. It is a figure which shows the operator of an upper right direction differentiation. It is a figure which shows the operator of right direction differentiation. It is a figure which shows the operator of a lower right direction differentiation. It is a figure which shows the operator of a downward differentiation. It is a figure which shows the operator of a lower left direction differentiation. It is a figure which shows the operator of left direction differentiation. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 2. FIG. 10 is a block diagram illustrating a motion vector detection device according to Embodiment 3. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the effect of the motion vector detection apparatus which concerns on Embodiment 3. FIG.

Explanation of symbols

12, 42 Density gradient calculation unit, 13, 43, 73 Representative point position determination unit, 14, 44, 74 Representative point memory, 15, 45, 48, 72, 75, 78 Difference absolute value calculation unit, 16, 46, 49 , 76, 79 Correlation value memory, 17, 47, 50, 77, 80 Correlation value comparison unit, 51 Gradient direction determination unit, 81 Pixel shift unit.

Claims (5)

A motion vector detection device that detects a motion vector by a representative point matching method from a video signal of a moving image composed of a plurality of temporally continuous frames,
A density gradient calculator that divides each screen of the plurality of frames into blocks of a predetermined size and calculates a density gradient value that is a luminance gradient value in each pixel in each of the blocks;
A representative point position determination unit that determines, as a representative point in each block, one pixel in which the density gradient value satisfies a predetermined condition among the pixels in each of the blocks;
A motion vector detection unit ;
A gradient direction correcting section ,
The motion vector detection unit detects a motion vector between the plurality of frames by a representative point matching method based on the representative point determined by the representative point position determination unit ;
The density gradient calculation unit calculates the density gradient value with respect to a gradient in a first direction, which is one direction among a plurality of directions, in each pixel in the block,
The representative point position determination unit determines the representative point in the block based on the density gradient value in the first direction,
The gradient direction correction unit is a pixel having a correlation higher than the representative point from the position of the representative point based on a correlation of luminance values between the representative point and each pixel in the block including the representative point. Detecting the direction toward the second direction for each block,
After the detection in the second direction, the density gradient calculation unit calculates the density gradient value with respect to the gradient in the second direction at each pixel in the next block,
After the detection in the second direction, the representative point position determination unit determines the representative point in the next block based on the density gradient value in the second direction . The motion vector detection device according to claim 1,
The gradient direction correction unit includes:
Difference absolute value calculation unit,
Correlation value memory;
A correlation value comparison unit;
Gradient direction determination unit
Including
The difference absolute value calculation unit is determined by the luminance value of each pixel in each block of the current frame of the plurality of frames and the representative point position determination unit in each block of the current frame. The difference absolute value with the luminance value of the representative point is calculated,
The correlation value memory accumulates the absolute difference value for every displacement of each pixel from the position of the representative point determined by the representative point position determination unit over all or part of the block. Adding, storing the addition result as a correlation value for each displacement,
The correlation value comparison unit detects a displacement at which the correlation value is the second smallest after the correlation value of the representative point,
The gradient direction determining unit determines, as the second direction, for each block, a direction from the position of the representative point toward the displacement at which the correlation value is the second smallest after the correlation value of the representative point. > Motion vector detector. A motion vector detection device that detects a motion vector by a representative point matching method from a video signal of a moving image composed of a plurality of temporally continuous frames,
Dividing each screen of the plurality of frames into blocks of a predetermined size, which is a difference absolute value between a luminance value in each pixel in each block and a luminance value in a pixel shifted by a predetermined position from each pixel, A first difference absolute value calculation unit for calculating a first difference absolute value;
A representative point position determination unit that determines, as a representative point in each block, one pixel having the maximum first difference absolute value among the pixels in each block;
A motion vector detection unit;
A pixel shift amount determination unit and
With
The motion vector detection unit detects a motion vector between the plurality of frames by a representative point matching method based on the representative point determined by the representative point position determination unit;
For each block, the pixel shift amount determination unit is configured to determine a pixel having the second highest correlation after the representative point based on a correlation between luminance values of the representative point and each pixel in the block including the representative point. Detecting a displacement from the representative point as a pixel shift amount, and updating the pixel shift amount as the predetermined position;
After the update of the predetermined position, the first difference absolute value calculation unit calculates the first difference calculation value based on the luminance value in the pixel shifted by the predetermined position after the update. Operate on pixels,
After the update of the predetermined position, the representative point position determination unit performs the next intra-block based on the first difference calculation value calculated based on the luminance value in the pixel shifted by the predetermined position after the update. A motion vector detection device for determining the representative point in the . The motion vector detection device according to claim 3 ,
The motion vector detection unit
Representative point memory,
A second difference absolute value calculation unit;
A first correlation value memory;
A first correlation value comparison unit;
Including
The representative point memory stores the position of the representative point determined by the representative point position determination unit and its luminance value for each of the blocks,
The second difference absolute value calculation unit includes a luminance value of each pixel in each block of a current frame of the plurality of frames, and the block in each block of a frame immediately before the current frame. Calculating a second difference absolute value with the luminance value of the representative point stored in the representative point memory;
The first correlation value memory accumulates and adds the second difference absolute value for each displacement of the pixel from the position of the representative point stored in the representative point memory over the entire block. And storing the addition result as a first correlation value for each displacement,
The first correlation value comparison unit is a motion vector detection device that detects a displacement that minimizes the first correlation value as a motion vector. The motion vector detection device according to claim 3 ,
The pixel shift amount determination unit includes:
A third difference absolute value calculation unit;
A second correlation value memory;
A second correlation value comparison unit;
Pixel shift unit and
Including
The third difference absolute value calculation unit includes a luminance value of each pixel in each block of the current frame of the plurality of frames and a representative point position determination unit in each block of the current frame. Calculating a third difference absolute value with the luminance value of the representative point determined;
The second correlation value memory stores the third difference absolute value for each displacement of the pixel from the position of the representative point determined by the representative point position determination unit over all or part of the block. Are accumulated, and the addition result is stored as a second correlation value for each displacement,
The second correlation value comparison unit detects a displacement at which the second correlation value is the second smallest after the correlation value of the representative point,
The pixel shift unit detects, as the pixel shift amount, the displacement at which the correlation value from the position of the representative point is the second smallest after the correlation value of the representative point for each block, and determines the pixel shift amount. A motion vector detection device that updates the predetermined position .