JP2006217066A - Motion vector detection method, motion vector detection apparatus, motion vector detection program, and program recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology capable of correctly detecting a motion vector of an object with a small number of operation times by correcting an initial displacement vector on the basis of a characteristic of a motion of a whole extracted image. <P>SOLUTION: Motion vectors of a plurality of blocks located in the vicinity of blocks to be detected in a present field are read from a vector memory 3 as a first candidate vector group of the initial displacement vector of the block to be detected, a whole image motion characteristic detection section 2 detects a change in a motion characteristic of the whole image between the present field and at least a one-preceding field to obtain an amount of change, an initial displacement vector correction section 4 obtains a second candidate vector group resulting from correcting the first candidate vector group on the basis of the amount of change, an initial displacement vector selection section 5 selects a motion vector with a value most appropriate to a motion of the block to be detected in the present field from the first and second candidate vector groups as the initial displacement vector, and a motion vector arithmetic unit 6 obtains the motion vector of the block to be detected by one or a plurality of arithmetic operations by using the initial displacement vector for a start point. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a motion vector detection method, a motion vector detection device, a motion vector detection program, and a program recording medium for moving image signals, and more particularly, for example, a television system conversion device that performs video signal conversion between different television systems. The present invention also relates to a motion vector detection method, a motion vector detection device, a motion vector detection program, and a program recording medium that can be suitably applied to a high-efficiency encoding device for image transmission.

  A technique for detecting a motion region in a moving image signal as a motion vector and performing motion correction using the motion vector, for example, when improving the inter-frame coding efficiency in high-efficiency coding at the time of image transmission, or It is particularly effective for reducing motion discontinuity due to field number conversion during television system conversion. For example, as a television system conversion device, Japanese Patent Application Laid-Open No. 01-309597 disclosed in Japanese Patent Application Laid-Open No. 01-309597 “Television Standard System Conversion Device” and Japanese Patent Application Laid-Open No. 03-280681 disclosed in Japanese Patent Application Laid-Open No. 03-280681 disclosed a motion using motion vectors. "Interpolation method", Journal of the Television Society shown in Non-Patent Document 1 (Vol. 45, No. 12, pages 1534 to 1543, published in 1991), or the National Convention of Television Society shown in Non-Patent Document 2 The technology described in the proceedings of the book (20-5, held in 1989) is proposed.

  Conventionally, as a motion vector detection method used in the television system conversion device described in each of the above-mentioned documents, for example, Japanese Patent Application Laid-Open No. 55-162683 shown in “Patent Document 3” ”And Japanese Patent Laid-Open No. 55-162684 shown in“ Patent Document 4 ”,“ Detective Method for Asymptotic Detection of Image Motion Vectors ”, or Japanese Patent Laid-Open No. 60-158786 shown in Patent Document 5. Detection methods using the iterative gradient method described in “Image Motion Amount Detection Method” and Japanese Patent Application Laid-Open No. 62-206980 “Initial Deviation Method in Motion Estimation of Moving Image” shown in Patent Document 6 have been proposed. .

  In particular, the latter motion vector detection method based on the iterative gradient method is smaller and more accurate than the pattern matching method, and can detect a motion vector. That is, the motion vector detection method based on the iterative gradient method has a predetermined predetermined size of m × n pixels including, for example, m pixels in the horizontal direction and n lines in the vertical direction, for each field of the digitized television signal. By subdividing into blocks, each block is subjected to repetitive gradient calculation based on the signal gradient in the screen and the physical correspondence of the signal difference value between the corresponding screens The amount of motion was estimated.

  In this case, in Japanese Patent Application Laid-Open No. 62-206980 of Patent Document 6, detection is already made in a plurality of peripheral blocks including a block corresponding to the detected block as an initial value when estimating the motion amount. By selecting an optimal motion vector detection target motion vector detection vector from among motion vector candidates as an initial displacement vector, and starting calculation from a value close to the true motion vector of the detected block For example, there has been proposed a technique for detecting a true motion vector by reducing the number of times of gradient method calculation, for example, by two gradient method calculations.

Japanese Patent Laid-Open No. 06-217266 “Motion Vector Detection Circuit” shown in Patent Document 7 discloses that at least between one block of image signals separated by at least one field or one frame in order to further improve the accuracy of motion vector detection. A method for detecting an initial displacement vector of motion has also been proposed.
Japanese Laid-Open Patent Publication No. 01-309597 Japanese Patent Laid-Open No. 03-280681 Japanese Unexamined Patent Publication No. 55-162683 Japanese Patent Laid-Open No. 55-162684 JP 60-158786 A Japanese Patent Laid-Open No. 62-206980 Japanese Patent Laid-Open No. 06-217266 The Journal of the Institute of Television Engineers of Japan (Vol. 45, No. 12, pages 1534 to 1543, published in 1991) Proceedings of the annual conference of the Institute of Television Engineers of Japan (20-5, 1989)

  However, in the conventional technique described above, when selecting an initial displacement vector from among a plurality of candidate vectors, the initial displacement vector is individually selected for each block, so that the entire image moves in a biased manner. In some situations, the accuracy of motion vector detection may be reduced when there is an object in the image that moves differently from the overall motion.

  For example, consider a case where the camera is panned while photographing a moving object. At this time, the background other than the moving object is moving at the same moving speed in the same direction by the panning operation, while the moving object is moving at a small speed near the center of the shooting screen. become. When the camera is panned in this way, the initial displacement vector of the block in which the moving object is present is selected from the block in which the object is present one field before in the conventional technique.

  However, since the moving speed of the object in the screen is actually a combination of the moving speed of panning and the moving speed of the object, if the moving speed of panning changes greatly, the moving speed of one field before In order to use the motion from the block in which the object exists as it is as the initial displacement vector, there is a problem that an error becomes large.

  In particular, when shooting a ball game or the like, since shooting is performed so as to catch the ball in the screen, when the ball moves greatly, panning is performed so that the ball is positioned substantially at the center of the screen. At this time, the moving speed of the ball before panning and the moving speed after panning change greatly on the screen, and the ball can be selected from any position before one field when selecting the initial displacement vector. An error occurs in the initial displacement vector.

  For example, if the ball movement direction is the same as the panning direction and the ball movement speed is lower than the panning movement speed, the ball on the screen moves in the opposite direction immediately after panning. It will have a vector, and the position of the initial displacement vector of the ball will be wrong even if it is selected from any position before one field.

Here, the accuracy of the initial displacement vector of the ball that is the subject greatly affects the detection of the motion vector of the ball, so if there is a change in the panning movement speed, an erroneous detection of the motion vector will result. Therefore, when the television system conversion or the like is performed, the ball is displayed at an incorrect position.
That is, the above-described conventional technique has a drawback in that the motion vector of the moving subject is erroneously detected because the initial displacement vector is not calculated in consideration of the motion characteristics of the entire image.

  The present invention has been made to solve the above-described problems, and extracts an initial displacement vector by extracting a motion feature of the entire screen and correcting the initial displacement vector based on the motion feature of the entire screen. It is an object of the present invention to provide a technique capable of preventing a false detection of a block and correctly detecting a motion vector for each block including a moving object with a small number of calculations.

  The first technical means divides a field of a moving image signal into a plurality of blocks having a predetermined size, and each of the divided blocks has a motion between at least one field apart. In a motion vector detection method of a moving image signal for detecting a motion vector representing a magnitude and direction, a full screen motion feature detection step for detecting a motion feature of the entire screen and calculating a change amount, and a detection for each block A storage step for accumulating at least one field of motion vectors, and a motion vector of a plurality of blocks located in the vicinity of the detected block among the motion vectors stored in the storing step is used for selecting an initial displacement vector of the detected block. Are read out as a first candidate vector group, and the full screen is further added to each of the first candidate vector group. A candidate vector correcting step for calculating a second candidate vector group by performing correction based on the change amount indicating a change in the full-screen motion feature detected by the feature detecting step, and a motion vector accumulated by the storing step. A motion vector having a value most suitable for the motion of the detected block is detected from the first candidate vector group read out from the second candidate vector group corrected by the candidate vector correcting step. An initial displacement vector selection step for selecting the initial displacement vector of the detected block, and a motion vector of the detected block is obtained by performing one or more operations using the initial displacement vector selected in the initial displacement vector selection step as a starting point. At the same time, the calculated motion vector Characterized in that it has a motion vector calculation step of storing by the storing step for vector calculation.

  A second technical means is the motion vector detection method according to the first technical means, wherein the full-screen motion feature detection step is shifted toward the center by a predetermined number of blocks from each of the four corners on the screen. The motion feature of the block at the selected position is detected as the motion feature of the entire screen, or at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center, The motion feature of a large area is detected as the motion feature of the entire screen, or is located at a position shifted toward the center by a predetermined number of blocks from the four corners on the screen and / or from the top, bottom, left and right sides of the screen. It is characterized in that either of the motion features of a plurality of block groups is detected as the motion feature of the entire screen.

  A third technical means is the motion vector detection method according to the first technical means, wherein the full-screen motion feature detection step includes a feature point on a predetermined image included in the video signal. Is detected as a movement feature of the entire screen.

  The fourth technical means divides the field of the moving image signal into a plurality of blocks having a predetermined size, and moves each moving block between moving image signals separated by at least one field. In a motion vector detection device for a moving image signal that detects a motion vector representing a size and a direction, a motion feature detection unit that detects motion features of the entire screen and calculates a change amount, and a motion vector detection device that is detected for each block Storage means for storing motion vectors for at least one field, and for selecting an initial displacement vector of a detected block, motion vectors of a plurality of blocks located in the vicinity of the detected block among the motion vectors stored by the storage means As a first candidate vector group, and the full-screen motion feature detection is further performed for each of the first candidate vector groups. A candidate vector correcting unit that calculates a second candidate vector group by performing correction based on the change amount indicating a change in the full-screen motion feature detected by the stage, and a motion vector stored in the storage unit. From the read first candidate vector group and the second candidate vector group corrected by the candidate vector correcting unit, a motion vector having a value most suitable for the motion of the detected block is determined as an initial displacement of the detected block. Initial displacement vector selection means for selecting as a vector, and performing one or more operations from the initial displacement vector selected by the initial displacement vector selection means to obtain and output a motion vector of the detected block; The obtained motion vector is stored in the storage step for calculating the motion vector of the moving image signal after the next field. Characterized in that it includes a motion vector calculation means for accumulating the.

  A fifth technical means is the motion vector detecting device according to the fourth technical means, wherein the full-screen motion feature detecting means is shifted toward the center by a predetermined number of blocks from the four corners on the screen. The motion feature of the block at the selected position is detected as the motion feature of the entire screen, or at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center, The motion feature of a large area is detected as the motion feature of the entire screen, or is located at a position shifted toward the center by a predetermined number of blocks from the four corners on the screen and / or from the top, bottom, left and right sides of the screen. It is characterized in that either of the motion features of a plurality of block groups is detected as the motion feature of the entire screen.

  A sixth technical means is the motion vector detection device according to the fourth technical means, wherein the full-screen motion feature detection means includes a feature point on a predetermined image included in the moving image signal. Is detected as a movement feature of the entire screen.

  A seventh technical means is characterized in that the motion vector detection method according to any one of the first to third technical means is a motion vector detection program realized as a program executable by a computer. .

  The eighth technical means is a program recording medium in which the motion vector detection program described in the seventh technical means is recorded on a computer-readable recording medium.

According to the motion vector detection method, the motion vector detection device, the motion vector detection program, and the program recording medium according to the present invention composed of the technical means as described above, the following effects can be obtained.
When the initial displacement vector is selected from the detected motion vectors and the motion vector is detected using the initial displacement vector, the variation amount of the motion feature of the entire screen is added to the detected motion vector. By using a motion vector, that is, a prediction vector, it is possible to select a more correct initial displacement vector without being affected by a change in motion over the entire screen.

  As a result, it is possible to reduce false detection of the initial displacement vector at the time of camera panning, tilting, zoom start point and camera operation speed change, and to greatly improve motion vector detection accuracy. Become.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments for implementing a motion vector detection method, a motion vector detection device, a motion vector detection program, and a program recording medium according to the present invention will be described with reference to the drawings. In the following description, an example of an embodiment related to a motion vector detection device according to the present invention will be described, and description of an embodiment related to a motion vector detection method, a motion vector detection program, and a program recording medium according to the present invention will be omitted. From the following detailed description of the motion vector detection device, a motion vector detection method applicable to the motion vector detection device, a vector detection program for executing the motion vector detection method as a program by a computer, and the vector detection program It is also possible to easily understand a program recording medium for recording the program on a computer-readable recording medium.

First, an embodiment of a motion vector detection apparatus according to the present invention for detecting a motion vector of a moving image signal will be described in detail with reference to FIG. FIG. 1 is a functional block configuration diagram illustrating an example of a functional configuration of a motion vector detection device according to the present invention, to which an example of a motion vector detection method according to the present invention is applied.
1 includes a field delay unit 1, a full screen motion feature detection unit 2, a vector memory 3, an initial displacement vector correction unit 4, an initial displacement vector selection unit 5, a motion vector calculation. The unit 6 is configured to be included.

  The motion vector detection apparatus 10 according to the present embodiment divides a moving image signal input for each field into a plurality of blocks having a predetermined size, for example, m pixels × n lines (m and n are integers). Thus, for each divided block, a motion for obtaining a motion vector representing the direction and magnitude of motion with the corresponding block in the moving image signal, for example, one field before delayed by the field delay unit 1 A first candidate vector group selected from the motion vectors already detected and stored in the vector memory 3, and a second candidate vector corrected by the initial displacement vector correction unit 4; Together with the group, the motion vector selected by the initial displacement vector selection unit 5 is selected as the initial displacement vector of the detected block, and the initial displacement vector As a starting point, with a small number of calculations for example, twice the gradient operation, and a vector operation unit 6 for obtaining the true motion vector of 該被 detection block correctly.

  In particular, the present embodiment is characterized in how to obtain a candidate vector of an initial displacement vector, and a motion vector by a camera motion such as panning, tilt, zoom, or the like, that is, a feature of the motion of the entire screen is represented by a full screen motion feature detection unit 2. In addition, the initial displacement vector correction unit 4 calculates the change in the motion characteristics of the entire screen as change amount data, and the motion vector candidate read from the vector memory 3 is detected by the initial displacement vector correction unit 4. A more accurate initial displacement vector is selected by performing correction based on the change amount data and outputting it as a candidate vector used for selection of the initial displacement vector.

  More specifically, the motion vector detection device 10 according to the present embodiment shown in FIG. 1 includes the full screen motion feature detection unit 2, the vector memory 3, the initial displacement vector correction unit 4, the initial displacement as described above. A vector selection unit 5 and a motion vector calculation unit 6 are included. The full-field motion feature detection unit 2, the initial displacement vector selection unit 5, and the motion vector calculation unit 6 are supplied with the current field signal and the previous field signal delayed by one field via the field delay unit 1, respectively. ing.

  The full-screen motion feature detection unit 2 divides the previous field signal into blocks of m pixels × n lines as described above, for example, and determines in advance to detect the motion features of the full screen from the divided blocks. By using a plurality of specific blocks and comparing them with the current field signal, the motion characteristics of the entire screen are detected. The initial displacement vector selection unit 5 uses the current field signal and the previous field signal as a reference for selecting an initial displacement vector for each divided block. In addition, the motion vector calculation unit 6 uses the current field signal and the previous field signal to detect the motion vector for each block starting from the initial displacement vector.

  The vector memory 3 is a storage circuit including a RAM (Random Access Memory) that accumulates motion vectors for at least one field already detected for each block, and its input terminal serves as an output terminal of the motion vector calculation unit 6. The motion vector detected by the motion vector calculation unit 7 is sequentially updated and accumulated at an address corresponding to the position of each block divided into, for example, 8 pixels × 8 lines. Has been.

  As shown in FIG. 2, the full-screen motion feature detection unit 2 compares a partial region or a plurality of blocks in the previous field with the image in the current field, and compares the background motion vector (that is, all the motion vectors). Motion characteristic of the screen) is detected and output to the initial displacement vector correction unit 5 as change amount data to be corrected for the initial displacement vector. The background motion vector detection method can be realized by using a normal motion vector detection method.

  FIG. 2 is a schematic diagram showing an example of a full-screen feature extraction block that is referred to when a background motion vector is detected as a full-screen motion feature in the motion vector detection apparatus 10 according to the present invention. This is an example of the position and size of the feature extraction block on the screen for detecting. In general, the detection of motion characteristics of the entire screen can be realized by performing a motion vector calculation on a part of blocks or areas of the screen. Although not particularly illustrated, at this time, the motion vector data stored in the vector memory 3 may be used for motion vector calculation.

  Further, the full-screen feature extraction unit 2 refers to a full-screen feature extraction block that is referred to when the motion vector indicating the presence / absence of motion in the background (direction and size when present) is detected as the motion feature of the full screen. It is preferable to use a block at a position where the background is surely reflected in the screen, as shown in FIG. 2 (A), at a position closer to the center by a predetermined number of blocks from the four corners of the screen, It is more preferable to use four blocks a, b, c and d located in the vicinity of the four corners of the screen. This is because the subject to be photographed often appears around the center of the screen, and on the other hand, in the case of the periphery of the screen, there is a high possibility that a feature image will be mixed from outside the screen, so it is difficult to detect the motion vector of the entire image Because.

  Alternatively, as shown in FIG. 2 (B), the four blocks a, b, c, and 4 in FIG. A plurality of block groups a ′, b ′, c ′, d ′ having an area larger than d may be used, and as shown in FIG. 2C, the four blocks a in FIG. , B, c, d, including a plurality of neighboring block groups (the number of blocks is arbitrary) and / or a plurality of block groups located closer to the center by a predetermined number of blocks from the top, bottom, left, and right sides of the screen. Thus, a change such as zoom may be detected.

  In addition, the full screen motion feature detection unit 2 not only detects only one motion feature vector on the entire screen, but also considers that there may be a difference in motion feature vectors depending on the zoom or the like. A motion feature vector may be detected for each feature extraction block. However, when performing correction, it is necessary to determine in advance which motion feature vector is used and which initial displacement vector of a block at which position is to be corrected. For example, the motion feature vector of each block from a to d in FIG. 2A is obtained by dividing the screen into four screen areas a ″, b ″, c ″, d ″ as shown in FIG. 2D. It is necessary to determine in advance that it will be used to correct the initial displacement vector in each case.

  A more specific configuration example of the full-screen motion feature detection unit 3 is shown in FIG. FIG. 3 is a block configuration diagram showing an example of a specific circuit configuration of the full-screen motion feature detection unit 2 of the motion vector detection device 10 according to the present invention. As shown in FIG. 3, the full-screen motion feature detection unit 2 includes a motion vector calculation unit 2a for obtaining a full-screen motion feature vector for each full-screen motion feature extraction block as illustrated in FIG. A full-screen motion feature vector memory 2b including a RAM for storing a full-screen motion feature vector for one field detected for each extraction block, and a full-screen motion feature vector of the current field, and a full-screen motion feature vector The vector difference unit 2c calculates a vector difference between two vectors of the previous field full-screen motion feature vector stored in the memory 2b, and the vector difference is used as change amount data indicating a change in the full-screen motion feature. Is configured to output as

  In the full-screen motion feature detection unit 2 shown in FIG. 3, the full-field motion image signal of the current field, the previous-field motion image signal of the previous field, and the full-field of the previous field stored in the full-screen motion feature vector memory 2b. An example is shown in which the full-screen motion feature vector for each full-screen motion feature extraction block calculated by the motion vector calculation unit 2a is calculated using the motion feature vector.

Also, the initial displacement vector correction unit 4 shown in FIG. 1 corrects the first candidate vector group stored in the vector memory 3 using the motion characteristics of the entire screen, and uses the first candidate vector group for selection of the initial displacement vector. A candidate vector correcting means for outputting as a second candidate vector group is configured. That is, the first candidate vector group is corrected based on the change amount data from the full screen motion feature detection unit 2 (that is, data indicating the change of the full screen motion feature) to obtain the second candidate vector group. FIG. 4 is a block configuration diagram showing an example of a specific circuit configuration of the initial displacement vector correction unit 4 of the motion vector detection device 10 according to the present invention. In the vector subtraction unit 4a, all the vectors from the first vector group are shown. In the example, the second candidate vector group is obtained by performing vector subtraction on the change amount data calculated by the screen motion feature detection unit 2.
Here, in the present embodiment, the first candidate vector group uses the motion vector of the previous field detected in eight blocks around the detected block as the candidate vector group for selecting the initial displacement vector of the detected block. And Note that the first candidate vector group is not limited to such an example, and may be determined from already detected motion vectors in other regions.

Specifically, the second candidate vector group is calculated in the initial displacement vector correction unit 4 by performing the following calculation. Assuming that the first candidate vector group stored in the vector memory 3 is Va and the motion feature vector that is the variation data from the full-screen motion feature detection unit 2 is Vs, the second candidate vector group Vb is (1).
Vb = Va−Vs (1)

Next, the initial displacement vector selection unit 5 shown in FIG. 1 selects the motion vector having the most suitable value for the motion of the detected block from the detected motion vectors obtained by the motion vector calculation of the previous field, as the starting point of the gradient method calculation. The selection circuit selects as an initial displacement vector, and selects an appropriate motion vector from the two candidate vector groups of the first candidate vector group and the second candidate vector group described above.
Here, the motion vector having the value most suitable for the motion of the detected block is the motion vector having the value closest to the motion vector to be finally obtained. However, since the correct motion vector is not known at the time of initial displacement vector selection, it is evaluated by using some index whether it is the closest value. There are various methods for determining this index. In the present invention, an evaluation method using the block matching method is described. In the block matching method, the difference between the pixel brightness of the detection block of the previous frame and the current frame pixel translated by the estimated motion vector is obtained for each pixel, and the sum of the difference is the smallest value. The motion vector has the most appropriate value.

  More specifically, the initial displacement vector selection unit 5 of this embodiment is configured as shown in FIG. 5, for example. The initial displacement vector selection unit 5 shown in FIG. 5 includes a coordinate conversion unit 5a, a subtraction unit 5b, an absolute value accumulation unit 5c, and a selection unit 5d.

  In the initial displacement vector selection unit 5, the coordinate conversion unit 5a, the subtraction unit 5b, and the absolute value accumulation unit 5c are for each candidate vector (for all candidate vectors of the first candidate vector group and the second candidate vector group). As a result, the selection unit 5d receives calculation results for all candidate vectors output from the absolute value accumulation unit 5c. In the selection unit 5d, for example, the candidate vector having the smallest output value from the absolute value accumulation unit 5c is determined as the most preferable candidate vector, and the candidate vector is selected as an initial displacement vector of the detected block. Output.

Finally, the motion vector calculation unit 6 shown in FIG. 1 is a calculation circuit that obtains a true motion vector by gradient method calculation using the initial displacement vector selected by the initial displacement vector selection unit 5 as a starting point. Note that the motion vector calculation method by the gradient method calculation is detailed in each of the above-mentioned patent documents and non-patent documents, and is not particularly described here, but iteratively how the initial displacement vector is used in the present invention. The gradient method will be described as an example.
Here, the motion vector calculation method in the motion vector calculation unit 6 employs an iterative gradient method that uses one or more gradient method operations, but is not limited to this. The calculation method may be used.

  For example, in the gradient method calculation, the motion displacement V1 estimated from the coordinate position obtained by displacing the previous field signal with the initial displacement vector V0 (α, β) as the starting point is represented by the following equation (2). ) And (3).

  In equations (2) and (3), Vx is the x-direction component of the difference between the motion vectors V0 and V1, and Vy is the y-direction component of the difference between the motion vectors V0 and V1. Here, Σ represents that a sum is obtained by calculating all coordinates in a block area of m pixels × n lines, for example, 8 pixels × 8 lines. ΔX is the gradient in the x direction of the image luminance at the coordinate of interest (difference value from the adjacent pixel in the x direction), and ΔY is the gradient in the y direction of the image luminance at the coordinate of interest (the difference value from the adjacent pixel in the y direction). ) And DFD (x, y) indicate inter-field difference values between the coordinates (x, y) of the previous field and the coordinates (x + α, y + β) of the current field. Further, sign (ΔX) and sign (ΔY) are codes indicating the direction of the gradient represented by any one of +1, −1, and 0, respectively.

For example, in the case of the two-time iterative gradient method, as shown in FIG. 6, the initial displacement vector is set as V0, the first displacement V1 and the second displacement V2 are obtained, and the motion vector V obtained by adding them to the vector is obtained. Is obtained by the following equation (4).
V = V0 + V1 + V2 (4)
FIG. 6 is a vector diagram for explaining a method of calculating the motion vector V by two iteration gradient methods.

According to Expression (4), as shown in FIG. 7, the image existing in the block of coordinates (m1, n1) in the previous field is moved to the block of the coordinate position of coordinates (m1 + α0, n1 + β0) in the current field. At this time, the amount of motion is obtained as a vector V. FIG. 7 is a schematic diagram for specifically explaining the motion vector V of the image moved between the previous field and the current field one field before.
In this way, the motion vector V obtained by the motion vector calculation unit 6 shown in FIG. 1 is accumulated in the vector memory 3 and used for calculating the motion vector for the next field and subsequent candidate vectors for selecting an initial displacement vector. Used as

  Next, the motion vector detection method for moving image signals according to the present embodiment in the above configuration will be further described together with the operation of the motion vector detection device 10 described above. In the operation state of the motion vector detection device 10, the current field signal and the previous field signal one field before input via the field delay unit 1 are the full-screen motion feature detection unit 2, the initial displacement vector selection unit 5, This is sequentially supplied to the motion vector calculation unit 6.

  First, the full screen motion feature detection unit 2 reads the initial displacement vector for full screen motion calculation from the memory 2b in which the already detected full screen motion feature vector is stored, and performs the motion vector calculation of the current field signal. A motion feature vector indicating the features of the entire screen is calculated. For the calculation at this time, the iterative gradient method as described above can also be used. Further, the full screen motion feature vector of the previous field is read from the memory 2b from the full screen motion feature vector calculated by the full screen motion feature detection unit 2, and is subtracted to obtain the motion feature vector of the full screen. The amount of change is calculated. The change amount of the motion characteristic vector of the full screen is input to the initial displacement vector correction unit 4 as change amount data of the output of the full screen motion feature detection unit 2.

  The initial displacement vector correction unit 4 sequentially reads out motion vectors of 8 blocks around the block corresponding to the detected block from the vector memory 3 as a candidate vector group (that is, a first candidate vector group), and further performs full-screen motion feature detection. The change amount data output from the unit 2 is subtracted from each motion vector of the first candidate vector group, and the vector subtraction result is set as the second candidate vector group.

  In the initial displacement vector selection unit 5, motion vectors of the eight blocks around the block corresponding to the detected block sequentially read from the vector memory 3, that is, the first candidate vector group, and the initial displacement vector correction unit 4 output The second candidate vector group is input, and each motion vector of each candidate vector group is supplied to the respective coordinate conversion unit 5a, and the target block of the previous field signal supplied from the field delay unit 1 is used as the motion. It is displaced by a vector, coordinate conversion to the current field is performed, and the coordinate conversion result is supplied to each subtraction unit 5b.

  Each subtraction unit 5b performs subtraction between the previous field signal coordinate-converted by the coordinate conversion unit 5a and the input current field signal to calculate a difference for each pixel, and each difference result Is supplied to the absolute value accumulating unit 5c. Each absolute value accumulating unit 5c calculates the absolute value of the difference of each pixel, accumulates the absolute value difference for the number of pixels of the block, and selects the accumulated result as a candidate vector of each block Each is output to the unit 5d.

  Next, the selection unit 5d that has received the accumulated results for each block compares the accumulated results of the respective blocks, detects the block having the smallest accumulated result, and determines the motion vector of the block as the initial displacement vector. And supplied to the motion vector calculation unit 6.

  The motion vector calculation unit 6 calculates a true motion vector of the current field signal from the previous field signal, starting from the supplied initial displacement vector. As described above, by extracting the motion characteristics of the entire screen and applying the initial displacement vector corrected based on the motion characteristics of the entire screen, erroneous detection of the initial displacement vector is prevented, and, for example, it is repeated as few as 2 times. The true motion vector for each block of the current field signal can be correctly calculated by the number of operations by the gradient method.

  Thus, in the motion vector detection method according to the present invention, camera operations such as camera panning, tilt, and zoom are extracted as motion feature vectors of the entire screen, and the extracted motion feature vectors of the entire screen are A more reliable initial displacement vector can be calculated by combining with a motion vector of a moving object other than the background inside.

  In the above description, the amount of delay output after being delayed by the field delay unit 1 is one field, and a motion vector representing the magnitude and direction of motion is calculated between the previous signal and the vector. The case where the data is stored in the memory 3 has been described. However, the present invention is not limited to such a case. The present invention is not limited to such a case, and may be configured to obtain a motion vector between two or more fields apart or from a plurality of fields. good. In this case, the field delay unit 1 delays several desired plural fields and outputs respective delayed field signals. For example, when it is desired to detect a motion vector using 1 field delay and 2 field delay, the field delay unit 1 outputs a 1 field delay signal and a 2 field delay signal. These are input to the full-screen motion feature detection unit 2, the initial vector selection unit 5, and the motion vector calculation unit 6, and are used for motion vector detection using a plurality of fields.

  Further, as a method for detecting the motion feature of the full screen in the full screen motion feature detection unit 2, in the above-described embodiment, a block or a region where a background is supposed to be displayed on the screen as shown in FIG. The case of using and detecting was explained as an example. However, according to the current image processing technology, in addition to such a method, for example, a feature point on the screen is extracted from a moving image, and the movement of the entire screen caused by panning, tilting, zooming, etc. of the camera is performed. It is not impossible to detect features.

  Here, with respect to the feature points on the screen, immovable characteristic objects are used. For example, when shooting a baseball game, the base position, the line position, tennis, soccer, football, etc. In court lines and volleyball, there are antennas and nets. In other words, if it is known in advance what kind of moving image the captured moving image is, the panning and tilting of the camera can be performed by using the predetermined feature points on the screen as described above. It is possible to guess the camera operation results such as zoom. As described above, a method of detecting the movement feature of the entire screen using the positional change of the extracted feature point may be used as the full-screen movement feature detection unit in the present invention.

It is a functional block block diagram which shows an example of a function structure of the motion vector detection apparatus which concerns on this invention. It is a schematic diagram which shows an example of the full screen feature extraction block referred when detecting the motion vector of a background as a motion feature of a full screen in the motion vector detection apparatus which concerns on this invention. It is a block block diagram which shows an example of the specific circuit structure of the full-screen motion feature detection part of the motion vector detection apparatus which concerns on this invention. It is a block block diagram which shows an example of the specific circuit structure of the initial displacement vector correction | amendment part of the motion vector detection apparatus which concerns on this invention. It is a block block diagram which shows an example of the specific circuit structure of the initial displacement vector selection part of the motion vector detection apparatus which concerns on this invention. It is a vector diagram for demonstrating the calculation method of the motion vector by two iteration gradient methods. It is a schematic diagram for demonstrating concretely the motion vector of the image moved between the front field and the present field.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Field delay part, 2 ... Full screen motion feature detection part, 2a ... Motion vector calculation part, 2b ... Full screen motion feature vector memory, 2c ... Vector difference part, 3 ... Vector memory, 4 ... Initial displacement vector correction part, 4a ... Vector subtraction unit, 5 ... Initial displacement vector selection unit, 5a ... Coordinate conversion unit, 5b ... Subtraction unit, 5c ... Absolute value accumulation unit, 5d ... Selection unit, 6 ... Motion vector calculation unit, 10 ... Motion vector detection device .

Claims (8)

  A motion image signal field is divided into a plurality of predetermined blocks of a predetermined size, and a motion representing the magnitude and direction of motion between the motion image signals separated by at least one field for each of the divided blocks. In a motion vector detection method of a moving image signal for detecting a vector, an entire screen motion feature detection step for detecting a motion feature of the entire screen and calculating a change amount, and a motion vector detected for each block for at least one field. A storage step for storing, and a first candidate vector group for selecting an initial displacement vector of the detected block from the motion vectors stored in the storing step, the motion vectors of a plurality of blocks located in the vicinity of the detected block As a full screen motion feature detection step for each of the first candidate vector group. A candidate vector correction step for calculating a second candidate vector group by performing correction based on the amount of change indicating a change in the full-screen motion feature detected by the step, and reading from the motion vectors accumulated by the storage step From the first candidate vector group and the second candidate vector group corrected by the candidate vector correction step, a motion vector having a value most suitable for the motion of the detected block is determined as an initial displacement vector of the detected block. An initial displacement vector selection step to be selected as the initial displacement vector, and the initial displacement vector selected in the initial displacement vector selection step is used as a starting point to perform one or more calculations to obtain and output the motion vector of the detected block. The calculated motion vector is used to calculate the motion vector of the moving image signal from the next field. Motion vector detecting method, characterized in that it has a motion vector calculating step, the accumulating by the storage step.   The motion vector detection method according to claim 1, wherein the full screen motion feature detection step displays motion features of blocks at positions shifted from the four corners on the screen by a predetermined number of blocks toward the center. It is detected as an overall motion feature, or a motion feature of an area larger than the block at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center, The motion characteristics of a plurality of block groups that are detected as features or that are shifted to the center by a predetermined number of blocks from the four corners on the screen and / or from the top, bottom, left, and right sides of the screen Or a motion vector detection method using any one of them.   The motion vector detection method according to claim 1, wherein the full-screen motion feature detection step uses a positional change of a predetermined feature point on the image included in the motion image signal as a motion of the entire screen. A motion vector detection method characterized by detecting as a feature.   A motion image signal field is divided into a plurality of predetermined blocks of a predetermined size, and a motion representing the magnitude and direction of motion between the motion image signals separated by at least one field for each of the divided blocks. In a motion vector detection device for a moving image signal for detecting a vector, a motion feature detection unit for detecting motion features of the entire screen and calculating a change amount, and at least one field of motion vectors detected for each block. A storage unit for storing, and a first candidate vector group for selecting an initial displacement vector of a detected block from motion vectors stored in the storage unit, the motion vectors of a plurality of blocks located in the vicinity of the detected block And the full-screen motion feature detection means further detects each of the first candidate vector group. Candidate vector correction means for calculating a second candidate vector group by performing correction based on the change amount indicating a change in the full-screen motion characteristics, and the first read out from the motion vectors stored in the storage means. An initial value for selecting a motion vector having a value most suitable for the motion of the detected block as an initial displacement vector of the detected block from the candidate vector group of the first candidate vector and the second candidate vector group corrected by the candidate vector correcting means A displacement vector selecting means, and performing the calculation one or more times from the initial displacement vector selected by the initial displacement vector selecting means to obtain and output a motion vector of the detected block, and the obtained motion vector , The moving image accumulated by the storing step for calculating the motion vector of the moving image signal after the next field. Motion vector detecting apparatus characterized by comprising a vector operation means.   5. The motion vector detecting apparatus according to claim 4, wherein the full screen motion feature detecting means displays motion features of blocks at positions shifted from the four corners on the screen by a predetermined number of blocks toward the center. It is detected as an overall motion feature, or a motion feature of an area larger than the block at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center, The motion characteristics of a plurality of block groups that are detected as features or that are shifted to the center by a predetermined number of blocks from the four corners on the screen and / or from the top, bottom, left, and right sides of the screen A motion vector detection apparatus using any one of the following.   5. The motion vector detecting device according to claim 4, wherein the full screen motion feature detecting means detects a positional change of a predetermined feature point on the image included in the motion image signal as a motion of the entire screen. A motion vector detection device characterized by detecting as a feature.   A motion vector detection program characterized by realizing the motion vector detection method according to claim 1 as a program executable by a computer.   A program recording medium, wherein the motion vector detection program according to claim 7 is recorded on a computer-readable recording medium.

Images