JP2006215655A - Method, apparatus, program and program storage medium for detecting motion vector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a motion vector of a subject with a minimized operation frequency by correcting an initial displacement vector based on overall motion characteristics of an extracted screen. <P>SOLUTION: Motion vectors of a plurality of blocks located in the vicinity of a detection object block in a current field are read from a vector memory 4 as a first candidate vector group, and a screen overall motion characteristic detection part 3 calculates a change quantity of overall motion characteristic of the screen from at least one field before based on a camera parameter such as the direction of a camera in photographing. An initial displacement vector correction part 5 corrects the first candidate vector group with the change quantity to determine a second candidate vector group. An initial displacement vector selection part 6 selects a motion vector with a value most suitable to the motion of the detection object block of the current field as an initial displacement vector, and a motion vector arithmetic part 7 determines a motion vector of the detecting object block with the initial displacement vector as a start point through one or more operations. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a motion vector detection method, a motion vector detection device, a motion vector detection program, and a program recording medium for moving image signals, and more particularly, for example, a television system conversion device that performs video signal conversion between different television systems. The present invention also relates to a motion vector detection method, a motion vector detection device, a motion vector detection program, and a program recording medium that can be suitably applied to a high-efficiency encoding device for image transmission.

  A technique for detecting a motion region in a moving image signal as a motion vector and performing motion correction using the motion vector, for example, when improving the inter-frame coding efficiency in high-efficiency coding at the time of image transmission, or It is particularly effective for reducing motion discontinuity due to field number conversion during television system conversion. For example, as a television system conversion device, Japanese Patent Application Laid-Open No. 01-309597 disclosed in Japanese Patent Application Laid-Open No. 01-309597 “Television Standard System Conversion Device” and Japanese Patent Application Laid-Open No. 03-280681 disclosed in Japanese Patent Application Laid-Open No. 03-280681 disclosed a motion using motion vectors. "Interpolation method", Journal of the Television Society shown in Non-Patent Document 1 (Vol. 45, No. 12, pages 1534 to 1543, published in 1991), or the National Convention of Television Society shown in Non-Patent Document 2 The technology described in the proceedings of the book (20-5, held in 1989) is proposed.

  Conventionally, as a motion vector detection method used in the television system conversion device described in each of the above-mentioned documents, for example, Japanese Patent Application Laid-Open No. 55-162683 shown in Patent Document 3 “Television Image Motion Detection Method” ”And Japanese Patent Laid-Open No. 55-162684 shown in“ Patent Document 4 ”,“ Detective Method for Asymptotic Detection of Image Motion Vectors ”, or Japanese Patent Laid-Open No. 60-158786 shown in Patent Document 5. Detection methods using the iterative gradient method described in “Image Motion Amount Detection Method” and Japanese Patent Application Laid-Open No. 62-206980 “Initial Deviation Method in Motion Estimation of Moving Image” shown in Patent Document 6 have been proposed. .

  In particular, the latter motion vector detection method based on the iterative gradient method is smaller and more accurate than the pattern matching method, and can detect a motion vector. That is, the motion vector detection method based on the iterative gradient method has a predetermined predetermined size of m × n pixels including, for example, m pixels in the horizontal direction and n lines in the vertical direction, for each field of the digitized television signal. By subdividing into blocks, each block is subjected to repetitive gradient calculation based on the signal gradient in the screen and the physical correspondence of the signal difference value between the corresponding screens The amount of motion was estimated.

  In this case, in Japanese Patent Application Laid-Open No. 62-206980 of Patent Document 6, detection is already made in a plurality of peripheral blocks including a block corresponding to the detected block as an initial value when estimating the motion amount. By selecting an optimal motion vector detection target motion vector detection vector from among motion vector candidates as an initial displacement vector, and starting calculation from a value close to the true motion vector of the detected block For example, a technique has been proposed in which the number of computations of the gradient method computation is reduced and, for example, a true motion vector is detected by two gradient computations.

Japanese Patent Laid-Open No. 06-217266 “Motion Vector Detection Circuit” shown in Patent Document 7 discloses that at least between one block of image signals separated by at least one field or one frame in order to further improve the accuracy of motion vector detection. A method for detecting an initial displacement vector of motion has also been proposed.
Japanese Laid-Open Patent Publication No. 01-309597 Japanese Patent Laid-Open No. 03-280681 Japanese Unexamined Patent Publication No. 55-162683 Japanese Patent Laid-Open No. 55-162684 JP 60-158786 A Japanese Patent Laid-Open No. 62-206980 Japanese Patent Laid-Open No. 06-217266 The Journal of the Institute of Television Engineers of Japan (Vol. 45, No. 12, pages 1534 to 1543, published in 1991) Proceedings of the annual conference of the Institute of Television Engineers of Japan (20-5, 1989)

  However, in the above-described conventional technique, when selecting an initial displacement vector from among a plurality of candidate vectors, an image is not included in the image under a situation where the entire image moves in a biased manner by a camera operation at the time of shooting. When there is an object that moves differently from the whole, the accuracy of motion vector detection may be reduced.

  For example, consider a case where the camera is panned while photographing a moving object. At this time, the background other than the moving object is moving at the same moving speed in the same direction by the panning operation, while the moving object is moving at a small speed near the center of the shooting screen. become. When the camera is panned in this way, the initial displacement vector of the block in which the moving object is present is selected from the block in which the object is present one field before in the conventional technique.

  However, since the moving speed of the object in the screen is actually a combination of the moving speed of panning and the moving speed of the object, if the moving speed of panning changes greatly, the moving speed of one field before In order to use the motion from the block in which the object exists as it is as the initial displacement vector, there is a problem that an error becomes large.

  In particular, when shooting a ball game or the like, since shooting is performed so as to catch the ball in the screen, when the ball moves greatly, panning is performed so that the ball is positioned substantially at the center of the screen. At this time, the moving speed of the ball before panning and the moving speed after panning change greatly on the screen, and the ball can be selected from any position before one field when selecting the initial displacement vector. An error occurs in the initial displacement vector.

  For example, if the ball movement direction is the same as the panning direction and the ball movement speed is lower than the panning movement speed, the ball on the screen moves in the opposite direction immediately after panning. It will have a vector, and the position of the initial displacement vector of the ball will be wrong even if it is selected from any position before one field.

Here, the accuracy of the initial displacement vector of the ball that is the subject greatly affects the detection of the motion vector of the ball, so if there is a change in the panning movement speed, an erroneous detection of the motion vector will result. Therefore, when the television system conversion or the like is performed, the ball is displayed at an incorrect position.
In other words, the above-described conventional technique has a drawback in that the motion vector of the moving subject is erroneously detected because the initial displacement vector is not calculated in consideration of the change in the motion of the entire image due to the operation of the camera. .

  The present invention has been made to solve the above-described problems, and extracts motion characteristics of the entire screen using at least camera parameters including information associated with camera operations such as camera orientation and zoom. By correcting the initial displacement vector based on the motion characteristics of the entire screen, we provide a technology that prevents erroneous detection of the initial displacement vector and that can correctly detect the motion vector for each block including a moving object with a small number of calculations. The purpose is to do.

  The first technical means divides a field of a moving image signal into a plurality of blocks having a predetermined size, and each of the divided blocks has a motion between at least one field apart. In a motion vector detection method of a motion image signal for detecting a motion vector representing a magnitude and direction, the camera parameter is selected from the motion image signal to which camera parameters including information on the camera orientation and zoom at the time of shooting are added. A camera parameter acquisition step to acquire, a full screen motion feature detection step of detecting a motion feature of the entire screen based on the camera parameter acquired by the camera parameter acquisition step, and calculating a change amount; A storage step of accumulating the detected motion vector for at least one field; and the storage step The motion vectors of a plurality of blocks located in the vicinity of the detected block are read out from the motion vectors accumulated by the above as a first candidate vector group for initial displacement vector selection of the detected block, and the first candidate Candidate vector correction step of calculating a second candidate vector group by performing correction based on the change amount indicating a change in the full screen motion feature detected by the full screen motion feature detection step for each of the vector groups. And a motion of a detected block from the first candidate vector group read out from the motion vectors accumulated in the storage step and the second candidate vector group corrected in the candidate vector correction step. The initial displacement that selects the motion vector with the most suitable value as the initial displacement vector of the detected block And calculating and outputting a motion vector of the detected block by performing one to a plurality of operations starting from the initial displacement vector selected in the initial displacement vector selection step. A motion vector calculation step for storing the motion vector of the moving image signal after the next field, and storing the motion vector in the storing step.

  A second technical means is the motion vector detection method according to the first technical means, wherein the full-screen motion feature detection step further includes a predetermined number of blocks from the four corners on the screen. The motion characteristics of a block at a position shifted to the position of the block are detected, or the movement of an area larger than the block at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center. A feature is detected, or a motion feature of a plurality of block groups located at positions shifted from the four corners on the screen and / or from the top, bottom, left, and right sides of the screen by a predetermined number of blocks. Or a motion feature detected by the full screen motion detection step, and a motion feature of the entire screen detected based on the camera parameters. , Using, and calculates the amount of change which indicates a change in the full screen motion features.

  A third technical means is the motion vector detection method according to the first technical means, wherein the full-screen motion feature detection step is further performed on a predetermined image included in the video signal. A feature point motion detection step for detecting a positional change of the feature point as a feature point motion, and the feature point motion detected by the feature point motion detection step and the motion of the entire screen detected based on the camera parameter And calculating the amount of change indicating a change in the full screen motion feature.

  The fourth technical means divides the field of the moving image signal into a plurality of blocks having a predetermined size, and moves each moving block between moving image signals separated by at least one field. In a motion vector signal motion vector detecting apparatus for detecting a motion vector representing a magnitude and direction, the camera parameter is selected from among the motion image signals to which camera parameters including information on the camera orientation and zoom at the time of shooting are added. Camera parameter acquisition means to acquire, full-screen motion feature detection means to detect the motion characteristics of the entire screen based on the camera parameters acquired by the camera parameter acquisition means, and calculate the amount of change, for each block Storage means for storing the detected motion vector for at least one field, and motion stored by the storage means The motion vectors of a plurality of blocks located in the vicinity of the detected block are read out from the vectors as a first candidate vector group for selecting an initial displacement vector of the detected block, and each of the first candidate vector groups is read out. Further, candidate vector correcting means for calculating a second candidate vector group by performing correction based on the amount of change indicating a change in the full screen motion feature detected by the full screen motion feature detecting means, and the storage means Of the first candidate vector group read out from the motion vectors accumulated by the above and the second candidate vector group corrected by the candidate vector correcting means, the value most suitable for the motion of the detected block is An initial displacement vector selecting means for selecting a motion vector as an initial displacement vector of the detected block; and the initial displacement vector The motion vector of the detected block is obtained by outputting one or more times from the initial displacement vector selected by the selection means to obtain and output the motion vector. And a motion vector calculating means for accumulating by the storage means for vector calculation.

  A fifth technical means is the motion vector detecting device according to the fourth technical means, wherein the full-screen motion feature detecting means is further moved closer to the center by a predetermined number of blocks from four corners on the screen. The motion characteristics of a block at a position shifted to the position of the block are detected, or the movement of an area larger than the block at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center. A feature is detected, or a motion feature of a plurality of block groups located at positions shifted from the four corners on the screen and / or from the top, bottom, left, and right sides of the screen by a predetermined number of blocks. Using a motion feature detected by the full-screen motion detection means, and a motion feature of the entire screen detected based on the camera parameters. And calculates the amount of change which indicates a change in the full screen motion features.

  A sixth technical means is the motion vector detecting device according to the fourth technical means, wherein the full-screen motion feature detecting means is further provided on a predetermined image included in the moving image signal. A feature point motion detecting unit that detects a positional change of the feature point as a feature point motion, and the feature point motion detected by the feature point motion detecting unit and the motion of the entire screen detected based on the camera parameters; And calculating the amount of change indicating a change in the full screen motion feature.

  A seventh technical means is characterized in that the motion vector detection method according to any one of the first to third technical means is a motion vector detection program realized as a program executable by a computer. .

  The eighth technical means is a program recording medium in which the motion vector detection program described in the seventh technical means is recorded on a computer-readable recording medium.

According to the motion vector detection method, the motion vector detection device, the motion vector detection program, and the program recording medium according to the present invention composed of the technical means as described above, the following effects can be obtained.
When the initial displacement vector is selected from the detected motion vectors and the motion vector is detected using the initial displacement vector, the amount of change in the motion of the entire screen by the camera operation is detected with respect to the detected motion vector. By using an added motion vector, that is, a prediction vector, it is possible to select a more correct initial displacement vector without being affected by changes caused by camera operations such as panning, tilt, and zoom.

  As a result, it is possible to reduce false detection of the initial displacement vector at the time of camera panning, tilting, zoom start point and camera operation speed change, and to greatly improve motion vector detection accuracy. Become.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments for implementing a motion vector detection method, a motion vector detection device, a motion vector detection program, and a program recording medium according to the present invention will be described with reference to the drawings. In the following description, an example of an embodiment related to a motion vector detection device according to the present invention will be described, and description of an embodiment related to a motion vector detection method, a motion vector detection program, and a program recording medium according to the present invention will be omitted. From the following detailed description of the motion vector detection device, a motion vector detection method applicable to the motion vector detection device, a vector detection program for executing the motion vector detection method as a program by a computer, and the vector detection program It is also possible to easily understand a program recording medium for recording the program on a computer-readable recording medium.

First, an embodiment of a motion vector detection apparatus according to the present invention for detecting a motion vector of a moving image signal will be described in detail with reference to FIG. FIG. 1 is a functional block configuration diagram illustrating an example of a functional configuration of a motion vector detection device according to the present invention, to which an example of a motion vector detection method according to the present invention is applied.
1 includes a field delay unit 1, a camera parameter acquisition unit 2, a full screen motion feature detection unit 3, a vector memory 4, an initial displacement vector correction unit 5, and an initial displacement vector selection. A unit 6 and a motion vector calculation unit 7 are included.

  The motion vector detection apparatus 10 according to the present embodiment divides a moving image signal input for each field into a plurality of blocks having a predetermined size, for example, m pixels × n lines (m and n are integers). Thus, for each divided block, a motion for obtaining a motion vector representing the direction and magnitude of motion with the corresponding block in the moving image signal, for example, one field before delayed by the field delay unit 1 A first candidate vector group selected from motion vectors already detected and stored in the vector memory 4; and a second candidate vector corrected by the initial displacement vector correction unit 5; Together with the group, the motion vector selected by the initial displacement vector selection unit 6 is selected as the initial displacement vector of the detected block, and the initial displacement vector As a starting point, with a small number of calculations for example, twice the gradient computing method, and a vector operation unit 7 for determining the true motion vector of 該被 detection block correctly.

  In particular, the present embodiment is characterized in how to obtain a candidate vector of an initial displacement vector, and a motion vector due to camera movement such as panning, tilt, and zoom is obtained from a moving image signal acquired by the camera parameter acquisition unit 2. Change amount data that is detected by the full screen motion feature detection unit 3 based on the camera parameters, and further detects the motion features of the full screen based on at least the camera parameters to indicate changes in the motion features of the full screen. The motion vector candidate read from the vector memory 4 is corrected based on the change amount data by the initial displacement vector correction unit 5 and output as a candidate vector used for selection of the initial displacement vector. Thus, the present embodiment is characterized in that a more accurate initial displacement vector is selected.

  More specifically, the motion vector detection device 10 according to the present embodiment shown in FIG. 1 is, as described above, the full-screen motion feature detection unit 3, the vector memory 4, the initial displacement vector correction unit 5, the initial displacement. A vector selection unit 6 and a motion vector calculation unit 7 are included. The initial displacement vector selection unit 6 and the motion vector calculation unit 7 are respectively supplied with the current field signal and the previous field signal delayed by one field via the field delay unit 1. Further, as shown in FIG. 8, the same field signal may be supplied to the full-screen motion feature detection unit 3 in some cases. FIG. 8 is a functional block configuration diagram showing another example of the functional configuration of the motion vector detection apparatus according to the present invention, which is different from FIG. An example in which the unit 3 is also supplied is shown.

  In FIG. 1, the full screen motion feature detection unit 3 calculates the motion features of the entire screen based on the camera parameters acquired by the camera parameter acquisition unit 2, and calculates change data of the motion of the entire screen from the changes. . At this time, for example, as shown in FIG. 8, the previous field signal is divided into blocks of m pixels × n lines as described above, and the characteristics of the motion of the entire screen from among the divided blocks. Even if the motion feature quantity of the entire screen calculated based on the camera parameters is corrected with a motion vector detected by using a plurality of predetermined specific blocks to detect the current block and comparing it with the current field signal good. The initial displacement vector selection unit 6 selects an initial displacement vector for each divided block using the current field signal and the previous field signal. The motion vector calculation unit 7 detects a motion vector for each block using the initial field vector as a starting point and the current field signal and the previous field signal.

  The vector memory 4 is a storage circuit including a RAM (Random Access Memory) that accumulates motion vectors for at least one field already detected for each block, and its input terminal serves as an output terminal of the motion vector calculation unit 7. The motion vector detected by the motion vector calculation unit 7 is sequentially updated and accumulated at an address corresponding to the position of each block divided into, for example, 8 pixels × 8 lines. Has been.

  Note that the full-screen motion feature detection unit 3 detects a motion vector in a moving image signal by comparing a partial region or a plurality of blocks in the previous field with an image in the current field using the configuration shown in FIG. In such a case, there is a method of detecting using a part of the screen as shown in FIG.

  FIG. 2 is a schematic diagram showing an example of a full screen feature extraction block that is referred to when the background motion vector is detected as a full screen motion feature in the motion vector detection circuit 10 according to the present invention. This is an example of the position and size of the feature extraction block on the screen for detecting. The detection of motion characteristics of the entire image can be generally realized by performing motion vector calculation on a part of blocks or areas of the screen. Although not particularly illustrated, at this time, motion vector data stored in the vector memory 4 may be used for motion vector calculation.

  Further, the full-screen feature extraction unit 3 refers to a full-screen feature extraction block that is referred to when the motion vector indicating the presence / absence of motion in the background (direction and size when present) is detected as the motion feature of the full screen. It is preferable to use a block at a position where the background is surely reflected in the screen, as shown in FIG. 2 (A), at a position closer to the center by a predetermined number of blocks from the four corners of the screen, It is more preferable to use four blocks a, b, c and d located in the vicinity of the four corners of the screen. This is because the subject to be photographed often appears around the center of the screen, and on the other hand, in the case of the periphery of the screen, there is a high possibility that a feature image will be mixed from the outside of the screen. Because.

  Alternatively, as shown in FIG. 2 (B), the four blocks a, b, c, and 4 in FIG. A plurality of block groups a ′, b ′, c ′, d ′ having an area larger than d may be used, and as shown in FIG. 2C, the four blocks a in FIG. , B, c, d, including a plurality of neighboring block groups (the number of blocks is arbitrary) and / or a plurality of block groups located closer to the center by a predetermined number of blocks from the top, bottom, left, and right sides of the screen. Thus, a change such as zoom may be detected.

  Further, the full screen motion feature detection unit 3 not only detects only one motion feature vector in the entire screen, but also considers that there is a difference in the motion feature vector depending on the zoom or the like. A motion feature vector may be detected for each feature extraction block. However, when performing correction, it is necessary to determine in advance which motion feature vector is used and which initial displacement vector of a block at which position is to be corrected. For example, the motion feature vector of each block from a to d in FIG. 2A is obtained by dividing the screen into four screen areas a ″, b ″, c ″, d ″ as shown in FIG. 2D. It is necessary to determine in advance that it will be used to correct the initial displacement vector in each case.

  A more specific configuration example of the full-screen motion feature detection unit 3 is shown in FIG. FIG. 3 is a block configuration diagram showing an example of a specific circuit configuration of the full-screen motion feature detection unit 3 of the motion vector detection circuit 10 according to the present invention. As shown in FIG. 3, the full-screen motion feature detection unit 3 uses a camera parameter related to the camera orientation and zoom acquired by the camera parameter acquisition unit 2 to theoretically calculate a motion vector that is a motion feature of the full screen. A theoretical motion vector calculation unit 3a for calculating, and a vector memory 3b including a RAM or the like for storing a full-screen motion feature vector for one field detected for each full-screen motion feature extraction block. The vector difference unit 3c calculates a vector difference between the screen motion feature vector and the full screen motion feature vector of the previous field stored in the vector memory 3b, and the vector difference is calculated as the full screen motion feature. It is configured to output as change amount data indicating the change of.

  The full-screen motion feature detection unit 3 shown in FIG. 8 has a block configuration as shown in FIG. 9. In the full-screen motion feature detection unit 3, the screen motion vector calculation unit 3d A full-screen motion feature vector is detected for each full-screen motion feature extraction block as illustrated, and using this detection result, a theoretical motion vector output from the theoretical motion vector calculation unit 3a is detected by the correction processing unit 3c. Further, an example is given in which correction is performed so that the full-screen motion feature vector detected for each full-screen motion feature extraction block is calculated.

  In the correction processing unit 3e shown in FIG. 9, contrary to the above example, first, the full screen motion feature vector detected for each full screen motion feature extraction block is detected by the screen motion vector calculation unit 3d. The full screen motion feature vector may be corrected by the full screen motion feature vector based on the camera parameter output from the theoretical motion vector calculation unit 3a. In any case, by using information on both the full-screen motion feature vector detected for each full-screen motion feature extraction block and the motion feature of the entire screen detected using the camera parameters, either The initial displacement vector can be obtained more flexibly as compared with the case of calculating the change amount data indicating the change in the motion characteristics of the entire screen using only one piece of information.

  Of course, as described above, the full-screen motion feature detection unit 3 of the motion vector detection device 10 according to the present invention only calculates change amount data indicating a change in the motion feature of the entire screen using the above-described information. However, the present invention is not limited to this, and the change amount data may be calculated using only one of the information.

Further, the initial displacement vector correction unit 5 shown in FIG. 1 corrects the first candidate vector group stored in the vector memory 4 using the motion characteristics of the entire screen, and uses the first candidate vector group for selection of the initial displacement vector. A candidate vector correcting means for outputting as a second candidate vector group is configured. In other words, the first candidate vector group is corrected based on the change amount data from the full screen motion feature detection unit 3 (that is, data indicating the change in the full screen motion feature), and the second candidate vector group is obtained. Ask. FIG. 4 is a block configuration diagram showing an example of a specific circuit configuration of the initial displacement vector correction unit 5 of the motion vector detection device 10 according to the present invention. The vector subtraction unit 5a uses the first candidate. In the example, the second candidate vector group is obtained by vector subtracting the change amount data of the full screen motion feature detection unit 3 from the vector group.
Here, in the present embodiment, as the first candidate vector group, the motion vectors in the previous field detected in the eight blocks around the detected block are set as the candidate vector group for selecting the initial displacement vector of the detected block. And Note that the first candidate vector group is not limited to such an example, and may be determined from already detected motion vectors in other regions.

Specifically, the second candidate vector group is calculated in the initial displacement vector correction unit 5 by performing the following calculation. Assuming that the first candidate vector group stored in the vector memory 4 is Va and the motion feature vector that is the variation data from the full-screen motion feature detection unit 3 is Vs, the second candidate vector group Vb is the following: (1).
Vb = Va−Vs (1)

Next, the initial displacement vector selection unit 6 shown in FIG. 1 selects a motion vector having a value most suitable for the motion of the detected block from the detected motion vectors obtained by the motion vector calculation of the previous field, as a starting point of the gradient method calculation. The selection circuit selects as an initial displacement vector, and selects an appropriate motion vector from the two candidate vector groups of the first candidate vector group and the second candidate vector group described above.
Here, the motion vector having the value most suitable for the motion of the detected block is the motion vector having the value closest to the motion vector to be finally obtained. However, since the correct motion vector is not known at the time of initial displacement vector selection, it is evaluated by using some index whether it is the closest value. There are various methods for determining this index. In the present invention, an evaluation method using the block matching method is described. In the block matching method, the difference between the pixel brightness of the detection block of the previous frame and the current frame pixel translated by the estimated motion vector is obtained for each pixel, and the sum of the difference is the smallest value. The motion vector has the most appropriate value.

  More specifically, the initial displacement vector selection unit 6 of the present embodiment is configured as shown in FIG. 5, for example. The initial displacement vector selection unit 6 shown in FIG. 5 includes a coordinate conversion unit 6a, a subtraction unit 6b, an absolute value accumulation unit 6c, and a selection unit 6d.

  In the initial displacement vector selection unit 6, the coordinate conversion unit 6a, the subtraction unit 6b, and the absolute value accumulation unit 6c are configured for each candidate vector (all candidate vectors of the first candidate vector group and the second candidate vector group, ie, motions). As a result, the selection unit 6d receives calculation results for all candidate vectors output from the absolute value accumulation unit 6c. In the selection unit 6d, for example, the candidate vector having the smallest output value from the absolute value accumulation unit 6c is determined as the most preferable candidate vector, and the candidate vector is selected as the initial displacement vector of the detected block. Output.

Finally, the motion vector calculation unit 7 shown in FIG. 1 is a calculation circuit that obtains a true motion vector by gradient method calculation using the initial displacement vector selected by the initial displacement vector selection unit 6 as a starting point. Note that the motion vector calculation method by the gradient method calculation is detailed in each of the above-mentioned patent documents and non-patent documents, and is not particularly described here, but iteratively how the initial displacement vector is used in the present invention. The gradient method will be described as an example.
Here, the motion vector calculation method in the motion vector calculation unit 7 employs an iterative gradient method using one or more gradient method operations, but is not limited to this, and is not limited to this. The calculation method may be used.

  For example, in the gradient method calculation, the motion displacement V1 estimated from the coordinate position obtained by displacing the previous field signal with the initial displacement vector V0 (α, β) as the starting point is represented by the following equation (2). ) And (3).

  In equations (2) and (3), Vx is the x-direction component of the difference between the motion vectors V0 and V1, and Vy is the y-direction component of the difference between the motion vectors V0 and V1. Here, Σ represents that a sum is obtained by calculating all coordinates in a block area of m pixels × n lines, for example, 8 pixels × 8 lines. ΔX is the gradient in the x direction of the image luminance at the coordinate of interest (difference value from the adjacent pixel in the x direction), and ΔY is the gradient in the y direction of the image luminance at the coordinate of interest (the difference value from the adjacent pixel in the y direction). ) And DFD (x, y) indicate inter-field difference values between the coordinates (x, y) of the previous field and the coordinates (x + α, y + β) of the current field. Further, sign (ΔX) and sign (ΔY) are codes indicating the direction of the gradient represented by any one of +1, −1, and 0, respectively.

For example, in the case of the two-time iterative gradient method, as shown in FIG. 6, the initial displacement vector is set as V0, the first displacement V1 and the second displacement V2 are obtained, and the motion vector V obtained by adding them to the vector is obtained. Is obtained by the following equation (4).
V = V0 + V1 + V2 (4)
FIG. 6 is a vector diagram for explaining a method of calculating the motion vector V by two iteration gradient methods.

According to Expression (4), as shown in FIG. 7, the image existing in the block of coordinates (m1, n1) in the previous field is moved to the block of the coordinate position of coordinates (m1 + α0, n1 + β0) in the current field. At this time, the amount of motion is obtained as a vector V. FIG. 7 is a schematic diagram for specifically explaining the motion vector V of the image moved between the previous field and the current field one field before.
In this way, the motion vector V obtained by the motion vector calculation unit 7 shown in FIG. 1 is stored in the vector memory 4 and is used as a candidate vector for selecting an initial displacement vector to be used for motion vector calculation for the next field and thereafter. Used as

  Next, the motion vector detection method for moving image signals according to the present embodiment in the above configuration will be further described together with the operation of the motion vector detection device 10 described above. In the operation state of the motion vector detection device 10, the current field signal and the previous field signal one field before input via the field delay unit 1 are sequentially transmitted to the initial displacement vector selection unit 6 and the motion vector calculation unit 7. Supplied. Similar field signals may also be supplied to the full-screen motion feature detection unit 3. At the same time, a moving image signal to which camera parameters at the time of current field shooting are added is input to the camera parameter acquisition unit 2. The camera parameter acquisition unit 2 acquires the current camera parameters included in the input moving image signal and outputs them to the theoretical motion vector calculation unit 3a of the full-screen motion feature detection unit 3.

  Here, the full screen motion feature detection unit 3 uses the camera parameters obtained from the camera parameter acquisition unit 2 in the theoretical motion vector calculation unit 3a to calculate the camera motion and the screen motion feature by zooming, The motion feature vector of the screen is calculated. The calculation for calculating the motion feature vector of the entire screen using the camera parameters is, for example, a kind of projective transformation described in Japanese Patent Application Laid-Open No. 11-41500 “Motion Vector Detection Device and Method and Recording Medium”. A method performed using a certain affine transformation may be used.

  If the moving image signal is a digital video or the like, the additional data can be transmitted simultaneously with the moving image data as a video stream. In the standards such as MPEG7 and MPEG21, the additional data In addition, camera parameters including information such as camera orientation and zoom at the time of shooting can be set.

  The motion feature vector calculated by the theoretical motion vector calculation unit 3a is stored in the vector memory 3b and used for calculation processing of the next field. Next, in the vector difference unit 3c, the motion feature vector of the previous field is extracted from the vector memory 3b, and the difference from the motion feature vector of the current field calculated by the theoretical motion vector calculation unit 3a is calculated as the motion of the entire screen. It is calculated as change amount data indicating the change of the feature and is output to the initial displacement vector correction unit 5.

  The initial displacement vector correction unit 5 sequentially reads out motion vectors of 8 blocks around the block corresponding to the detected block from the vector memory 4 as a candidate vector group (that is, a first candidate vector group), and further performs full-screen motion feature detection. The change amount data output from the unit 3 and the motion vectors of the first candidate vector group are vector-added, and the vector addition result is set as a second candidate vector group.

  In the initial displacement vector selection unit 6, motion vectors of the eight blocks around the block corresponding to the detected block sequentially read from the vector memory 4, that is, the first candidate vector group, and the initial displacement vector correction unit 5 output The second candidate vector group is input, and each motion vector of each candidate vector group is supplied to the respective coordinate conversion unit 6a, and the target block of the previous field signal supplied from the field delay unit 1 is used as the motion. The coordinates are converted to the current field by being displaced by a vector, and the luminance information of the block corresponding to the coordinates of the coordinate conversion result and the target block of the previous field is supplied to each subtracting section 6b.

  Each subtraction unit 6b performs subtraction between the luminance information of the previous field obtained by the coordinate conversion unit 6a and the luminance information of the block corresponding to the coordinates of the input conversion result of the current field, The difference for each pixel is calculated, and each difference result is supplied to the absolute value accumulation unit 6c. Each absolute value accumulating unit 6c calculates the absolute value of the difference between each pixel, accumulates the absolute value difference for the number of pixels in the block, and selects the accumulated result as a candidate vector for each block. Each is output to the unit 6d.

  Next, the selection unit 6d that has received the accumulated results for each block compares the accumulated results of the respective blocks, for example, detects a block having the smallest accumulated result, and initially sets the motion vector of the block. The displacement vector is selected and supplied to the motion vector calculator 7.

  The motion vector calculator 7 calculates the true motion vector of the current field signal from the previous field signal, starting from the supplied initial displacement vector. As described above, by extracting the motion characteristics of the entire screen and applying the initial displacement vector corrected based on the motion characteristics of the entire screen, erroneous detection of the initial displacement vector is prevented, and, for example, it is repeated as few as 2 times. The true motion vector for each block of the current field signal can be correctly calculated by the number of operations by the gradient method.

  Thus, in the motion vector detection method according to the present invention, camera parameters indicating camera operations such as panning, tilt, and zoom of the camera are acquired by the camera parameter acquisition unit 2, and the full-screen motion feature detection unit 3 Based on at least the camera parameters, a motion feature vector of the entire screen is obtained by calculation, and the obtained motion feature vector of the entire screen is determined by the initial displacement vector correction unit 5 as a motion vector of a moving object other than the background in the screen. By combining, a more accurate initial displacement vector can be calculated.

  In the above description, the amount of delay output after being delayed by the field delay unit 1 is one field, and a motion vector representing the magnitude and direction of motion is calculated between the previous signal and the vector. Although the case where the data is stored in the memory 4 has been described, the present invention is not limited to such a case. The present invention may be configured to obtain a motion vector between two or more fields apart from each other or a plurality of fields. good. In this case, the field delay unit 1 delays several desired plural fields and outputs respective delayed field signals. For example, when it is desired to detect a motion vector using 1 field delay and 2 field delay, the field delay unit 1 outputs a 1 field delay signal and a 2 field delay signal. These are input to the full-screen motion feature detection unit 3, the initial vector selection unit 6, and the motion vector calculation unit 7, and are used for motion vector detection using a plurality of fields.

  Further, in the above-described embodiment of FIG. 8, the full-screen motion feature detection unit 3 first calculates the full-screen motion feature based on the camera parameters as described above, and the full-screen motion feature is shown in FIG. The case where correction is performed using the motion feature detected using the full-screen motion feature extraction block as shown in the above example has been described, but conversely, the background is displayed on the screen as shown in FIG. Even if it is configured to correct the motion feature of the full screen detected using the block or region that seems to be (full screen motion feature extraction block) using the motion feature calculated based on the camera parameters Alternatively, the motion characteristics of the entire screen may be calculated by performing appropriate calculations using information on the motion characteristics of both at the same time.

  Further, the full screen motion feature detection unit 3 detects a motion feature used in combination with the full screen motion feature calculated based on the camera parameters by using a full screen motion feature extraction block as shown in FIG. It is not limited to movement features. According to the current image processing technique, in addition to such a method, for example, feature points on a screen may be extracted from a moving image to detect motion characteristics of the entire screen.

  Here, with respect to the feature points on the screen, immovable characteristic objects are used. For example, when shooting a baseball game, the base position, the line position, tennis, soccer, football, etc. In court lines and volleyball, there are antennas and nets. In other words, if it is known in advance what kind of moving image the captured moving image is, the motion characteristics of the entire screen can be obtained by using the predetermined feature points on the screen as described above. Can be estimated. As described above, the method of detecting the motion feature of the entire screen using the estimated positional change of the feature point may be used as the full-screen motion feature detection unit in the present invention.

  Alternatively, it is configured to use only one information of a motion feature detected using a full screen motion feature extraction block or a feature point on the screen, or a motion feature of a full screen calculated based on camera parameters. May be. For example, the full-screen motion feature detection unit 3 uses the camera parameter acquisition unit 2 without detecting a full-screen motion feature using a specific full-screen feature extraction block or a feature point on the screen as shown in FIG. Using only the acquired camera parameters, calculate the motion feature vector of the entire screen in the current field, and use this calculation result to calculate the amount of change from the motion feature vector of the entire screen in the already detected field. Of course, it may be configured to output to the initial displacement vector correction unit 5.

It is a functional block block diagram which shows an example of a function structure of the motion vector detection apparatus which concerns on this invention. It is a schematic diagram which shows an example of the full screen feature extraction block referred when detecting the vector which a background moves as a motion feature of a full screen in the motion vector detection circuit which concerns on this invention. It is a block block diagram which shows an example of the specific circuit structure of the full-screen motion feature detection part of the motion vector detection circuit which concerns on this invention. It is a block block diagram which shows an example of the specific circuit structure of the initial displacement vector correction | amendment part of the motion vector detection apparatus which concerns on this invention. It is a block block diagram which shows an example of the specific circuit structure of the initial displacement vector selection part of the motion vector detection apparatus which concerns on this invention. It is a vector diagram for demonstrating the calculation method of the motion vector V by the iterative gradient method twice. It is a schematic diagram for demonstrating concretely the motion vector V of the image moved between the front field of 1 field before, and the present field. It is a functional block block diagram which shows another example of a function structure of the motion vector detection apparatus which concerns on this invention. It is a block block diagram which shows another example of the specific circuit structure of the full screen motion characteristic detection part of the motion vector detection circuit which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Field delay part, 2 ... Camera parameter acquisition part, 3 ... Full screen motion characteristic detection part, 3a ... Theoretical motion vector calculation part, 3b ... Vector memory, 3c ... Vector difference part, 3d ... Screen motion vector calculation part, 3e ... Correction processing unit, 4 ... Vector memory, 5 ... Initial displacement vector correction unit, 5a ... Vector subtraction unit, 6 ... Initial displacement vector selection unit, 6a ... Coordinate conversion unit, 6b ... Subtraction unit, 6c ... Absolute value accumulation unit, 6d ... selection unit, 7 ... motion vector calculation unit, 10 ... motion vector detection device.

Claims (8)

  A motion image signal field is divided into a plurality of predetermined blocks of a predetermined size, and a motion representing the magnitude and direction of motion between the motion image signals separated by at least one field for each of the divided blocks. In a motion vector detection method of a moving image signal for detecting a vector, a camera parameter acquisition step for acquiring the camera parameter from a moving image signal to which camera parameters including information on a camera direction and zoom at the time of shooting are added; , Based on the camera parameters acquired in the camera parameter acquisition step, a motion characteristic detection step for detecting a motion characteristic of the entire screen and calculating a change amount; and at least a motion vector detected for each block A storage step for storing one field and the storage step The motion vectors of a plurality of blocks located in the vicinity of the detected block are read out as a first candidate vector group for initial displacement vector selection of the detected block from the first vector, and each of the first candidate vector groups A candidate vector correction step for calculating a second candidate vector group by performing correction based on the change amount indicating a change in the full screen motion feature detected by the full screen motion feature detection step, and the storage A value most suitable for the motion of the detected block from among the first candidate vector group read from the motion vectors accumulated in the step and the second candidate vector group corrected in the candidate vector correction step. The initial displacement vector selection step for selecting the motion vector of the target as the initial displacement vector of the detected block And performing one or more operations starting from the initial displacement vector selected in the initial displacement vector selection step to obtain and output a motion vector of the detected block, And a motion vector calculation step of storing the motion vector in the storage step for calculating a motion vector of the moving image signal.   The motion vector detection method according to claim 1, wherein the full-screen motion feature detection step further includes motion features of blocks at positions shifted from the four corners on the screen toward the center by a predetermined number of blocks. Or the motion feature of a region larger than the block at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center, or on the screen Full-screen motion consisting of detecting motion characteristics of a plurality of block groups located at positions deviated toward the center by a predetermined number of blocks from the four corners of the screen and / or from the top, bottom, left, and right sides of the screen. A full screen motion feature using a motion feature detected by the full screen motion detection step and a motion feature of the entire screen detected based on the camera parameters. The motion vector detection method and calculates the amount of change which indicates a change.   The motion vector detection method according to claim 1, wherein the full-screen motion feature detection step further includes a positional change of a feature point on a predetermined image included in the moving image signal. A feature point motion detection step for detecting the motion as a motion, and the feature point motion detected by the feature point motion detection step and the motion feature of the entire screen detected based on the camera parameters. A motion vector detection method characterized by calculating the amount of change indicating a change in feature.   A motion image signal field is divided into a plurality of predetermined blocks of a predetermined size, and a motion representing the magnitude and direction of motion between the motion image signals separated by at least one field for each of the divided blocks. In a motion vector detection device for a moving image signal for detecting a vector, camera parameter acquisition means for acquiring the camera parameter from a moving image signal to which camera parameters including information on the camera orientation and zoom at the time of shooting are added , Based on the camera parameters acquired by the camera parameter acquisition means, detecting the motion characteristics of the entire screen and calculating the amount of change, and at least a motion vector detected for each block Storage means for storing one field, and motion vectors stored by the storage means The motion vectors of a plurality of blocks located in the vicinity of the detection block are read out as a first candidate vector group for selecting an initial displacement vector of the detected block, and all of the first candidate vector groups are further Candidate vector correction means for calculating a second candidate vector group by performing correction based on the amount of change indicating a change in the whole screen movement feature detected by the screen movement feature detection means; and motion accumulated by the storage means From among the first candidate vector group read out from the vector and the second candidate vector group corrected by the candidate vector correcting means, a motion vector having a value most suitable for the motion of the detected block is detected. Initial displacement vector selection means for selecting as an initial displacement vector of the block, and selection by the initial displacement vector selection means In addition, the motion vector of the detected block is calculated and output one or more times from the initial displacement vector as a starting point, and the calculated motion vector is used to calculate the motion vector of the moving image signal after the next field. A motion vector calculating means for accumulating by the storage means.   5. The motion vector detection apparatus according to claim 4, wherein the full-screen motion feature detection means further includes a motion feature of a block at a position shifted toward the center by a predetermined number of blocks from four corners on the screen. Or the motion feature of a region larger than the block at each position shifted from the four corners on the screen by a predetermined number of blocks toward the center, or on the screen Full-screen motion consisting of detecting motion characteristics of a plurality of block groups located at positions deviated toward the center by a predetermined number of blocks from the four corners of the screen and / or from the top, bottom, left, and right sides of the screen. A detection unit that indicates a change in the full screen motion feature using the motion feature detected by the full screen motion detection unit and the motion feature of the entire screen detected based on the camera parameter; Motion vector detection unit and calculates the serial variation.   5. The motion vector detection apparatus according to claim 4, wherein the full-screen motion feature detection means further includes a positional change of a feature point on a predetermined image included in the moving image signal. A feature point motion detection unit that detects the motion, and the feature point motion detected by the feature point motion detection unit and the motion feature of the entire screen detected based on the camera parameter, A motion vector detection device that calculates the amount of change indicating a change in feature.   A motion vector detection program characterized by realizing the motion vector detection method according to claim 1 as a program executable by a computer.   A program recording medium, wherein the motion vector detection program according to claim 7 is recorded on a computer-readable recording medium.

Images