JP4027398B1 - Motion vector detection device - Google Patents

Abstract

A motion vector detection apparatus capable of detecting a motion vector between frame images quickly and accurately.
In a motion vector detection device for detecting a motion vector between frame images related to a previous frame and a current frame (back frame), first, vertical edges are obtained for a plurality of image regions obtained by dividing the frame image in the vertical direction. After emphasis, projection is performed in the vertical direction to generate projection data for one horizontal line. Next, an intersection between the projection data and the first threshold is searched, and projection data for the current frame corresponding to the motion vector detection range centered on each intersection is extracted. Then, each extracted projection data is added for each distance from each intersection described above, and a motion vector in the horizontal direction of the frame image is detected from the addition result. As described above, the motion vector can be detected with high accuracy and speed.
[Selection] Figure 3

Description

  The present invention relates to a motion vector detection apparatus that detects a motion vector between frame images related to a previous frame and a subsequent frame that are in a time series relationship.

  As a motion vector detection technique between frame images used for camera shake correction or the like, there is a technique of detecting a motion vector from continuously captured images using a representative point matching method. In this representative point matching method, a representative point is set at a fixed position of the image of the previous frame in continuously captured images, and the correlation calculation of the representative point is performed while shifting the image of the current frame in the two-dimensional direction. Thus, the amount of deviation with the highest correlation is detected as a motion vector.

  However, since the motion vector detection technique described above requires that the representative point has a certain or higher luminance gradient, for example, when a monochrome image such as a document image is targeted, the luminance at all the representative points. The gradient becomes low, and the amount of motion (motion vector) cannot be detected accurately.

  As a technique for improving this problem, there is one disclosed in Patent Document 1, for example. In this technique, first, feature points of an image having a luminance gradient are extracted, and representative point matching processing is performed using the extracted feature points as representative points, thereby enabling favorable detection of motion vectors.

Japanese Patent No. 3534551

  However, although the technique of Patent Document 1 can reliably set a representative point where a luminance gradient exists, the correlation of the representative point is still performed while shifting the pixels little by little over the entire search range in the frame image. A considerable amount of calculation is required for each (feature point). Therefore, unless the number of representative points is kept below a certain level, it is difficult to quickly detect a motion vector, and real-time processing becomes difficult. On the other hand, if there are too few representative points, the accuracy of motion vector detection will be reduced.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a motion vector detection device capable of detecting a motion vector between frame images with high accuracy and speed.

  A motion vector detection device according to a first aspect of the present invention is a motion vector detection device that detects a motion vector between frame images related to a previous frame and a rear frame that are in a time-series relationship. ) With respect to a frame image read out by repeating pixel scanning of a horizontal line in order in the vertical direction, edge enhancement means for enhancing a vertical edge for a predetermined image area in the frame image, and (b) edge by the edge enhancement means Projection means for projecting the image in which the image is emphasized in the vertical direction and generating projection data having a data array for one horizontal line, and (c) projection data obtained by the projection means for the previous frame A waveform obtained by graphing array element values in the order of the elements of the data array intersects with a straight line where the array element values are a predetermined constant value. (D) projection data obtained by the projection means with respect to the subsequent frame, the specifying means for specifying the position of the intersection array element; and (d) a predetermined range of data array centered on the position of the array element at each intersection. And (e) an adding means for adding the values of array elements having the same relative position with respect to the position of the array element at each intersection for each data array in a predetermined range extracted by the extracting means And (f) detecting means for detecting a motion vector in the horizontal direction of the frame image based on the addition result added by the adding means.

  A motion vector detection device according to a second aspect of the present invention is a motion vector detection device that detects a motion vector between frame images related to a previous frame and a subsequent frame that are in a time-series relationship. ) With respect to a frame image read out by repeating pixel scanning of a horizontal line in order in the vertical direction, edge enhancement means for enhancing a horizontal edge for a predetermined image area in the frame image, and (b) edge by the edge enhancement means A projection means for projecting the image in which the image is emphasized in the horizontal direction and generating projection data having a data array for one vertical line; and (c) the projection data obtained by the projection means for the previous frame A waveform obtained by graphing array element values in the order of the elements of the data array intersects with a straight line where the array element values are a predetermined constant value. (D) projection data obtained by the projection means with respect to the subsequent frame, the specifying means for specifying the position of the intersection array element; and (d) a predetermined range of data array centered on the position of the array element at each intersection. And (e) an adding means for adding the values of array elements having the same relative position with respect to the position of the array element at each intersection for each data array in a predetermined range extracted by the extracting means And (f) detecting means for detecting a motion vector in the vertical direction of the frame image based on the addition result added by the adding means.

  According to the motion vector detection device of the present invention, a predetermined image region in a frame image is emphasized in the vertical direction, and the edge-enhanced image is projected in the vertical direction to have a data array for one horizontal line. An array element at each intersection where a waveform that generates projection data and the waveform of the array element values in the order of the data array in relation to the projection data of the previous frame intersects a straight line where the array element value is a predetermined constant value Specify the position of. Then, a data array of a predetermined range centered on the position of the array element at each intersection is extracted from the projection data of the subsequent frame, and the extracted data array of the predetermined range is relative to the position of the array element at each intersection. Values of array elements having the same position are added, and a motion vector in the horizontal direction of the frame image is detected based on the addition result. As a result, motion vectors between frame images can be detected with high accuracy and speed.

  Further, according to the motion vector detection device of the present invention, a data array for one vertical line is obtained by emphasizing a horizontal edge for a predetermined image region in a frame image and projecting the edge enhanced image in the horizontal direction. At each intersection where a waveform in which the values of the array elements are graphed in the order of the elements of the data array with respect to the projection data of the previous frame and a straight line where the values of the array elements are a predetermined constant value intersect. Specify the position of the array element. Then, a data array of a predetermined range centered on the position of the array element at each intersection is extracted from the projection data of the subsequent frame, and the extracted data array of the predetermined range is relative to the position of the array element at each intersection. Values of array elements having the same position are added, and a motion vector in the vertical direction of the frame image is detected based on the addition result. As a result, motion vectors between frame images can be detected with high accuracy and speed.

<Embodiment 1>
<Configuration of motion vector detection device>
FIG. 1 is a block diagram showing a main configuration of a motion vector detection device 1 according to Embodiment 1 of the present invention.

  The motion vector detection device 1 detects, for example, an image motion vector that represents the motion of a subject in a screen in a moving image, and the motion between frame images related to a previous frame and a subsequent frame that are in a chronological order. A vector is detected. The motion vector detected by the motion vector detection device 1 is used for camera shake correction of the video camera. Note that the frame image input to the motion vector detection device 1 is read out by performing horizontal pixel scanning and vertical line scanning as shown in FIG.

  The motion vector detection device 1 includes an input terminal 11, a horizontal direction motion vector detection unit 2 and a vertical direction motion vector detection unit 4, and two output terminals 12 and 13. The functions of the horizontal motion vector detection unit 2 and the vertical motion vector detection unit 4 may be realized in software by, for example, a microprocessor, and most of the functions are realized in hardware (partly software). It may be. Of course, it goes without saying that all functions may be realized by hardware.

  For example, image data such as image (moving image) data or a video signal acquired by an image sensor in a video camera is input to the input terminal 11. In the image data input to the input terminal 11, a horizontal motion vector is detected by the horizontal motion vector detection unit 2, and the detected horizontal motion vector is output from the output terminal 12.

  On the other hand, the vertical motion vector detector 4 detects a vertical motion vector for the image data input to the input terminal 11. The vertical motion vector detected by the vertical motion vector detection unit 4 is output from the output terminal 13.

  Below, the structure of the horizontal direction motion vector detection part 2 and the vertical direction motion vector detection part 4 is demonstrated in order.

  FIG. 3 is a block diagram showing the main configuration of the horizontal motion vector detection unit 2.

  The horizontal motion vector detection unit 2 includes an input terminal 20, a vertical image segmentation unit 21, a vertical edge extraction filtering unit 22, a vertical block projection unit 23, and a first vertical block projection data maximum value storage. A unit 24, a bit number reduction unit 25, and a first vertical block projection line memory 26. The horizontal motion vector detection unit 2 includes a second vertical block projection line memory 27, a second vertical block projection data maximum value storage unit 28, a first threshold intersection search unit 29, and a vertical direction. A block projection data reading unit 30, a vertical block horizontal motion vector calculating unit 31, a horizontal motion vector determining unit 32, and an output terminal 33 are provided. The function of each part of the horizontal motion vector detection unit 2 described above will be briefly described (specific operations will be described in detail later).

  The vertical image dividing unit 21 divides the frame image input to the input terminal 20 in the vertical direction and outputs a block divided in the vertical direction (hereinafter also referred to as “vertical block”).

  The vertical edge extraction filtering unit 22 performs a filtering process for performing edge extraction for each block divided by the vertical image dividing unit 21.

  The vertical block projection unit 23 projects the edge-enhanced vertical direction block output from the vertical edge extraction filtering unit 22 in the vertical direction and outputs projection data for each vertical block.

  The first vertical block projection data maximum value storage unit 24 stores the maximum absolute value in the vertical block projection data of the current frame output from the vertical block projection unit 23. The first vertical block projection data maximum value storage unit 24 is described later based on the maximum value of the vertical block projection data (hereinafter also referred to as “first vertical block projection data”) of the current frame. A second threshold value is calculated.

  Based on the maximum value of the first vertical block projection data stored in the first vertical block projection data maximum value storage unit 24, the bit number reduction unit 25 uses the vertical direction output from the vertical block projection unit 23. Reduce the number of bits in the projection data of the block.

  The first vertical block projection line memory 26 stores the first vertical block projection data whose bit number has been reduced by the bit number reduction unit 25.

  The second vertical block projection line memory 27 converts the vertical block projection data sent from the first vertical block projection line memory 26 into the projection data of the vertical block of the previous frame (hereinafter referred to as “second vertical block projection data”). It is also saved as “direction block projection data”.

  The second vertical block projection data maximum value storage unit 28 uses the maximum value of the projection data of the vertical block output from the first vertical block projection data maximum value storage unit 24 as the second vertical block projection data. Save as the maximum value of direction block projection data. Further, the second vertical block projection data maximum value storage unit 28 calculates a first threshold, which will be described later, based on the maximum value of the second vertical block projection data.

  The first threshold value intersection search unit 29 calculates the waveform of the second vertical block projection data related to the previous frame stored in the second vertical block projection line memory 27 and the second vertical block projection data maximum value. A point where the first threshold value calculated by the storage unit 28 intersects is obtained, and information on this intersection (hereinafter also referred to as “first threshold value intersection”) is output.

  The vertical block projection data reading unit 30 has a first vertical direction within a predetermined range (motion vector detection range) before and after the first threshold intersection obtained by the first threshold intersection search unit 29. The block projection data is read from the first vertical block projection line memory 26.

  The vertical block horizontal motion vector calculation unit 31 uses the second threshold output from the first vertical block projection data maximum value storage unit 24 and reads the first block read by the vertical block projection data reading unit 30. Each of the vertical block projection data of 1 is converted into n-values (n is an integer of 2 or more) to reduce the number of bits, and the n-valued projection data is added for each distance from the first threshold intersection. .

  The horizontal motion vector determination unit 32 determines the horizontal motion vector of the image based on the output from the vertical block horizontal motion vector calculation unit 31. The horizontal motion vector of the image determined here is output from the output terminal 33.

  FIG. 4 is a block diagram showing a main configuration of the vertical block projection unit 23.

  The vertical block projection unit 23 includes an input terminal 231, an adder 232 that performs data addition for each horizontal line, and a vertical projection temporary storage memory 233 as a buffer memory that sequentially stores data added by the adder 232. And an output terminal 234.

  An image whose vertical edges are emphasized by the vertical edge extraction filtering unit 22 is input to the input terminal 231 for each vertical block. The image input to the input terminal 231 is projected in the vertical direction by the adder 232 and the vertical projection temporary storage memory 233 for each vertical block. Specifically, the adder 232 adds the data for one horizontal line of the vertical block and the data for one horizontal line read from the vertical projection temporary memory 233, and the addition result is the vertical projection temporary. It is returned again to the storage memory 233 and stored. When the addition of all the horizontal lines in the vertical block is completed by repeating such an addition operation, the addition data of all the horizontal lines, that is, the projection data in the vertical direction is output from the output terminal 234.

  FIG. 5 is a block diagram illustrating a main configuration of the vertical block horizontal motion vector calculation unit 31.

  The vertical block horizontal motion vector calculation unit 31 adds three input terminals 311 to 313, an n-value calculator 314, and the first vertical block projection data converted to n-value by the n-value generator 314. An adder 315 to perform, a horizontal motion vector addition memory 316 as a buffer memory for sequentially storing data added by the adder 315, a peak detector 317, and an output terminal 318 are provided.

  The first vertical block projection data output from the vertical block projection data reading unit 30 is input to the input terminal 311, and the first vertical block projection data maximum value storage unit 24 is input to the input terminal 312. The output second threshold value is input. In addition, the first threshold value intersection information output from the first threshold value intersection search unit 29 is input to the input terminal 313.

  The n-value converter 314 converts the first vertical block projection data input from the input terminal 311 into n-values (for example, ternarization) based on the second threshold value input from the input terminal 312.

  The adder 315 adds the first vertical block projection data n-valued by the n-value calculator 314 and the data read from the horizontal motion vector addition memory 316 around the first threshold intersection. Then, the addition result is stored in the horizontal motion vector addition memory 316 again. When the addition for all the first threshold intersections related to the vertical block is completed by repeating such an addition operation, the addition data related to the vertical block is output to the peak detector 317.

  The peak detector 317 detects a peak position (horizontal motion vector described later) in the added data output from the adder 315. The peak position detected by the peak detector 317 is input to the horizontal direction motion vector determination unit 32 via the output terminal 318.

  Next, the configuration of the vertical motion vector detection unit 4 will be described.

  FIG. 6 is a block diagram showing a main configuration of the vertical motion vector detection unit 4.

  The vertical motion vector detection unit 4 includes an input terminal 40, a horizontal image segmentation unit 41, a horizontal edge extraction filtering unit 42, a horizontal block projection unit 43, and a first horizontal block projection line memory 44. The first horizontal block projection data maximum value storage unit 45 is provided. The vertical motion vector detection unit 4 includes a second horizontal block projection line memory 46, a second horizontal block projection data maximum value storage unit 47, a third threshold intersection search unit 48, and a horizontal direction. A block projection data reading unit 49, a horizontal block vertical motion vector calculation unit 50, a vertical motion vector determination unit 51, and an output terminal 52 are provided. The function of each part of the above-described vertical motion vector detection unit 4 will be briefly described (specific operations will be described in detail later).

  The horizontal image dividing unit 41 divides the frame image input to the input terminal 40 in the horizontal direction and outputs a block divided in the horizontal direction (hereinafter also referred to as “horizontal block”).

  The horizontal direction edge extraction filtering unit 42 performs a filtering process for performing edge extraction for each block divided by the horizontal direction image dividing unit 41.

  The horizontal block projection unit 43 projects the edge-enhanced horizontal block output from the horizontal edge extraction filtering unit 42 in the horizontal direction and outputs projection data for each horizontal block.

  The first horizontal block projection line memory 44 converts the horizontal block projection data output from the horizontal block projection unit 43 into the horizontal block projection data of the current frame (hereinafter referred to as “first horizontal block projection data”). As “)”.

  The first horizontal block projection data maximum value storage unit 45 stores the maximum value of the horizontal block projection data of the current frame output from the horizontal block projection unit 43.

  The second horizontal block projection line memory 46 converts the horizontal block projection data sent from the first horizontal block projection line memory 44 into projection data (hereinafter referred to as “second horizontal block projection data” of the horizontal block of the previous frame). It is also saved as “direction block projection data”.

  The second horizontal block projection data maximum value storage unit 47 uses the maximum value of the horizontal block projection data output from the first horizontal block projection data maximum value storage unit 45 as the second horizontal block projection data maximum value for the previous frame. Save as the maximum value of direction block projection data. The second horizontal block projection data maximum value storage unit 47 calculates a third threshold value and a fourth threshold value, which will be described later, based on the maximum value of the second horizontal block projection data regarding the previous frame.

  The third threshold intersection search unit 48 includes a second horizontal block projection data related to the previous frame stored in the second horizontal block projection line memory 46, and a second horizontal block projection data maximum value storage unit. The point where the third threshold value calculated at 47 intersects is obtained, and information on this intersection (hereinafter also referred to as “third threshold value intersection”) is output.

  The horizontal block projection data reading unit 49 has a first horizontal direction within a predetermined range (motion vector detection range) before and after the third threshold intersection obtained by the third threshold intersection search unit 48. Block projection data is read from the first horizontal block projection line memory 44.

  The horizontal block vertical motion vector calculation unit 50 uses the fourth threshold value output from the second horizontal block projection data maximum value storage unit 47, and is read by the horizontal block projection data read unit 49. Each horizontal block projection data of 1 is converted into n-values (n is an integer of 2 or more) to reduce the number of bits, and the n-valued projection data is added for each distance from the third threshold intersection. .

  The vertical motion vector determination unit 51 determines the vertical motion vector of the image based on the output from the horizontal block vertical motion vector calculation unit 50. The vertical motion vector of the image determined here is output from the output terminal 52.

  FIG. 7 is a block diagram showing a main configuration of the horizontal block projection unit 43.

  The horizontal block projection unit 43 includes an input terminal 431, an adder 432 for adding one horizontal line data of the horizontal block, and a horizontal projection temporary as a buffer memory for sequentially storing the data added by the adder 432. A storage memory 433 and an output terminal 434 are provided.

  The input terminal 431 receives an image in which horizontal edges are emphasized by the horizontal edge extraction filtering unit 42. The image input to the input terminal 431 is projected in the horizontal direction by the adder 432 and the horizontal projection temporary storage memory 433 for each horizontal block. Specifically, with respect to the pixels of one horizontal line in the horizontal block, the addition result that is added by the adder 432 and stored in the horizontal projection temporary storage memory 433 up to the previous pixel and the pixel input from the input terminal 431. Are added by the adder 432, and the addition result is returned to the horizontal projection temporary storage memory 433 and stored. When the addition of one horizontal line in the horizontal block is completed by repeating such an addition operation, the addition data of all the pixels in one horizontal line of the horizontal block, that is, the projection data in the horizontal direction is output from the output terminal 434. Will be. Thereafter, similarly, the projection data in the horizontal direction of the next horizontal block is processed, and when the processing of all the horizontal blocks for one horizontal line is completed, the processing of the next one horizontal line is performed.

  FIG. 8 is a block diagram showing a main configuration of the horizontal block vertical motion vector calculation unit 50.

  The horizontal block vertical motion vector calculation unit 50 adds three input terminals 501 to 503, an n-value converter 504, and the first horizontal block projection data converted to n-value by the n-value generator 504. An adder 505 to be performed, a vertical motion vector addition memory 506 as a buffer memory for sequentially storing data added by the adder 505, a peak detector 507, and an output terminal 508 are provided.

  The first horizontal block projection data output from the horizontal block projection data reading unit 49 is input to the input terminal 501, and the second horizontal block projection data maximum value storage unit 47 is input to the input terminal 502. The output fourth threshold value is input. In addition, the information about the third threshold value intersection output from the third threshold value intersection search unit 48 is input to the input terminal 503.

  The n-value converter 504 converts the first horizontal block projection data input from the input terminal 501 into n-values (for example, ternarization) based on the fourth threshold value input from the input terminal 502.

  The adder 505 adds the first horizontal block projection data converted to the n-value by the n-value generator 504 and the data read from the vertical motion vector addition memory 506 around the third threshold value intersection. Then, the addition result is stored in the vertical motion vector addition memory 506 again. When the addition for all the third threshold intersections regarding the horizontal block is completed by repeating such an addition operation, the addition data regarding the horizontal block is output to the peak detector 507.

  The peak detector 507 detects a peak position (vertical motion vector described later) in the added data output from the adder 505. The peak position detected by the peak detector 507 is input to the vertical direction motion vector determination unit 51 via the output terminal 508.

  The operation of the motion vector detection apparatus 1 having the above configuration will be described below.

<Operation of Motion Vector Detection Device 1>
First, the operation of the horizontal motion vector detection unit 2 that detects the horizontal motion vector of an image in the motion vector detection device 1 will be described.

  As shown in FIG. 2, when a frame image read out by sequentially repeating pixel scanning of the horizontal line in the vertical direction is input to the input terminal 20 of the horizontal direction motion vector detecting unit 2 shown in FIG. 21 is divided into blocks in the vertical direction. That is, the vertical image dividing unit 21 sets a plurality of image regions (predetermined image regions) obtained by dividing the frame image in the vertical direction. Thereby, in the subsequent processing, processing and management are performed for each vertical block.

  In the vertical image division unit 21, for example, as shown in FIG. 9A, image data of 640 pixels × 480 pixels is converted into seven vertical blocks vb0 to vb0 having a width of 64 pixels (division width) in the vertical direction. Divided into vb6. Although the image data vb7 at the lowest stage in the vertical direction is not used in the detection of the horizontal motion vector, the seven vertical blocks vb0 to vb6 occupying most of the frame image are used for the detection of the horizontal motion vector. Therefore, it is considered that there is no particular problem in terms of detection accuracy. Here, the division width and the number of divisions shown in FIG. 9A are not essential, and the number of divisions may be set to 1, for example.

  The image divided by the vertical image dividing unit 21 into seven vertical blocks vb0 to vb6 as shown in FIG. 9A is extracted, in other words, edge components extending in the vertical direction by the vertical edge extraction filtering unit 22. For example, a filtering process for emphasizing an image portion that changes sharply in the horizontal direction is performed. As a filter used in this filtering processing, a (1, -1,) 2-tap filter that simply takes a difference from a pixel adjacent in the horizontal direction, or (-1, 2, -1) corresponding to a second order differential. It is possible to use a 3-tap filter. Note that it is not essential to use such a filter, and any filter may be used as long as the output value increases in an image portion where the luminance change in the horizontal direction increases.

  The image data in which the vertical edge is emphasized for each vertical block (image region) vb0 to vb6 by the vertical edge extraction filtering unit 22 is input to the vertical block projection unit 23, and the vertical block projection unit 23 Projected vertically. This projection can reduce the noise component in the vertical direction (between lines) and can enhance the edge component in the vertical direction, so that the vertical edge corresponding to the feature point can be emphasized to improve the motion vector detection accuracy. It will be. The operation of the vertical block projection unit 23 will be described with reference to the conceptual diagram of FIG.

  In the vertical block projection unit 23, when the input of the vertical blocks vb0 to vb6 shown in FIG. 9A is completed, that is, the final pixels of the final line in the vertical blocks vb0 to vb6 with edge enhancement are input. At this point, all array elements of the vertical direction block projection data vn0 to vn6 (FIG. 9B) having the data array Mv for one horizontal line are generated. The data array Mv has array elements of the total number of pixels on the horizontal line (for example, 640). However, in a 2-tap filter such as (1, -1), the effective element is the total number of pixels of the horizontal line − In a 3-tap filter such as 1 (for example, 639) and (−1, 2, −1), the effective elements are the total number of pixels in the horizontal line−2 (for example, 638).

  Specifically, as shown in FIG. 4, the image data of the vertical blocks vb0 to vb6 (image data subjected to vertical edge emphasis) input via the input terminal 231 are sequentially added to the adder 232. Is input. In the adder 232, first, the first horizontal line of the input vertical block is written into the vertical projection temporary storage memory 233 without reading the data in the vertical projection temporary storage memory 233. Next, the adder 232 reads the addition result up to the previous line stored in the vertical projection temporary storage memory 233, performs addition with the horizontal one line of the vertical block input from the input terminal 231, and The addition result is written back to the vertical projection temporary storage memory 233. When the final horizontal line in the vertical block is input to the adder 232, the addition result up to the previous line stored in the vertical projection temporary storage memory 233 is read and the vertical block input from the input terminal 231 is read. And the addition result, that is, the data obtained by adding all the horizontal lines of the vertical block is used as projection data of the vertical block, and the first vertical block projection data maximum from the output terminal 234 is obtained. The data is output to the value storage unit 24 and the bit number reduction unit 25. The adder 232 temporarily stores the projection data obtained by adding all the horizontal lines of the vertical block in the vertical projection temporary storage memory 233, and when the first line of the next vertical block is input, the vertical projection is performed. The projection data in the temporary storage memory 233 may be read and output from the output terminal 234.

  The first vertical block projection data maximum value storage unit 24 calculates the maximum value related to the projection data of the vertical block output from the vertical block projection unit 23 and stores it as the vertical block projection data maximum value of the current frame. To do. That is, the first vertical block projection data maximum value storage unit 24 obtains the maximum value in the array element of the data array Mv (FIG. 9B) relating to the projection data of the current frame (subsequent frame), and calculates this maximum value. Remember.

  Further, the first vertical block projection data maximum value storage unit 24 calculates the second threshold value based on the calculated vertical block projection data maximum value of the current frame. For example, if the maximum value regarding the projection data of the vertical block of the current frame is P1max, the second threshold value α2 is calculated by the following equation (1).

α2 = P1max × k1 (1)
Here, k1 is a predetermined coefficient, and 0 <k1 <1.

  Note that the calculation of the second threshold value α2 is not limited to the above equation (1), and it is sufficient that the second threshold value α2 increases as the vertical block projection data maximum value P1max of the current frame increases. Alternatively, a conversion table may be used.

  The bit number reduction unit 25 determines the effective bit range of the projection data based on the maximum value of the vertical block projection data of the current frame input from the first vertical block projection data maximum value storage unit 24. Then, the upper invalid bit and the lower invalid bit set based on the valid bit range are reduced from the vertical block projection data input from the vertical block projection unit 23, and the vertical direction of the current frame configured only by the valid bits The block projection data is output to the first vertical block projection line memory 26.

  The first vertical block projection line memory 26 stores the vertical block projection data of the current frame whose number of bits has been reduced by the bit number reduction unit 25 as the first vertical block projection data for each vertical block. . That is, the first vertical block projection line memory 26 stores the number of bits (each array in the bit number reduction unit 25 based on the maximum value of the projection data obtained by the first vertical block projection data maximum value storage unit 24. Projection data of the current frame (subsequent frame) in which the element data length) is reduced is stored.

  The second vertical block projection line memory 27 stores the vertical block projection data read from the first vertical block projection line memory 26 as second vertical block projection data relating to the previous frame. Similarly, in the second vertical block projection data maximum value storage unit 28, the maximum value of the vertical block projection data output from the first vertical block projection data maximum value storage unit 24 is the second value related to the previous frame. Are stored as the maximum value of vertical block projection data. In other words, the maximum value of the vertical block projection data of the current frame stored in the first vertical block projection data maximum value storage unit 24 is the second vertical block projection data maximum value of the next frame. This is stored in the unit 28 as the maximum value of the vertical block projection data of the previous frame.

  The second vertical block projection data maximum value storage unit 28 sets a first threshold value (predetermined constant value) based on the maximum value of the second vertical block projection data regarding the previous frame. For example, assuming that the maximum value of the vertical block projection data of the previous frame is P2max, the first threshold value α1 is calculated by the following equation (2).

α1 = P2max × k2 (2)
Here, k2 is a predetermined coefficient, and 0 <k2 <1.

  Note that the calculation of the first threshold value α1 is not limited to the above equation (2), and it is sufficient if the first threshold value α1 increases as the vertical block projection data maximum value P2max of the previous frame increases. Alternatively, a conversion table may be used.

  The first threshold intersection search unit 29 outputs the second vertical block projection data read from the second vertical block projection line memory 27 and the second vertical block projection data maximum value storage unit 28. An intersection with the first threshold value (first threshold value intersection) is searched in the horizontal direction. The information on the first threshold intersection obtained by the first threshold intersection searching unit 29 is output to the vertical block projection data reading unit 30 and the vertical block horizontal motion vector calculating unit 31.

  The vertical block projection data read unit 30 outputs the (first) vertical block projection of the current frame corresponding to the motion vector detection range centered on the first threshold intersection output from the first threshold intersection search unit 29. Data is read from the first vertical block projection line memory 26. That is, the first threshold intersection is A (i) (where i = 1, 2,... P, p is the total number of detected first threshold intersections), and the motion vector detection range is the first threshold. Assuming the range from (−V) to (+ V) (where V is a positive integer) centered on the intersection, the projection data read from the first vertical block projection line memory 26 is (A (i ) −V) to (A (i) + V), which is partial data of the first vertical block projection data.

  The first vertical block projection data read from the first vertical block projection line memory 26 by the vertical block projection data reading unit 30 is output to the vertical block horizontal motion vector calculation unit 31.

  The first vertical block projection data output from the vertical block projection data reading unit 30 is input to the n-value converter 314 via the input terminal 311 of the vertical block horizontal motion vector calculation unit 31 shown in FIG. Is done. The n-value converter 314 outputs the first vertical block projection data based on the second threshold value output from the first vertical block projection data maximum value storage unit 24 and input via the input terminal 312. n-valued. In other words, the n-value converter 314 converts the data length of each array element related to the projection data of the current frame extracted by the vertical block projection data reading unit 30 to the data length of each array element, for example, a ternization process. And compress. The processing of the n-value converter 314 will be described with reference to FIG.

  FIG. 10 is a diagram for explaining the n-value conversion processing in the n-value conversion device 314. Note that FIG. 10 shows an example of the ternary processing where n = 3.

  If the first vertical block projection data input to the n-value converter 314 is D, and the second threshold value input via the input terminal 312 is α2 (α2 is a positive integer), the n-value generator 314 Then, when D <(− α2), (−1) is output, when (−α2) ≦ D ≦ α2, 0 is output, and when D> α2, 1 is output. Done. That is, in the ternarization process in the n-valued unit 314, regarding the second threshold value α2 (α2> 0) set based on the maximum value of the array element of the data array Mv in the projection data of the current frame (subsequent frame), 3 when the value of the array element relating to the projection data of the current frame extracted by the vertical block projection data reading unit 30 is smaller than (−α2), (−α2) or more and α2 or less, and 3 Trinization is performed in stages.

  With regard to the n-value conversion processing by the n-value generator 314, the partial waveform of the first vertical block projection data of the current frame output from the vertical block projection data reading unit 30 is compared with the first threshold intersection. Therefore, it is important to ensure bit accuracy that can identify peaks and valleys in the waveform of the first vertical block projection data. It is.

  On the other hand, when the value of the first vertical block projection data output from the vertical block projection data reading unit 30 is adopted without reducing the number of bits in the n-value converter 314, the first There is a possibility that a small amplitude peak or valley corresponding to the vertical block projection data D> α2 or a small amplitude peak corresponding to D <(− α2) or a small amplitude peak corresponding to D <(− α2) may be buried. There is inappropriate. Furthermore, since the small peaks and valleys corresponding to (−α2) ≦ D ≦ α2 are easily affected by noise and have a large number, the peaks and valleys having an appropriate amplitude in which the characteristics of the image appear are buried.

  Therefore, the n-value generator 314 improves the motion vector detection accuracy by performing an appropriate n-value process on the first vertical block projection data.

  The operations of the first threshold intersection searching unit 29, the vertical block projection data reading unit 30, and the vertical block horizontal motion vector calculating unit 31 described above will be specifically described with reference to FIG. Note that the horizontal axes of FIGS. 11A and 11B indicate the positions of the array elements of the projection data array Mv (FIG. 9B). In FIG. 11A, each of the first threshold intersections A (1) to A (8) is indicated by a circle.

  The first threshold value intersection search unit 29 outputs the second vertical block projection data maximum value storage unit 28 in the waveform of the second vertical block projection data W2 for the previous frame as shown in FIG. First threshold value intersections A (1) to A (8) that intersect the first threshold value α1 thus obtained are obtained. That is, the first threshold value intersection search unit 29 relates to the projection data obtained by the vertical block projection unit 23 with respect to the previous frame, and the array element values in the order of the elements in the projection data array Mv (FIG. 9B). The position of the array element at each first threshold value intersection where the waveform W2 in which the value of the array element intersects the straight line where the value of the array element is the first threshold value (predetermined constant value) α1 is specified.

  Next, the vertical block projection data reading unit 30 performs the first vertical detection for a predetermined motion vector detection range centered on each of the first threshold intersections A (1) to A (8) shown in FIG. The vertical block projection data of the current frame is read from the direction block projection line memory 26. That is, the vertical block projection data reading unit 30 outputs from the vertical block projection unit 23 a data array in a predetermined range centered on the position of the array element of the projection data (data array) at each first threshold intersection. Extracted from the projection data of the current frame (subsequent frame) whose bit number has been reduced by the bit number reduction unit 25. For example, for the first threshold value intersection A (7), as shown in FIG. 11B, the first threshold value intersection A (7) is centered on (A (7) -V) to (A (7) + V). The first vertical block projection data W1 corresponding to the range up to (the waveform portion within the rectangular broken line) is read out.

  Then, the first vertical block projection data read by the vertical block projection data reading unit 30 uses the second threshold value α2 output from the first vertical block projection data maximum value storage unit 24. The n-value unit 314 of the vertical block horizontal direction motion vector calculation unit 31 performs n-value processing. For example, for the first threshold intersection A (7), the first threshold is based on the magnitude relationship of the first vertical block projection data W1 with respect to the second threshold α2 and (−α2) shown in FIG. With respect to the range from (A (7) −V) to (A (7) + V) centering on the intersection A (7) (the waveform portion in the rectangular broken line), the first vertical as shown in FIG. The direction block projection data is ternarized. As a result, the vertical block projection data of the current frame is represented by “−1”, “0”, or “1”.

  In the n-value converter 314, n-value conversion processing as shown in FIG. 11C is performed on all the first threshold value intersections output from the first threshold value intersection search unit 29.

  Next, in the adder 315 (FIG. 5), the n-value generator 314 converts the n-value into the n-value centering on the first threshold-value intersection output from the first threshold-value intersection search unit 29 via the input terminal 313. The first vertical block projection data is added to the previous n-valued first vertical block projection data read from the horizontal motion vector addition memory 316, and the addition result Are again stored in the horizontal motion vector addition memory 316. Then, the adder 315 adds the n-valued first vertical block projection data relating to the last one of all the first threshold intersections detected by the first threshold intersection search unit 29 to obtain one When the addition process for the vertical block is terminated, the addition result is output to the peak detector 317. In addition, after the addition process in the adder 315 is completed, the peak detection unit 317 may read the addition result from the horizontal motion vector addition memory 316 that stores the addition results regarding all the first threshold intersections. good.

  The peak detector 317 includes all the first threshold intersections detected by the first threshold intersection search unit 29 in a motion vector detection range (the above-described ± V range) centered on each first threshold intersection. Data obtained by adding the corresponding n-valued first vertical block projection data (hereinafter also referred to as “vertical block n-valued data”) is input. The value is detected by peak detector 317. The peak position of the vertical block n-valued data detected by the peak detector 317 becomes a horizontal motion vector obtained from the vertical block, and is output to the horizontal motion determination unit 32 via the output terminal 318. .

  The operations of the adder 315 and the peak detection unit 317 described above will be specifically described with reference to FIG. In FIG. 12A, in the motion vector detection range (± V range) centering on the first threshold intersections A (1) to A (8) shown in FIG. The block projection data is conceptually shown as three values.

  In the adder 315, when ternary data around the first threshold intersections A (1) to A (8) shown in FIG. 12 (a) are sequentially input, these data are added, and FIG. Vertical block n-valued data as shown in b) is generated. Specifically, in the adder 315, the vertical block projection data reading unit 30 extracts the vertical block projection data (data array Mv (FIG. 9B)) and the n-value unit 314 compresses the data length. For each data array of the motion vector detection range (predetermined range) related to the frame, values of array elements having the same relative position with respect to the position of the array element at each first threshold intersection are added.

  When the vertical block n-valued data generated by the adder 315 is input to the peak detector 317, the horizontal peak position hv shown in FIG. 12B is detected as the horizontal motion vector of the vertical block. Is done. That is, the peak detector 317 detects a motion vector in the horizontal direction of the frame image based on the addition result added by the adder 315.

  In principle, the peak detector 317 searches for a peak position having the maximum value in the motion vector detection range (± V range). If there are a plurality of the same maximum values, the peak detector 317 is close to the origin 0 ( If the distance to the origin 0 is the same, the negative position) is given priority. It should be noted that the maximum value at the positive position may be prioritized without being limited to such a peak position detection rule, and the tendency of the motion vector in the previous frame is used for detection of the peak position in the current frame. Also good.

  The horizontal motion vector calculated for each vertical block by the peak detector 317 is sequentially input to the horizontal motion vector determination unit 32 via the output terminal 318.

  In the horizontal direction motion vector determination unit 32, the horizontal direction motion vector of the entire image is determined based on the horizontal direction motion vector of each vertical direction block sequentially output from the vertical direction block horizontal direction motion vector calculation unit 31. In the horizontal direction motion vector determination unit 32, for example, the horizontal direction motion vector having the highest appearance frequency is determined as the horizontal direction motion vector of the entire image. It is not essential to determine the horizontal motion vector having the highest appearance frequency as the horizontal motion vector of the entire image. The horizontal motion of each vertical block output from the vertical block horizontal motion vector calculation unit 31 is not necessarily required. You may make it determine the average value of a vector as a horizontal direction motion vector of the whole image.

  As described above, since the horizontal motion vector determination unit 32 determines the horizontal motion vector for the entire frame image based on the horizontal motion vector of each vertical block, even when there is motion in a part of the image. The false reaction to the movement can be suppressed. It is not always necessary to use the entire frame image when detecting the horizontal motion vector for the entire frame image. The horizontal motion vector is detected from a part of the frame image and used as the horizontal motion vector for the entire frame image. Also good.

  Next, the operation of the vertical motion vector detection unit 4 that detects the vertical motion vector of the image in the motion vector detection device 1 will be described.

  As shown in FIG. 2, when a frame image read out by repeating the pixel scanning of the horizontal line in order in the vertical direction is input to the input terminal 40 of the vertical direction motion vector detection unit 4 shown in FIG. In 41, the blocks are horizontally divided. That is, the horizontal image dividing unit 41 sets a plurality of image areas (predetermined image areas) obtained by dividing the frame image in the horizontal direction. As a result, in the subsequent processing, processing and management are performed for each horizontal block.

  In the horizontal image dividing unit 41, for example, as shown in FIG. 13A, image data of 640 pixels × 480 pixels is divided into 10 vertical blocks hb0 to hb0 having a width (divided width) of 64 pixels in the horizontal direction. Divided into hb9. Note that the division width and the number of divisions shown in FIG. 13A are not essential, and the number of divisions may be set to 1, for example.

  The image divided by the horizontal image dividing unit 41 into 10 horizontal blocks hb0 to hb9 as shown in FIG. 13A is extracted, in other words, edge components extending in the horizontal direction by the horizontal edge extraction filtering unit 42. For example, a filtering process for emphasizing an image portion that changes sharply in the vertical direction is performed. As a filter used in this filtering processing, a (1, -1,) 2-tap filter that simply takes a difference from a pixel adjacent in the horizontal direction, or (-1, 2, -1) corresponding to a second order differential. It is possible to use a 3-tap filter. It is not essential to use such a filter, and any filter may be used as long as the output value increases in an image portion where the luminance change is large in the vertical direction.

  The image data in which the horizontal edge is enhanced for each horizontal block (image region) hb0 to hb9 by the horizontal edge extraction filtering unit 42 is input to the horizontal block projection unit 43, and the horizontal block projection unit 43 outputs the image data. Projected horizontally. This projection can reduce the noise component in the horizontal direction (between lines) and can further enhance the edge component in the horizontal direction, so that the horizontal edge corresponding to the feature point can be emphasized and the motion vector detection accuracy can be improved. It will be. The operation of the horizontal block projection unit 43 will be described with reference to the conceptual diagram of FIG.

  In the horizontal block projection unit 43, when the input of one horizontal line of each of the horizontal blocks hb0 to hb9 shown in FIG. 13A is completed, the horizontal block projection data hn0 having the data array Mh for one vertical line is completed. One array element of .about.hn9 (FIG. 13B) is generated. Therefore, all the array elements are arranged when the input of the last horizontal line of each horizontal block hb0 to hb9 is completed. The data array Mh has array elements of the total number of pixels of the vertical line (for example, 480), but in a 2-tap filter such as (1, -1), the effective element is the total number of pixels of the vertical line − In a 3-tap filter such as 1 (for example, 479) and (-1, 2, -1), the effective elements are the total number of pixels in the vertical line -2 (for example, 478).

  Specifically, as shown in FIG. 7, the image data (image data subjected to edge enhancement in the horizontal direction) of the horizontal blocks hb0 to hb9 input via the input terminal 431 is input to the adder 432. Is done. The adder 432 first writes the data in the horizontal projection temporary storage memory 433 without reading the data in the horizontal projection temporary storage memory 433 at the head of one horizontal line of the input horizontal block. Next, the adder 432 reads the addition result up to the previous pixel of one horizontal line stored in the horizontal projection temporary storage memory 433, and the current pixel of one horizontal line of the horizontal block input from the input terminal 431. And the addition result is written back to the horizontal projection temporary storage memory 433. When the final pixel of one horizontal line in the horizontal block is input to the adder 432, the addition result up to the previous pixel stored in the horizontal projection temporary storage memory 433 is read and the horizontal input from the input terminal 431 is read. Addition with the final pixel of one horizontal line of the direction block, and output the addition result, that is, data obtained by adding all the pixels of one horizontal line of the horizontal block as projection data for one horizontal line of the horizontal block The data is output from the terminal 434 to the first horizontal block projection line memory 44 and the first horizontal block projection data maximum value storage unit 45. The projection data obtained by adding all the pixels of one horizontal line of the horizontal block in the adder 432 is temporarily stored in the horizontal projection temporary storage memory 433, and the first pixel of one horizontal line of the next horizontal block is input. In this case, the projection data in the horizontal projection temporary storage memory 433 may be read out and output from the output terminal 434.

  The first horizontal block projection line memory 44 stores the horizontal block projection data of the current frame input from the horizontal block projection unit 43 for each horizontal block as first horizontal block projection data.

  The first horizontal block projection data maximum value storage unit 45 calculates the maximum value related to the projection data of the horizontal block inputted from the horizontal block projection unit 43 and stores it as the horizontal block projection data maximum value of the current frame. To do.

  The second horizontal block projection line memory 46 stores the horizontal block projection data read from the first horizontal block projection line memory 44 as second horizontal block projection data related to the previous frame. Similarly, in the second horizontal block projection data maximum value storage unit 47, the maximum value of the horizontal block projection data output from the first horizontal block projection data maximum value storage unit 45 is the second value related to the previous frame. Is stored as the maximum horizontal block projection data. In other words, the maximum value of the horizontal block projection data of the current frame stored in the first horizontal block projection data maximum value storage unit 45 is stored in the second horizontal block projection data maximum value of the next frame. This is stored in the unit 47 as the maximum value of the horizontal block projection data of the previous frame.

  The second horizontal block projection data maximum value storage unit 47 calculates a fourth threshold value based on the calculated horizontal block projection data maximum value of the previous frame. For example, if the maximum value related to the projection data of the horizontal block of the previous frame is P4max, the fourth threshold value α4 is calculated by the following equation (3).

α4 = P4max × k4 (3)
Here, k4 is a predetermined coefficient, and 0 <k4 <1.

  Note that the calculation of the fourth threshold value α4 is not limited to the above equation (3), and it is sufficient if the fourth threshold value α4 increases as the horizontal block projection data maximum value P4max of the previous frame increases. Alternatively, a conversion table may be used. For the fourth threshold α4, it is not essential to use the maximum value of the second horizontal block projection data for each horizontal block. For example, the maximum horizontal block projection data for the entire image of the previous frame is used. The fourth threshold value may be calculated using the value, and the same fourth threshold value may be employed for the entire image.

  The second horizontal block projection data maximum value storage unit 47 sets a third threshold (predetermined constant value) based on the maximum value of the second horizontal block projection data regarding the previous frame. For example, if the maximum value of the horizontal block projection data of the previous frame is P3max, the third threshold value α3 is calculated by the following equation (4).

α3 = P3max × k3 (4)
Here, k3 is a predetermined coefficient, and 0 <k3 <1.

  Note that the calculation of the third threshold value α3 is not limited to the above equation (4), and it is sufficient if the third threshold value α3 increases as the horizontal block projection data maximum value P3max of the previous frame increases. Alternatively, a conversion table may be used. For the third threshold α3, it is not essential to use the maximum value of the second horizontal block projection data for each horizontal block. For example, the maximum horizontal block projection data for the entire image of the previous frame is used. The third threshold value may be calculated using the value, and the same third threshold value may be adopted for the entire image.

  The third threshold intersection search unit 48 outputs the second horizontal block projection data read from the second horizontal block projection line memory 46 and the second horizontal block projection data maximum value storage unit 47. The intersection with the third threshold value (third threshold value intersection) is searched in the vertical direction. The information on the third threshold intersection obtained by the third threshold intersection searching unit 48 is input to the horizontal block projection data reading unit 49 and the horizontal block vertical motion vector calculating unit 50.

  The horizontal block projection data reading unit 49 outputs the (first) horizontal block projection of the current frame corresponding to the motion vector detection range centered on the third threshold intersection output from the third threshold intersection search unit 48. Data is read from the first horizontal block projection line memory 44. That is, the third threshold intersection is B (i) (where i = 1, 2,... Q, q is the total number of detected third threshold intersections), and the motion vector detection range is the third threshold. Assuming the range from (−U) to (+ U) (where U is a positive integer) centered on the intersection, the projection data read from the first horizontal block projection line memory 44 is (B (i ) −U) to (B (i) + U), which is partial data of the first horizontal block projection data.

  The first horizontal block projection data read from the first horizontal block projection line memory 44 by the horizontal block projection data reading unit 49 is output to the horizontal block vertical motion vector calculation unit 50.

  The first horizontal block projection data output from the horizontal block projection data reading unit 49 is input to the n-value converter 504 via the input terminal 501 of the horizontal block vertical motion vector calculation unit 50 shown in FIG. Is done. The n-value converter 504 outputs the first horizontal block projection data based on the fourth threshold value output from the second horizontal block projection data maximum value storage unit 47 and input via the input terminal 502. n-valued. That is, the n-value converter 504 compresses the data length of each array element of the data array Mh (FIG. 13B) regarding the projection data of the current frame extracted by the horizontal block projection data reading unit 49. About the process of this n value digitizer 504, the process similar to the n value digitizer 314 mentioned above is performed, for example, a 3 value process is performed. That is, in the ternarization processing in the n-value quantizer 504, the horizontal block projection is performed with respect to the fourth threshold value α4 (α4> 0) set based on the maximum value of the array element of the data array Mh in the projection data of the previous frame. Three values in three stages when the value of the array element relating to the projection data of the current frame extracted by the data reading unit 49 is smaller than (−α4), (−α4) or more and α4 or less, and greater than α4. Is done. Note that the n-value generator 504 does not necessarily have the same characteristics as the n-value generator 314, and may have different characteristics from the n-value generator 504.

  The operations of the third threshold value intersection searching unit 48, the horizontal block projection data reading unit 49, and the horizontal block vertical motion vector calculating unit 50 described above will be specifically described with reference to FIG. 14A and 14B indicate the positions of the array elements of the projection data array Mh (FIG. 13B). In FIG. 14A, the third threshold value intersections B (1) to B (6) are indicated by circles.

  In the third threshold intersection search unit 48, the second horizontal block projection data maximum value storage unit 47 outputs the waveform of the second vertical block projection data W4 for the previous frame as shown in FIG. The third threshold value intersections B (1) to B (6) that intersect with the third threshold value α3 are obtained. That is, the third threshold value intersection search unit 48 sets the array element values in the order of the elements in the projection data array Mh (FIG. 13B) with respect to the projection data obtained by the horizontal block projection unit 43 with respect to the previous frame. The position of the array element at each of the third threshold intersections where the waveform W4 and the straight line where the value of the array element is the third threshold value (predetermined constant value) α3 intersect is specified.

  Next, in the horizontal block projection data reading unit 49, a first horizontal vector is detected for a predetermined motion vector detection range centered on each of the third threshold intersections B (1) to B (6) shown in FIG. The horizontal block projection data of the current frame is read from the direction block projection line memory 44. That is, the horizontal block projection data reading unit 49 obtains a data array in a predetermined range centered on the position of the array element of the projection data (data array) at each third threshold intersection at the horizontal block projection unit 43. Extracted from the projection data of the current frame (back frame). For example, with respect to the third threshold value intersection B (4), as shown in FIG. 14B, the third threshold value intersection B (4) is centered on (B (4) -U) to (B (4) + U). The first horizontal block projection data W3 corresponding to the range up to (the waveform portion within the rectangular broken line) is read out.

  The first horizontal block projection data read out by the horizontal block projection data reading unit 49 uses the fourth threshold value α4 output from the second horizontal block projection data maximum value storage unit 47. The n-value unit 504 of the horizontal block vertical motion vector calculation unit 50 performs n-value processing. For example, for the third threshold value intersection B (4), the third threshold value is based on the magnitude relationship of the first horizontal block projection data W3 with respect to the fourth threshold values α4 and (−α4) shown in FIG. For the range from (B (4) -U) to (B (4) + U) centering on the intersection B (4) (the waveform portion within the rectangular broken line), the first horizontal as shown in FIG. The direction block projection data is ternarized. As a result, the horizontal block projection data of the current frame is represented by “−1”, “0”, or “1”.

  In the n-value converter 504, n-value conversion processing as shown in FIG. 14C is performed on all the third threshold value intersections output from the third threshold value intersection search unit 48.

  Next, the adder 505 (FIG. 8) is n-valued by the n-value generator 504 around the third threshold value intersection output from the third threshold value intersection search unit 48 via the input terminal 503. The first horizontal block projection data is added to the addition value of the first horizontal block projection data converted to n-values read from the vertical motion vector addition memory 506, and the addition result Are stored in the vertical motion vector addition memory 506 again. Then, the adder 505 adds the n-valued first horizontal block projection data regarding the last one of all the third threshold intersections detected by the third threshold intersection search unit 48 to obtain one When the addition process for the horizontal block is finished, the addition result is output to the peak detector 507. In addition, after the addition process in the adder 505 is completed, the peak detection unit 507 may read out the addition result from the vertical motion vector addition memory 506 storing the addition results regarding all the third threshold intersections. good.

  The peak detector 507 includes a motion vector detection range (the above-described ± U range) centered on each third threshold intersection for all the third threshold intersections detected by the third threshold intersection search unit 48. Data obtained by adding the corresponding n-valued first horizontal block projection data (hereinafter also referred to as “horizontal block n-valued data”) is input. The value is detected by peak detector 507. The peak position of the horizontal block n-valued data detected by the peak detector 507 becomes the vertical motion vector obtained from the horizontal block, and is output to the vertical motion determination unit 51 via the output terminal 508. .

  The operations of the adder 505 and the peak detector 507 described above will be specifically described with reference to FIG. In FIG. 15A, the first horizontal direction in the motion vector detection range (± U range) centering on the third threshold intersections B (1) to B (6) shown in FIG. The block projection data is conceptually shown as three values.

  When the adder 505 sequentially inputs the ternary data around the third threshold intersections B (1) to B (6) shown in FIG. 15A, these data are added together, and FIG. Horizontal block n-valued data as shown in b) is generated. Specifically, the adder 505 extracts the current block projection data (data array Mh (FIG. 13B)) from the horizontal block projection data reading unit 49 and the n-value unit 504 compresses the data length. For each data array of the motion vector detection range (predetermined range) related to the frame, values of array elements having the same relative position with respect to the position of the array element at each third threshold intersection are added.

  When the horizontal block n-valued data generated by the adder 505 is input to the peak detector 507, the vertical peak position vv shown in FIG. 15B is detected as the vertical motion vector of the horizontal block. Is done. That is, the peak detector 507 detects a motion vector in the vertical direction of the frame image based on the addition result added by the adder 505.

  In principle, the peak detector 507 searches for a peak position having a maximum value in the motion vector detection range (± U range). When there are a plurality of the same maximum values, the peak detector 507 is close to the origin 0 ( If the distance to the origin 0 is the same, the negative position) is given priority. It should be noted that the maximum value at the positive position may be prioritized without being limited to such a peak position detection rule, and the tendency of the motion vector in the previous frame is used for detection of the peak position in the current frame. Also good.

  The vertical motion vector calculated for each horizontal block by the peak detector 507 is sequentially input to the vertical motion vector determination unit 51 via the output terminal 508.

  The vertical motion vector determination unit 51 determines the vertical motion vector of the entire image based on the vertical motion vector of each horizontal block sequentially output from the horizontal block vertical motion vector calculation unit 50. The vertical motion vector determination unit 51 determines, for example, the vertical motion vector having the highest appearance frequency as the vertical motion vector of the entire image. It is not essential to determine the vertical motion vector having the highest appearance frequency as the vertical motion vector of the entire image, and the vertical motion of each horizontal block output from the horizontal block vertical motion vector calculation unit 50 is not necessarily required. You may make it determine the average value of a vector as a vertical direction motion vector of the whole image.

  As described above, since the vertical motion vector determination unit 51 determines the vertical motion vector for the entire frame image based on the vertical motion vector of each horizontal block, even when there is motion in a part of the image. The false reaction to the movement can be suppressed. It is not always necessary to use the entire frame image when detecting the vertical motion vector for the entire frame image. The vertical motion vector is detected from a part of the frame image and is used as the vertical motion vector for the entire frame image. Also good.

  By the above operation of the motion vector detecting device 1, the vertical block projection data is generated by projecting in the vertical direction for each vertical block in which the vertical edge is emphasized, and the vertical block projection data of the previous frame is generated. The vertical block projection data of the current frame corresponding to the motion vector detection range centered on the intersection with the first threshold value α1 (first threshold value intersection) is read and added for each distance from each first threshold value intersection. Since the motion vector in the horizontal direction is detected from the addition result, the motion of the vertical edge of the image corresponding to the feature point can be tracked with a small amount of calculation. Similarly, horizontal block projection data is generated by projecting in the horizontal direction for each horizontal block in which the edge in the horizontal direction is emphasized, and the intersection of the horizontal block projection data of the previous frame and the third threshold value α3. The horizontal frame projection data of the current frame corresponding to the motion vector detection range centered on (third threshold intersection) is read and added for each distance from each third threshold intersection, and the vertical direction is determined from the addition result. Since the motion vector is detected, the movement of the horizontal edge of the image corresponding to the feature point can be tracked with a small amount of calculation. As a result, the detection of motion vectors in the horizontal direction and the vertical direction can be performed quickly with high accuracy.

  Further, the bit number reduction unit 25 of the motion vector detection device 1 uses the maximum value of the vertical block projection data of the current frame obtained by the first vertical block projection data maximum value storage unit 24 to determine the vertical direction of the current frame. Since the number of bits of the block projection data is reduced, the amount of calculation in the subsequent processing can be suppressed, and the motion vector can be detected more quickly.

  Further, in the motion vector detection device 1, an appropriate first threshold value intersection is set in order to obtain the maximum value of the second vertical block projection data regarding the previous frame and to set the first threshold value α1 based on this maximum value. it can. Similarly, since the maximum value of the second horizontal block projection data regarding the previous frame is obtained and the third threshold value α3 is set based on this maximum value, an appropriate third threshold value intersection can be set.

  Further, in the motion vector detection device 1, the vertical image is divided by the vertical image dividing unit 21 and then the vertical edge of each divided image (vertical block) is emphasized. Therefore, the vertical edge is emphasized. The divided image can be easily generated. Similarly, after the frame image is divided in the horizontal direction by the horizontal image dividing unit 41, the horizontal edge of each divided image (horizontal block) is emphasized. Therefore, the divided image in which the horizontal edge is emphasized is simplified. Can be generated.

  Further, in the motion vector detection device 1, the n-value generator 314 of the vertical block horizontal motion vector calculation unit 31 converts the projection data read by the vertical block projection data reading unit 30 into n-values. The amount of calculation can be reduced, and motion vectors can be detected more quickly. In particular, since the n-value converter 314 performs ternarization based on the second threshold value α2 set based on the maximum value of the vertical block projection data of the current frame, real-time processing is performed while ensuring a relatively large dynamic range. Possible motion vector detection can be realized. Also, the memory can be greatly reduced.

  Similarly, since the projection data read out by the horizontal block projection data reading unit 49 is converted into n values in the n-value converter 504 of the horizontal block vertical motion vector calculation unit 50, the subsequent calculation amount can be reduced. The motion vector can be detected more quickly. In particular, since the n-value converter 504 performs ternarization based on the fourth threshold value α4 set based on the maximum value of the horizontal block projection data of the previous frame, an appropriate fourth threshold value α4 can be set, and dynamic Motion vector detection capable of real-time processing can be realized without greatly degrading the range. Also, the memory can be greatly reduced.

<Embodiment 2>
<Configuration of motion vector detection device>
The motion vector detection device 1A according to the second embodiment of the present invention has a configuration similar to that of the motion vector detection device 1 according to the first embodiment shown in FIG. 1, but is perpendicular to the horizontal motion vector detection unit. The configuration differs from that of the directional motion vector detection unit. The configuration of the horizontal direction motion vector detection unit 2A and the vertical direction motion vector detection unit 4A in the motion vector detection device 1A will be described with reference to FIGS.

  FIG. 16 is a block diagram showing a main configuration of the horizontal direction motion vector detection unit 2A. In FIG. 16, parts having the same functions as those in the first embodiment are denoted by the same reference numerals.

  In the horizontal direction motion vector detection unit 2A, the arrangement of the vertical direction image division unit 21 and the vertical direction edge extraction filtering unit 22 is reversed with respect to the horizontal direction motion vector detection unit 2 of the first embodiment. Below, operation | movement of this horizontal direction motion vector detection part 2A is demonstrated.

  As shown in FIG. 2, when a frame image read by sequentially repeating pixel scanning of the horizontal line in the vertical direction is input to the input terminal 20 of the horizontal motion vector detection unit 2A, the vertical edge extraction filtering unit 22 Extraction of edge components extending in the vertical direction, in other words, filtering processing for emphasizing image portions that change sharply in the horizontal direction is performed.

  When the frame image in which the vertical edge is emphasized by the vertical edge extraction filtering unit 22 is input to the vertical image dividing unit 21, it is divided into blocks in the vertical direction. That is, the vertical image dividing unit 21 sets a plurality of image areas (predetermined image areas) obtained by dividing the frame image subjected to edge enhancement in the vertical direction. Thereby, in the subsequent processing, processing and management are performed for each vertical block.

  The image data divided by the vertical image dividing unit 21 is input to the vertical block projecting unit 23. The processes after the vertical block projecting unit 23 are the horizontal direction motion vector detecting unit 2 of the first embodiment. The same processing is performed.

  FIG. 17 is a block diagram showing a main configuration of the vertical motion vector detection unit 4A. In FIG. 17, the same reference numerals are given to parts having the same functions as those in the first embodiment.

  In the vertical direction motion vector detection unit 4A, the arrangement of the horizontal direction image division unit 41 and the horizontal direction edge extraction filtering unit 42 is reversed with respect to the vertical direction motion vector detection unit 4 of the first embodiment. Hereinafter, the operation of the vertical motion vector detection unit 4A will be described.

  As shown in FIG. 2, when a frame image read out by repeating the pixel scanning of the horizontal line in order in the vertical direction is input to the input terminal 40 of the vertical direction motion vector detection unit 4A, the horizontal direction edge extraction filtering unit 42 Extraction of the edge component extending in the horizontal direction, in other words, filtering processing for emphasizing an image portion that changes sharply in the vertical direction is performed.

  When the frame image in which the horizontal edge is emphasized by the horizontal edge extraction filtering unit 42 is input to the horizontal image dividing unit 41, the frame image is divided into blocks in the horizontal direction. That is, the horizontal image dividing unit 41 sets a plurality of image areas (predetermined image areas) obtained by dividing the frame image subjected to edge enhancement in the horizontal direction. Thereby, in subsequent processing, processing and management are performed for each horizontal block.

  The image data divided by the horizontal image dividing unit 41 is input to the horizontal block projecting unit 43. The processing after the horizontal block projecting unit 43 is the vertical motion vector detecting unit 4 of the first embodiment. The same processing is performed.

  The operation of the motion vector detection device 1A described above provides the same effects as those of the first embodiment.

  Then, in the motion vector detection device 1A, the vertical edge extraction filtering unit 22 emphasizes the edge in the vertical direction with respect to the frame image, and then the vertical image dividing unit 21 divides the frame image subjected to edge enhancement in the vertical direction. A divided image with emphasized vertical edges can be easily generated. Similarly, the horizontal edge extraction filtering unit 42 emphasizes the horizontal edge of the frame image, and then the edge-enhanced frame image is divided in the horizontal direction by the horizontal image dividing unit 41. Therefore, the horizontal edge is emphasized. The divided image can be easily generated.

  In the motion vector detection device 1A, it is essential to arrange the vertical edge extraction filtering unit 22 and the horizontal edge extraction filtering unit 42 separately in the horizontal motion vector detection unit 2A and the vertical motion vector detection unit 4A. In addition, a horizontal / vertical edge extraction filtering unit in which the functions of both the vertical edge extraction filtering unit 22 and the horizontal edge extraction filtering unit 42 are integrated is provided, and the output is provided to the vertical image dividing unit 21 and the horizontal direction. You may make it make each image division part 41 input.

<Modification>
For the vertical edge extraction filtering unit and the horizontal edge extraction filtering unit in each of the above embodiments, after the filtering process for performing edge extraction, for example, “there is an edge” by the threshold process for determining the magnitude relationship with a predetermined threshold The binarization of “no edge” or the binarization of “with positive edge”, “without edge”, and “with negative edge” may be performed. By performing such threshold processing, it is possible to treat edges having a luminance change (luminance gradient) equal to or greater than a predetermined threshold in the same way.

  In each of the above-described embodiments, the data output from the bit number reduction unit 25 is converted into the first vertical block projection line memory 26 and the second vertical block projection line memory 27 by, for example, switch switching for each frame. Alternatively, the input may be made alternately. Similarly, the data output from the vertical (horizontal) direction block projection unit is converted into, for example, a first vertical (horizontal) direction block projection data maximum value storage unit and a second vertical (horizontal) direction block by switching switches for each frame. Alternatively, the data may be alternately input to the projection data maximum value storage unit.

  In the present invention, the “previous frame” is not limited to the previous frame, but includes two or more previous frames with respect to the subsequent frame (for example, the current frame). For example, when a frame in the middle is skipped by the frame thinning process, the frame before the skipped frame is included.

It is a block diagram which shows the principal part structure of the motion vector detection apparatus 1 which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the pixel scanning direction in a frame image. 3 is a block diagram showing a main configuration of a horizontal motion vector detection unit 2. FIG. FIG. 4 is a block diagram showing a main configuration of a vertical block projection unit 23. FIG. 6 is a block diagram showing a main configuration of a vertical block horizontal motion vector calculation unit 31. FIG. 3 is a block diagram illustrating a main configuration of a vertical motion vector detection unit 4. FIG. 4 is a block diagram illustrating a main configuration of a horizontal block projection unit 43. FIG. 6 is a block diagram showing a main configuration of a horizontal block vertical motion vector calculation unit 50. It is a figure for demonstrating operation | movement of the vertical direction image division part 21 and the vertical direction block projection part 23. FIG. It is a figure for demonstrating the n-value conversion process in the n-value conversion device. It is a figure for demonstrating operation | movement of the 1st threshold value intersection search part 29, the vertical direction block projection data reading part 30, and the vertical direction block horizontal direction motion vector calculation part 31. FIG. 6 is a diagram for explaining operations of an adder 315 and a peak detector 317. FIG. It is a figure for demonstrating operation | movement of the horizontal direction image division part 41 and the horizontal direction block projection part 43. FIG. It is a figure for demonstrating operation | movement of the 3rd threshold value intersection search part 48, the horizontal direction block projection data reading part 49, and the horizontal direction block vertical direction motion vector calculation part 50. FIG. FIG. 10 is a diagram for explaining operations of an adder 505 and a peak detection unit 507. It is a block diagram which shows the principal part structure of 2 A of horizontal direction motion vector detection parts in 1 A of motion vector detection apparatuses which concern on Embodiment 2 of this invention. It is a block diagram which shows the principal part structure of 4 A of vertical direction motion vector detection parts in 1 A of motion vector detection apparatuses.

Explanation of symbols

1, 1A motion vector detection device, 2, 2A horizontal direction motion vector detection unit, 4, 4A vertical direction motion vector detection unit, 21 vertical direction image segmentation unit, 22 vertical direction edge extraction filtering unit, 23 vertical direction block projection unit, 24 first vertical block projection data maximum value storage unit, 25 bit number reduction unit, 26 first vertical block projection line memory, 27 second vertical block projection line memory, 28 second vertical block projection Data maximum value storage unit, 29 first threshold intersection search unit, 30 vertical block projection data reading unit, 31 vertical block horizontal motion vector calculation unit, 32 horizontal motion vector determination unit, 41 horizontal image segmentation unit, 42 horizontal edge extraction filtering unit, 43 horizontal block projection unit, 44 first Horizontal block projection line memory, 45 First horizontal block projection data maximum value storage unit, 46 Second horizontal block projection line memory, 47 Second horizontal block projection data maximum value storage unit, 48 Third Threshold intersection search unit, 49 horizontal block projection data read unit, 50 horizontal block vertical motion vector calculation unit, 51 vertical motion vector determination unit, hb0 to hb9 horizontal block, hn0 to hn9 horizontal block projection data, vb0 ˜vb7 vertical block, vn0 to vn6 vertical block projection data.

Claims (13)

A motion vector detection device that detects a motion vector between frame images related to a preceding frame and a following frame that are in a time series front-rear relationship,
(a) with respect to a frame image read out by repeating pixel scanning of a horizontal line in order in the vertical direction, edge enhancement means for enhancing a vertical edge for a predetermined image area in the frame image;
(b) Projection means for projecting the image in which the edge is enhanced by the edge enhancement means in the vertical direction and generating projection data having a data array for one horizontal line;
(c) a waveform obtained by graphing array element values in the order of the elements of the data array with respect to the projection data obtained by the projection means for the previous frame, and a straight line in which the array element values are a predetermined constant value. A specifying means for specifying the position of the array element at each intersection that intersects;
(d) extraction means for extracting a predetermined range of data array centered on the position of the array element at each intersection from the projection data obtained by the projection means for the subsequent frame;
(e) for each data array in a predetermined range extracted by the extracting means, an adding means for adding values of array elements having the same relative position to the position of the array element at each intersection;
(f) detection means for detecting a motion vector in the horizontal direction of the frame image based on the addition result added by the addition means;
A motion vector detection device comprising: The motion vector detection device according to claim 1,
Maximum value obtaining means for obtaining a maximum value in the array element of the projection data;
Reduction means for reducing the data length of each array element related to the projection data based on the maximum value obtained by the maximum value acquisition means;
Storage means for storing projection data whose data length has been reduced by the reduction means for the subsequent frame;
Further comprising
The specifying means is:
(c-1) A waveform obtained by graphing array element values in the order of the elements of the data array with respect to the projection data whose data length has been reduced by the reduction means for the previous frame, and the array element values are a predetermined constant value. Position specifying means for specifying the position of the array element at each intersection where the straight line intersects,
And having
The extraction means includes
(d-1) A data array of a predetermined range centered on the position of the array element of each intersection specified by the position specifying means, from the projection data of the frame after the data length stored in the storage means is reduced Means to extract,
A motion vector detection apparatus comprising: The motion vector detection device according to claim 1 or 2,
Maximum value specifying means for obtaining a maximum value in an array element of projection data relating to the previous frame;
Further comprising
The motion vector detecting device, wherein the predetermined constant value is set based on a maximum value obtained by the maximum value specifying means. The motion vector detection device according to any one of claims 1 to 3,
The edge enhancement means includes
(a-1) means for setting a plurality of image areas obtained by dividing the frame image in the vertical direction as the predetermined image areas;
(a-2) means for enhancing vertical edges for each of the plurality of image regions;
A motion vector detection apparatus comprising: The motion vector detection device according to any one of claims 1 to 3,
The edge enhancement means includes
(a-3) enhancement means for enhancing the vertical edge of the frame image;
(a-4) means for setting, as the predetermined image area, a plurality of image areas obtained by dividing a frame image whose edges are emphasized by the enhancement means in the vertical direction;
A motion vector detection apparatus comprising: The motion vector detection device according to any one of claims 1 to 5,
Compression means for compressing the data length of each array element relating to the data array in a predetermined range extracted by the extraction means;
Further comprising
The adding means includes
(e-1) means for adding the values of array elements having the same relative position with respect to the position of the array element at each intersection for each data array in a predetermined range whose data length is compressed by the compression means;
A motion vector detection apparatus comprising: The motion vector detection device according to claim 6,
The compression means includes
Means for ternarizing the values of each array element relating to the data array in a predetermined range extracted by the extracting means;
Have
In the ternary processing, with respect to the threshold Th (Th> 0) set based on the maximum value of the array elements in the projection data of the subsequent frame, the values of the array elements related to the projection data extracted by the extraction unit are ( A motion vector detection device characterized in that ternarization is performed in three stages when it is smaller than -Th), when it is larger than (-Th) and smaller than Th, and larger than Th. A motion vector detection device that detects a motion vector between frame images related to a preceding frame and a following frame that are in a time series front-rear relationship,
(a) with respect to a frame image that is read out by repeating pixel scanning of a horizontal line in order in the vertical direction, edge enhancement means that emphasizes a horizontal edge for a predetermined image region in the frame image;
(b) Projection means for projecting in the horizontal direction the image whose edge is enhanced by the edge enhancement means and generating projection data having a data array for one vertical line;
(c) a waveform obtained by graphing array element values in the order of the elements of the data array with respect to the projection data obtained by the projection means for the previous frame, and a straight line in which the array element values are a predetermined constant value. A specifying means for specifying the position of the array element at each intersection that intersects;
(d) extraction means for extracting a predetermined range of data array centered on the position of the array element at each intersection from the projection data obtained by the projection means for the subsequent frame;
(e) for each data array in a predetermined range extracted by the extracting means, an adding means for adding values of array elements having the same relative position to the position of the array element at each intersection;
(f) detection means for detecting a motion vector in the vertical direction of the frame image based on the addition result added by the addition means;
A motion vector detection device comprising: The motion vector detection device according to claim 8,
Maximum value specifying means for obtaining a maximum value in an array element of projection data relating to the previous frame;
Further comprising
The motion vector detecting device, wherein the predetermined constant value is set based on a maximum value obtained by the maximum value specifying means. The motion vector detection device according to claim 8 or 9,
The edge enhancement means includes
(a-1) means for setting a plurality of image areas obtained by dividing the frame image in the horizontal direction as the predetermined image areas;
(a-2) means for enhancing a horizontal edge for each of the plurality of image regions;
A motion vector detection apparatus comprising: The motion vector detection device according to claim 8 or 9,
The edge enhancement means includes
(a-3) enhancement means for enhancing horizontal edges of the frame image;
(a-4) means for setting, as the predetermined image area, a plurality of image areas obtained by horizontally dividing a frame image whose edges are emphasized by the enhancement means;
A motion vector detection apparatus comprising: The motion vector detection device according to any one of claims 8 to 11,
Compression means for compressing the data length of each array element relating to the data array in a predetermined range extracted by the extraction means;
Further comprising
The adding means includes
(e-1) means for adding the values of array elements having the same relative position with respect to the position of the array element at each intersection for each data array in a predetermined range whose data length is compressed by the compression means;
A motion vector detection apparatus comprising: The motion vector detection device according to claim 12,
The compression means includes
Means for ternarizing the values of each array element relating to the data array in a predetermined range extracted by the extracting means;
Have
In the ternary processing, with respect to the threshold Th (Th> 0) set based on the maximum value of the array element in the projection data of the previous frame, the value of the array element related to the projection data extracted by the extraction unit is ( A motion vector detection device characterized in that ternarization is performed in three stages when it is smaller than -Th), when it is larger than (-Th) and smaller than Th, and larger than Th.

Images