JP4313820B2 - Motion vector detection device - Google Patents

Description

  The present invention relates to a motion vector detection apparatus that detects a motion vector between frame images related to a previous frame and a subsequent frame that are in a time series relationship.

  As a motion vector detection technique between frame images used for camera shake correction or the like, as described in Patent Document 1, from a difference image for each of a plurality of temporally continuous images, a pixel with a peripheral pixel for each pixel Compare the values, estimate the direction of motion for each direction, calculate the average of the sum of those motion vectors, calculate the motion vector for each pixel, calculate the local motion vector (optical flow) ing.

  At this time, in order to reduce (reduce) the influence of noise on the optical flow obtained for each pixel, a smoothing process such as averaging the sum of initial estimated motion vectors including the surrounding eight pixels is performed. Yes.

  Alternatively, in Patent Document 2, a smoothing method similar to that in Patent Document 1 is applied to projection data, the input image is divided into partial areas, and motion vectors calculated for each of the partial areas are used. The motion vector of the entire image is calculated.

  In general, as a technique for removing noise, a 3-tap low-pass filter composed of (1/4, 1/2, 1/4) as described in FIG. Often used.

JP-A-10-240945 JP 2002-63578 A Japanese Patent No. 3314556

  In the smoothing methods disclosed in Patent Document 1 and Patent Document 2, the calculated motion vector is smoothed by taking the average of the surrounding pixel results and the original image used for motion vector calculation or the projection data itself. On the other hand, noise removal is not performed. For this reason, the motion vector calculation process is performed using noise-containing data, and there is a problem in that the motion vector cannot be accurately calculated when the influence of noise reaches the surrounding pixels.

  Further, in the smoothing method using the low-pass filter disclosed in Patent Document 3, the high-frequency components (characteristics) of the applied original image and projection data are significantly impaired. ) May be lost.

  The present invention has been made to solve the above-described problems, and smoothing means that can remove (reduce) only noise components without impairing the characteristics (features) necessary for motion vector calculation. It is an object of the present invention to provide a motion vector detection device having the above.

  The motion vector detection device according to claim 1 according to the present invention is a motion vector detection device for detecting a motion vector between frame images related to a previous frame and a rear frame that are in a time series relationship, and is a horizontal line. With respect to a frame image that is read out by repeating the pixel scanning in order in the vertical direction, an edge emphasizing unit that emphasizes an edge in the vertical direction for a predetermined image region in the frame image, and an image in which the edge is emphasized by the edge emphasizing unit Projection means for projecting in the vertical direction and generating projection data having a data array for one horizontal line; smoothing means for removing smoothing noise superimposed on the projection data to obtain smooth projection data; and Regarding the smoothed projection data with respect to the previous frame, the values of the array elements are set in the order of the elements of the data array. A specifying means for specifying the position of the array element at each intersection where the rough waveform and the straight line where the value of the array element is a predetermined constant value intersect, and a predetermined range centered on the position of the array element at each intersection For each of the data elements in a predetermined range extracted by the extraction means, relative to the position of the array element at each intersection. Adding means for adding values of array elements having the same position; and detecting means for detecting a horizontal motion vector for a predetermined image area of the frame image based on the addition result added by the adding means. ing.

  The motion vector detection device according to claim 2 according to the present invention is a motion vector detection device for detecting a motion vector between frame images related to a previous frame and a subsequent frame that are in a time series relationship, and is a horizontal line. With respect to a frame image that is read out by repeating the pixel scanning in order in the vertical direction, an edge enhancement unit that emphasizes a horizontal edge in a predetermined image region in the frame image, and an image in which the edge is enhanced by the edge enhancement unit Projection means for projecting in the horizontal direction and generating projection data having a data arrangement for one vertical line; smoothing means for removing smoothing noise superimposed on the projection data to obtain smooth projection data; and The values of array elements are grouped in the order of the elements of the data array with respect to the smoothed projection data with respect to the previous frame. A specifying means for specifying the position of the array element at each intersection where the flattened waveform and the straight line having the predetermined value of the array element intersect, and a predetermined range centered on the position of the array element at each intersection For each of the data elements in a predetermined range extracted by the extraction means, relative to the position of the array element at each intersection. Adding means for adding values of array elements having the same position; and detecting means for detecting a vertical motion vector for a predetermined image area of the frame image based on the addition result added by the adding means. ing.

  According to the motion vector detection device of the first aspect of the present invention, the smoothing means removes the superimposed fluctuation noise without impairing the characteristics of the projection data necessary for motion vector detection. Horizontal direction motion vector detection is possible.

  According to the motion vector detection device of the second aspect of the present invention, the smoothing means removes the superimposed fluctuation noise without impairing the characteristics of the projection data necessary for motion vector detection. Vertical motion vector detection is possible.

<Embodiment 1>
<Configuration of motion vector detection device>
FIG. 1 is a block diagram showing a main configuration of a motion vector detection device 1 according to Embodiment 1 of the present invention.

  The motion vector detection device 1 detects, for example, an image motion vector that represents the motion of a subject in a screen in a moving image, and the motion between frame images related to a previous frame and a subsequent frame that are in a chronological order. A vector is detected. The motion vector detected by the motion vector detection device 1 is used for camera shake correction of the video camera. Note that the frame image input to the motion vector detection device 1 is read out by performing horizontal pixel scanning and vertical line scanning as shown in FIG.

  The motion vector detection device 1 includes an input terminal 11, a horizontal direction motion vector detection unit 2 and a vertical direction motion vector detection unit 4, and two output terminals 12 and 13. The functions of the horizontal motion vector detection unit 2 and the vertical motion vector detection unit 4 may be realized by software by, for example, a microprocessor, and most of the functions are realized by hardware (partly software). It may be. Of course, it goes without saying that all functions may be realized by hardware.

  For example, image data such as image (moving image) data or a video signal acquired by an image sensor in a video camera is input to the input terminal 11. In the image data input to the input terminal 11, a horizontal motion vector is detected by the horizontal motion vector detection unit 2, and the detected horizontal motion vector is output from the output terminal 12.

  On the other hand, the vertical motion vector detector 4 detects a vertical motion vector for the image data input to the input terminal 11. The vertical motion vector detected by the vertical motion vector detection unit 4 is output from the output terminal 13.

  Below, the structure of the horizontal direction motion vector detection part 2 and the vertical direction motion vector detection part 4 is demonstrated in order.

FIG. 3 is a block diagram showing the main configuration of the horizontal motion vector detection unit 2.
The horizontal motion vector detection unit 2 includes an input terminal 20, a vertical image segmentation unit 21, a vertical edge extraction filtering unit 22, a vertical block projection unit 23, and a first vertical block projection data maximum value storage. A unit 24, a bit number reduction unit 25, a first vertical block projection line memory 26, and a vertical block projection data smoothing unit 34. The horizontal motion vector detection unit 2 includes a second vertical block projection line memory 27, a second vertical block projection data maximum value storage unit 28, a first threshold intersection search unit 29, and a vertical direction. A block projection data reading unit 30, a vertical block horizontal motion vector calculating unit 31, a horizontal motion vector determining unit 32, and an output terminal 33 are provided. The function of each part of the horizontal motion vector detection unit 2 described above will be briefly described (specific operations will be described in detail later).

  The vertical image dividing unit 21 divides the frame image input to the input terminal 20 in the vertical direction and outputs a block divided in the vertical direction (hereinafter also referred to as “vertical block”).

  The vertical edge extraction filtering unit 22 performs a filtering process for performing edge extraction for each block divided by the vertical image dividing unit 21.

  The vertical block projection unit 23 projects the edge-enhanced vertical direction block output from the vertical edge extraction filtering unit 22 in the vertical direction and outputs projection data for each vertical block.

The first vertical block projection data maximum value storage unit 24 stores the maximum value in the vertical block projection data of the current frame output from the vertical block projection unit 23.
(Hereinafter, also referred to as “vertical block projection data maximum value of the current frame” or “first vertical block projection data maximum value”). The first vertical block projection data maximum value storage unit 24 calculates a second threshold value to be described later based on the first vertical block projection data maximum value.

  The bit number reduction unit 25 is configured to output the vertical block output from the vertical block projection unit 23 based on the first vertical block projection data maximum value stored in the first vertical block projection data maximum value storage unit 24. Reduce the number of bits of projection data. The projection data of the vertical block in which the number of bits is reduced is referred to as “first vertical block projection data”.

  The first vertical block projection line memory 26 stores the first vertical block projection data whose bit number has been reduced by the bit number reduction unit 25.

  The vertical block projection data smoothing unit 34 reads the vertical block projection data stored in the first vertical block projection line memory 26, removes the fluctuation noise superimposed on the vertical block projection data, and performs smoothing. The converted projection data (hereinafter referred to as smoothed projection data) is written back to the first vertical block projection line memory 26. Accordingly, the “projection data” in the subsequent processing is “smoothed projection data (smoothed projection data)”, including the case where it is not particularly distinguished and described, and “first vertical block projection”. “Data” is also the smoothed projection data.

  The second vertical block projection line memory 27 converts the vertical block projection data sent from the first vertical block projection line memory 26 into the projection data of the vertical block of the previous frame (hereinafter referred to as “second vertical block projection data”). It is also saved as “direction block projection data”.

  The second vertical block projection data maximum value storage unit 28 uses the first vertical block projection data maximum value output from the first vertical block projection data maximum value storage unit 24 as the “second Is stored as “vertical block projection data maximum value” (also referred to as “vertical block projection data maximum value of previous frame”). Further, the second vertical block projection data maximum value storage unit 28 calculates a first threshold value to be described later based on the second vertical block projection data maximum value.

  The first threshold value intersection search unit 29 calculates the waveform of the second vertical block projection data related to the previous frame stored in the second vertical block projection line memory 27 and the second vertical block projection data maximum value. A point where the first threshold value calculated by the storage unit 28 intersects is obtained, and information on this intersection (hereinafter also referred to as “first threshold value intersection”) is output.

  The vertical block projection data reading unit 30 has a first vertical direction within a predetermined range (motion vector detection range) before and after the first threshold intersection obtained by the first threshold intersection search unit 29. The block projection data is read from the first vertical block projection line memory 26.

  The vertical block horizontal motion vector calculation unit 31 uses the second threshold output from the first vertical block projection data maximum value storage unit 24 and reads the first block read by the vertical block projection data reading unit 30. Each of the vertical block projection data of 1 is converted into n-values (n is an integer of 2 or more) to reduce the number of bits, and the n-valued projection data is added for each distance from the first threshold intersection. .

  The horizontal motion vector determination unit 32 determines the horizontal motion vector of the image based on the output from the vertical block horizontal motion vector calculation unit 31. The horizontal motion vector of the image determined here is output from the output terminal 33.

FIG. 4 is a block diagram showing a main configuration of the vertical block projection unit 23.
The vertical block projection unit 23 includes an input terminal 231, an adder 232 that performs data addition for each horizontal line, and a vertical projection temporary storage memory 233 as a buffer memory that sequentially stores data added by the adder 232. And an output terminal 234.

  An image whose vertical edges are emphasized by the vertical edge extraction filtering unit 22 is input to the input terminal 231 for each vertical block. The image input to the input terminal 231 is projected in the vertical direction by the adder 232 and the vertical projection temporary storage memory 233 for each vertical block. Specifically, the adder 232 adds the data for one horizontal line of the vertical block and the data for one horizontal line read from the vertical projection temporary memory 233, and the addition result is the vertical projection temporary. It is returned again to the storage memory 233 and stored. When the addition of all the horizontal lines in the vertical block is completed by repeating such an addition operation, the addition data of all the horizontal lines, that is, the projection data in the vertical direction is output from the output terminal 234.

  FIG. 5 is a block diagram illustrating a main configuration of the vertical block horizontal motion vector calculation unit 31.

  The vertical block horizontal motion vector calculation unit 31 adds three input terminals 311 to 313, an n-value calculator 314, and the first vertical block projection data converted to n-value by the n-value generator 314. An adder 315 to perform, a horizontal motion vector addition memory 316 as a buffer memory for sequentially storing data added by the adder 315, a peak detector 317, and an output terminal 318 are provided.

  The first vertical block projection data output from the vertical block projection data reading unit 30 is input to the input terminal 311, and the first vertical block projection data maximum value storage unit 24 is input to the input terminal 312. The output second threshold value is input. In addition, the first threshold value intersection information output from the first threshold value intersection search unit 29 is input to the input terminal 313.

  The n-value converter 314 converts the first vertical block projection data input from the input terminal 311 into n-values (for example, ternarization) based on the second threshold value input from the input terminal 312.

  The adder 315 adds the first vertical block projection data n-valued by the n-value calculator 314 and the data read from the horizontal motion vector addition memory 316 around the first threshold intersection. Then, the addition result is stored in the horizontal motion vector addition memory 316 again. When the addition for all the first threshold intersections related to the vertical block is completed by repeating such an addition operation, the addition data related to the vertical block is output to the peak detector 317.

  The peak detector 317 detects a peak position (horizontal motion vector described later) in the added data output from the adder 315. The peak position detected by the peak detector 317 is input to the horizontal direction motion vector determination unit 32 via the output terminal 318.

Next, the configuration of the vertical motion vector detection unit 4 will be described.
FIG. 6 is a block diagram showing a main configuration of the vertical motion vector detection unit 4.

  The vertical motion vector detection unit 4 includes an input terminal 40, a horizontal image segmentation unit 41, a horizontal edge extraction filtering unit 42, a horizontal block projection unit 43, and a first horizontal block projection line memory 44. The first horizontal block projection data maximum value storage unit 45 and the horizontal block projection data smoothing unit 53 are provided. The vertical motion vector detection unit 4 includes a second horizontal block projection line memory 46, a second horizontal block projection data maximum value storage unit 47, a third threshold intersection search unit 48, and a horizontal direction. A block projection data reading unit 49, a horizontal block vertical motion vector calculation unit 50, a vertical motion vector determination unit 51, and an output terminal 52 are provided. The function of each part of the above-described vertical motion vector detection unit 4 will be briefly described (specific operations will be described in detail later).

  The horizontal image dividing unit 41 divides the frame image input to the input terminal 40 in the horizontal direction and outputs a block divided in the horizontal direction (hereinafter also referred to as “horizontal block”).

  The horizontal direction edge extraction filtering unit 42 performs a filtering process for performing edge extraction for each block divided by the horizontal direction image dividing unit 41.

  The horizontal block projection unit 43 projects the edge-enhanced horizontal block output from the horizontal edge extraction filtering unit 42 in the horizontal direction and outputs projection data for each horizontal block.

  The first horizontal block projection line memory 44 converts the horizontal block projection data output from the horizontal block projection unit 43 into the horizontal block projection data of the current frame (hereinafter referred to as “first horizontal block projection data”). As “)”.

  The horizontal block projection data smoothing unit 53 reads the horizontal block projection data stored in the first horizontal block projection line memory 44, removes the fluctuation noise superimposed on the horizontal block projection data, and performs smoothing. The converted projection data (hereinafter referred to as smoothed projection data) is written back to the first horizontal block projection line memory 44. Accordingly, the “projection data” in the subsequent processing is “smoothed projection data (smoothed projection data)”, including the case where it is not particularly distinguished and described, and “first horizontal block projection”. “Data” is also the smoothed projection data.

  The first horizontal block projection data maximum value storage unit 45 outputs the maximum value of the projection data of the horizontal block of the current frame output from the horizontal block projection unit 43 (hereinafter, “maximum horizontal block projection data of the current frame”). Value ”or“ first horizontal block projection data maximum value ”).

  The second horizontal block projection line memory 46 converts the horizontal block projection data sent from the first horizontal block projection line memory 44 into projection data (hereinafter referred to as “second horizontal block projection data” of the horizontal block of the previous frame). It is also saved as “direction block projection data”.

  The second horizontal block projection data maximum value storage unit 47 uses the first horizontal block projection data maximum value output from the first horizontal block projection data maximum value storage unit 45 as the “second ”Horizontal direction block projection data maximum value” (also referred to as “horizontal block projection data maximum value of previous frame”). Further, the second horizontal block projection data maximum value storage unit 47 calculates a third threshold value and a fourth threshold value, which will be described later, based on the second horizontal block projection data maximum value.

  The third threshold intersection search unit 48 includes a second horizontal block projection data related to the previous frame stored in the second horizontal block projection line memory 46, and a second horizontal block projection data maximum value storage unit. The point where the third threshold value calculated at 47 intersects is obtained, and information on this intersection (hereinafter also referred to as “third threshold value intersection”) is output.

  The horizontal block projection data reading unit 49 has a first horizontal direction within a predetermined range (motion vector detection range) before and after the third threshold intersection obtained by the third threshold intersection search unit 48. Block projection data is read from the first horizontal block projection line memory 44.

  The horizontal block vertical motion vector calculation unit 50 uses the fourth threshold value output from the second horizontal block projection data maximum value storage unit 47, and is read by the horizontal block projection data read unit 49. Each horizontal block projection data of 1 is converted into n-values (n is an integer of 2 or more) to reduce the number of bits, and the n-valued projection data is added for each distance from the third threshold intersection. .

  The vertical motion vector determination unit 51 determines the vertical motion vector of the image based on the output from the horizontal block vertical motion vector calculation unit 50. The vertical motion vector of the image determined here is output from the output terminal 52.

FIG. 7 is a block diagram showing a main configuration of the horizontal block projection unit 43.
The horizontal block projection unit 43 includes an input terminal 431, an adder 432 for adding one horizontal line data of the horizontal block, and a horizontal projection temporary as a buffer memory for sequentially storing the data added by the adder 432. A storage memory 433 and an output terminal 434 are provided.

  The input terminal 431 receives an image in which horizontal edges are emphasized by the horizontal edge extraction filtering unit 42. The image input to the input terminal 431 is projected in the horizontal direction by the adder 432 and the horizontal projection temporary storage memory 433 for each horizontal block. Specifically, with respect to the pixels of one horizontal line in the horizontal block, the addition result that is added by the adder 432 and stored in the horizontal projection temporary storage memory 433 up to the previous pixel and the pixel input from the input terminal 431. Are added by the adder 432, and the addition result is returned to the horizontal projection temporary storage memory 433 and stored. When the addition of one horizontal line in the horizontal block is completed by repeating such an addition operation, the addition data of all the pixels in one horizontal line of the horizontal block, that is, the projection data in the horizontal direction is output from the output terminal 434. Will be. Hereinafter, similarly, the projection data in the horizontal direction of the next horizontal block is processed, and when the processing of one horizontal line of all the horizontal blocks is completed, the processing of the next one horizontal line is performed.

  FIG. 8 is a block diagram showing a main configuration of the horizontal block vertical motion vector calculation unit 50.

  The horizontal block vertical motion vector calculation unit 50 adds three input terminals 501 to 503, an n-value converter 504, and the first horizontal block projection data converted to n-value by the n-value generator 504. An adder 505 to be performed, a vertical motion vector addition memory 506 as a buffer memory for sequentially storing data added by the adder 505, a peak detector 507, and an output terminal 508 are provided.

  The first horizontal block projection data output from the horizontal block projection data reading unit 49 is input to the input terminal 501, and the second horizontal block projection data maximum value storage unit 47 is input to the input terminal 502. The output fourth threshold value is input. In addition, the information about the third threshold value intersection output from the third threshold value intersection search unit 48 is input to the input terminal 503.

  The n-value converter 504 converts the first horizontal block projection data input from the input terminal 501 into n-values (for example, ternarization) based on the fourth threshold value input from the input terminal 502.

  The adder 505 adds the first horizontal block projection data converted to the n-value by the n-value generator 504 and the data read from the vertical motion vector addition memory 506 around the third threshold value intersection. Then, the addition result is stored in the vertical motion vector addition memory 506 again. When the addition for all the third threshold intersections regarding the horizontal block is completed by repeating such an addition operation, the addition data regarding the horizontal block is output to the peak detector 507.

  The peak detector 507 detects a peak position (vertical motion vector described later) in the added data output from the adder 505. The peak position detected by the peak detector 507 is input to the vertical direction motion vector determination unit 51 via the output terminal 508.

  The operation of the motion vector detection apparatus 1 having the above configuration will be described below.

<Operation of Motion Vector Detection Device 1>
<Operation of Horizontal Motion Vector Detection Unit 2>
First, the operation of the horizontal motion vector detection unit 2 that detects the horizontal motion vector of an image in the motion vector detection device 1 will be described.

  As shown in FIG. 2, when a frame image read out by sequentially repeating pixel scanning of the horizontal line in the vertical direction is input to the input terminal 20 of the horizontal direction motion vector detecting unit 2 shown in FIG. 21 is divided into blocks in the vertical direction. That is, in the vertical image dividing unit 21, a plurality of image regions (vertical blocks) obtained by dividing the frame image in the vertical direction are set. Thereby, in the subsequent processing, processing and management are performed for each vertical block.

  In the vertical image division unit 21, for example, as shown in FIG. 9A, image data of 640 pixels × 480 pixels is converted into seven vertical blocks vb0 to vb0 having a width of 64 pixels (division width) in the vertical direction. Divided into vb6. Although the image data vb7 at the lowest stage in the vertical direction is not used in the detection of the horizontal motion vector, the seven vertical blocks vb0 to vb6 occupying most of the frame image are used for the detection of the horizontal motion vector. Therefore, it is considered that there is no particular problem in terms of detection accuracy. Here, it is not essential to use the division width and the number of divisions shown in FIG.

  The image divided by the vertical image dividing unit 21 into seven vertical blocks vb0 to vb6 as shown in FIG. 9A is extracted, in other words, edge components extending in the vertical direction by the vertical edge extraction filtering unit 22. For example, a filtering process for emphasizing an image portion that changes sharply in the horizontal direction is performed.

  As a filter used in this filtering processing, a (1, -1,) 2-tap filter that simply takes a difference from a pixel adjacent in the horizontal direction, or (-1, 2, -1) corresponding to a second order differential. It is possible to use a 3-tap filter. Note that it is not essential to use such a filter, and any filter may be used as long as the output value increases in an image portion where the luminance change in the horizontal direction increases.

  The image data in which the vertical edge is emphasized for each vertical block (image region) vb0 to vb6 by the vertical edge extraction filtering unit 22 is input to the vertical block projection unit 23, and the vertical block projection unit 23 Projected vertically. This projection can reduce the noise component in the vertical direction (between lines) and can enhance the edge component in the vertical direction, so that the vertical edge corresponding to the feature point can be emphasized to improve the motion vector detection accuracy. It will be. The operation of the vertical block projection unit 23 will be described with reference to the conceptual diagram of FIG.

  In the vertical block projection unit 23, when the input of the vertical blocks vb0 to vb6 shown in FIG. 9A is completed, that is, the final pixels of the final line in the vertical blocks vb0 to vb6 with edge enhancement are input. At this point, all array elements of the vertical direction block projection data vn0 to vn6 (FIG. 9B) having the data array Mv for one horizontal line are generated. The data array Mv has array elements of the total number of pixels on the horizontal line (for example, 640). However, in a 2-tap filter such as (1, -1), the effective element is the total number of pixels of the horizontal line − In a 3-tap filter such as 1 (for example, 639) and (−1, 2, −1), the effective elements are the total number of pixels in the horizontal line−2 (for example, 638).

  Specifically, as shown in FIG. 4, the image data of the vertical blocks vb0 to vb6 (image data subjected to vertical edge emphasis) input via the input terminal 231 are sequentially added to the adder 232. Is input.

  In the adder 232, first, the first horizontal line of the input vertical block is written into the vertical projection temporary storage memory 233 without reading the data in the vertical projection temporary storage memory 233. Next, the adder 232 reads the addition result up to the previous line stored in the vertical projection temporary storage memory 233, performs addition with the horizontal one line of the vertical block input from the input terminal 231, and The addition result is written back to the vertical projection temporary storage memory 233.

  When the final horizontal line in the vertical block is input to the adder 232, the addition result up to the previous line stored in the vertical projection temporary storage memory 233 is read and the vertical block input from the input terminal 231 is read. And the addition result, that is, the data obtained by adding all the horizontal lines of the vertical block is used as projection data of the vertical block, and the first vertical block projection data maximum from the output terminal 234 is obtained. The data is output to the value storage unit 24 and the bit number reduction unit 25.

  The adder 232 temporarily stores the projection data obtained by adding all the horizontal lines of the vertical block in the vertical projection temporary storage memory 233, and when the first line of the next vertical block is input, the vertical projection is performed. The projection data in the temporary storage memory 233 may be read and output from the output terminal 234.

  The first vertical block projection data maximum value storage unit 24 calculates the maximum value regarding the projection data of the vertical block output from the vertical block projection unit 23, and the first vertical block projection data maximum value (current Frame vertical block projection data maximum value). That is, the first vertical block projection data maximum value storage unit 24 obtains the maximum value in the array element of the data array Mv (FIG. 9B) relating to the projection data of the current frame (subsequent frame), and calculates this maximum value. Remember.

  Further, the first vertical block projection data maximum value storage unit 24 calculates the second threshold value based on the calculated vertical block projection data maximum value of the current frame. For example, if the maximum value of the vertical direction block projection data of the current frame is P1max, the second threshold value α2 is calculated by the following equation (1).

α2 = P1max × k1 (1)
Here, k1 is a predetermined coefficient, and 0 <k1 <1.

  Note that the calculation of the second threshold value α2 is not limited to the above equation (1), and it is sufficient that the second threshold value α2 increases as the vertical block projection data maximum value P1max of the current frame increases. Alternatively, a conversion table may be used.

  In the bit number reduction unit 25, the projection data is based on the vertical block projection data maximum value (first vertical block projection data maximum value) of the current frame input from the first vertical block projection data maximum value storage unit 24. Determine the effective bit range of. Then, the upper invalid bit and the lower invalid bit set based on the valid bit range are reduced from the vertical block projection data input from the vertical block projection unit 23, and the vertical direction of the current frame configured only by the valid bits The block projection data is output to the first vertical block projection line memory 26.

  The first vertical block projection line memory 26 stores the vertical block projection data of the current frame whose number of bits has been reduced by the bit number reduction unit 25 as the first vertical block projection data for each vertical block. . That is, in the first vertical block projection line memory 26, the bit number reduction unit 25 based on the first vertical block projection data maximum value obtained by the first vertical block projection data maximum value storage unit 24. Projection data of the current frame (subsequent frame) in which the number of bits (data length of each array element) is reduced is stored.

  The vertical block projection data stored in the first vertical block projection line memory 26 is read out to the vertical block projection data smoothing unit 34 when necessary data is prepared, and smoothed. Thereafter, the data is written back to the first vertical block projection line memory 26. For example, when the smoothing process is performed using three consecutive points in the horizontal direction of Pa, Pb, and Pc, “when all necessary data is collected” means when three points are aligned in the horizontal direction. At this time, it is possible to smooth the center point Pb. Details of the smoothing process in the vertical block projection data smoothing unit 34 will be described later.

  The first vertical block projection line memory 26 reads the vertical block projection data (smoothed data) of the previous frame at a timing when new first vertical block projection data relating to the current frame is input. The second vertical block projection line memory 27 is provided. The second vertical block projection line memory 27 stores the vertical block projection data of the previous frame read from the first vertical block projection line memory 26 as second vertical block projection data.

  Similarly, the first vertical block projection data maximum value storage unit 24 stores the first vertical direction already stored at the timing when the new first vertical block projection data maximum value for the current frame is updated. The block projection data maximum value is read and provided to the second vertical block projection data maximum value storage unit 28. In the second vertical block projection data maximum value storage unit 28, the first vertical block projection data maximum value read from the first vertical block projection data maximum value storage unit 24 is the second vertical block projection data maximum value related to the previous frame. Is stored as the maximum vertical block projection data value (vertical block projection data maximum value of the previous frame).

  In other words, the vertical block projection data maximum value of the current frame stored in the first vertical block projection data maximum value storage unit 24 is the second vertical block projection data maximum value storage unit in the next frame. 28 is stored as the vertical block projection data maximum value of the previous frame.

  The second vertical block projection data maximum value storage unit 28 sets a first threshold (predetermined constant value) based on the second vertical block projection data maximum value for the previous frame. For example, if the maximum value of vertical block projection data for the previous frame is P2max, the first threshold value α1 is calculated by the following equation (2).

α1 = P2max × k2 (2)
Here, k2 is a predetermined coefficient, and 0 <k2 <1.

  Note that the calculation of the first threshold value α1 is not limited to the above equation (2), and it is sufficient if the first threshold value α1 increases as the vertical block projection data maximum value P2max of the previous frame increases. Alternatively, a conversion table may be used.

  The first threshold intersection search unit 29 outputs the second vertical block projection data read from the second vertical block projection line memory 27 and the second vertical block projection data maximum value storage unit 28. An intersection with the first threshold value (first threshold value intersection) is searched in the horizontal direction. The information on the first threshold intersection obtained by the first threshold intersection searching unit 29 is output to the vertical block projection data reading unit 30 and the vertical block horizontal motion vector calculating unit 31.

  The vertical block projection data read unit 30 outputs the (first) vertical block projection of the current frame corresponding to the motion vector detection range centered on the first threshold intersection output from the first threshold intersection search unit 29. Data is read from the first vertical block projection line memory 26. That is, the first threshold intersection is A (i) (where i = 1, 2,... P, p is the total number of detected first threshold intersections), and the motion vector detection range is the first threshold. Assuming the range from (−V) to (+ V) (where V is a positive integer) centered on the intersection, the projection data read from the first vertical block projection line memory 26 is (A (i ) −V) to (A (i) + V), which is partial data of the first vertical block projection data.

  The first vertical block projection data read from the first vertical block projection line memory 26 by the vertical block projection data reading unit 30 is output to the vertical block horizontal motion vector calculation unit 31.

  The first vertical block projection data output from the vertical block projection data reading unit 30 is input to the n-value converter 314 via the input terminal 311 of the vertical block horizontal motion vector calculation unit 31 shown in FIG. Is done. The n-value converter 314 outputs the first vertical block projection data based on the second threshold value output from the first vertical block projection data maximum value storage unit 24 and input via the input terminal 312. n-valued. In other words, the n-value converter 314 converts the data length of each array element related to the projection data of the current frame extracted by the vertical block projection data reading unit 30 to the data length of each array element, for example, a ternization process. And compress. The processing of the n-value converter 314 will be described with reference to FIGS. 10 (a) to 10 (c).

  FIG. 10A to FIG. 10C are diagrams for explaining the n-value conversion processing in the n-value conversion unit 314. Note that FIGS. 10A to 10C show an example of the ternary processing in which n = 3.

  If the first vertical block projection data input to the n-value converter 314 is D, and the second threshold value input via the input terminal 312 is α2 (α2 is a positive integer), the n-value generator 314 Then, when D <(− α2), (−1) is output, when (−α2) ≦ D ≦ α2, 0 is output, and when D> α2, 1 is output. Done. That is, in the ternarization process in the n-valued unit 314, regarding the second threshold value α2 (α2> 0) set based on the maximum value of the array element of the data array Mv in the projection data of the current frame (subsequent frame), 3 when the value of the array element relating to the projection data of the current frame extracted by the vertical block projection data reading unit 30 is smaller than (−α2), (−α2) or more and α2 or less, and 3 Trinization is performed in stages.

  With regard to the n-value conversion processing by the n-value generator 314, the partial waveform of the first vertical block projection data of the current frame output from the vertical block projection data reading unit 30 is compared with the first threshold intersection. Therefore, it is important to ensure bit accuracy that can identify peaks and valleys in the waveform of the first vertical block projection data. It is.

  On the other hand, when the value of the first vertical block projection data output from the vertical block projection data reading unit 30 is adopted without reducing the number of bits in the n-value converter 314, the first There is a possibility that a small amplitude peak or valley corresponding to the vertical block projection data D> α2 or a small amplitude peak corresponding to D <(− α2) or a small amplitude peak corresponding to D <(− α2) may be buried. There is inappropriate. Furthermore, since the small peaks and valleys corresponding to (−α2) ≦ D ≦ α2 are easily affected by noise and have a large number, the peaks and valleys having an appropriate amplitude in which the characteristics of the image appear are buried.

  Therefore, the n-value generator 314 improves the motion vector detection accuracy by performing an appropriate n-value process on the first vertical block projection data.

  The operations of the first threshold intersection searching unit 29, the vertical block projection data reading unit 30, and the vertical block horizontal motion vector calculating unit 31 described above will be described with reference to FIGS. 11 (a) to 11 (c). Specific explanation will be given. Note that the horizontal axes of FIGS. 11A and 11B indicate the positions of the array elements of the projection data array Mv (FIG. 9B). In FIG. 11A, each of the first threshold intersections A (1) to A (8) is indicated by a circle.

  The first threshold value intersection search unit 29 outputs the second vertical block projection data maximum value storage unit 28 in the waveform of the second vertical block projection data W2 for the previous frame as shown in FIG. First threshold value intersections A (1) to A (8) that intersect the first threshold value α1 thus obtained are obtained. That is, the first threshold value intersection search unit 29 relates to the projection data obtained by the vertical block projection unit 23 with respect to the previous frame, and the array element values in the order of the elements in the projection data array Mv (FIG. 9B). The position of the array element at each first threshold crossing point where the waveform W2 and the linear function line having the array element value of the first threshold value (predetermined constant value) α1 intersect is specified.

  Next, the vertical block projection data reading unit 30 performs the first vertical detection for a predetermined motion vector detection range centered on each of the first threshold intersections A (1) to A (8) shown in FIG. The vertical block projection data of the current frame is read from the direction block projection line memory 26. That is, the vertical block projection data reading unit 30 outputs from the vertical block projection unit 23 a data array in a predetermined range centered on the position of the array element of the projection data (data array) at each first threshold intersection. Extracted from the projection data of the current frame (subsequent frame) whose bit number has been reduced by the bit number reduction unit 25. For example, for the first threshold value intersection A (7), as shown in FIG. 11B, the first threshold value intersection A (7) is centered on (A (7) -V) to (A (7) + V). The first vertical block projection data W1 corresponding to the range up to (the waveform portion within the rectangular broken line) is read out.

  Then, the first vertical block projection data read by the vertical block projection data reading unit 30 uses the second threshold value α2 output from the first vertical block projection data maximum value storage unit 24. The n-value unit 314 of the vertical block horizontal direction motion vector calculation unit 31 performs n-value processing. For example, for the first threshold intersection A (7), the first threshold is based on the magnitude relationship of the first vertical block projection data W1 with respect to the second threshold α2 and (−α2) shown in FIG. With respect to the range from (A (7) −V) to (A (7) + V) centering on the intersection A (7) (the waveform portion in the rectangular broken line), the first vertical as shown in FIG. The direction block projection data is ternarized. As a result, the vertical block projection data of the current frame is represented by “−1”, “0”, or “1”.

  In the n-value converter 314, n-value conversion processing as shown in FIG. 11C is performed on all the first threshold value intersections output from the first threshold value intersection search unit 29.

  Next, in the adder 315 (FIG. 5), the n-value generator 314 converts the n-value into the n-value centering on the first threshold-value intersection output from the first threshold-value intersection search unit 29 via the input terminal 313. The first vertical block projection data is added to the previous n-valued first vertical block projection data read from the horizontal motion vector addition memory 316, and the addition result Are again stored in the horizontal motion vector addition memory 316. Then, the adder 315 adds the n-valued first vertical block projection data relating to the last one of all the first threshold intersections detected by the first threshold intersection search unit 29 to obtain one When the addition process for the vertical block is terminated, the addition result is output to the peak detector 317. In addition, after the addition process in the adder 315 is completed, the peak detection unit 317 may read the addition result from the horizontal motion vector addition memory 316 that stores the addition results regarding all the first threshold intersections. good.

  The peak detector 317 includes all the first threshold intersections detected by the first threshold intersection search unit 29 in a motion vector detection range (the above-described ± V range) centered on each first threshold intersection. Data obtained by adding the corresponding n-valued first vertical block projection data (hereinafter also referred to as “vertical block n-valued data”) is input. The value is detected by peak detector 317. The peak position of the vertical block n-valued data detected by the peak detector 317 becomes a horizontal motion vector obtained from the vertical block, and is output to the horizontal motion determination unit 32 via the output terminal 318. .

  The operations of the adder 315 and the peak detector 317 described above will be specifically described with reference to FIGS. 12 (a) and 12 (b). In FIG. 12A, in the motion vector detection range (± V range) centering on the first threshold intersections A (1) to A (8) shown in FIG. The block projection data is conceptually shown as three values.

  In the adder 315, when ternary data around the first threshold intersections A (1) to A (8) shown in FIG. 12 (a) are sequentially input, these data are added, and FIG. Vertical block n-valued data as shown in b) is generated. Specifically, in the adder 315, the vertical block projection data reading unit 30 extracts the vertical block projection data (data array Mv (FIG. 9B)) and the n-value unit 314 compresses the data length. For each data array of the motion vector detection range (predetermined range) related to the frame, values of array elements having the same relative position with respect to the position of the array element at each first threshold intersection are added.

  When the vertical block n-valued data generated by the adder 315 is input to the peak detector 317, the horizontal peak position hv shown in FIG. 12B is detected as the horizontal motion vector of the vertical block. Is done. That is, the peak detector 317 detects a motion vector in the horizontal direction of the frame image based on the addition result added by the adder 315.

  Next, the smoothing process in the vertical direction block projection data smoothing unit 34 will be described in detail with reference to FIGS. The vertical block projection data stored in the first vertical block projection line memory 26 is read out to the vertical block projection data smoothing unit 34 and subjected to smoothing processing, and then the first vertical block projection is performed. It is written back to the line memory 26. 16 to 19, an example in which smoothing processing is performed using three horizontal array elements Pa, Pb, and Pc in the horizontal direction will be described. Note that a, b, and c are values at points Pa, Pb, and Pc, respectively.

  FIG. 16 shows a smoothing process when the variation pattern of the projection data is “convex downward, a> 0, b ≧ 0, c> 0”.

  For detection of motion vectors, it is important to gradually increase and decrease by a plurality of consecutive points in the projection data, and unevenness due to a small number of points causes a drop in accuracy due to the relationship with the threshold value, so it is desirable to remove them. Therefore, in the pattern shown in FIG. 16, the downwardly convex shape is eliminated by changing the value b of the central point Pb to the average value of the values a and c.

  Similarly, FIG. 17 shows a smoothing process when the variation pattern of the projection data is “convex upward, a <0, b ≦ 0, c <0”. Also in this case, by changing the value b of the center point Pb to the average value of the values a and c, the shape that is convex upward is eliminated.

  On the other hand, FIG. 18 shows a smoothing process when the variation pattern of the projection data is “convex downward, a> 0, b <0, c> 0”.

  In this case, b <−α2, and if the processing proceeds as it is, the value b is based on the second threshold value −α2 during the n-value conversion processing (in this example, ternarization) by the n-value converter 314. When the values of the front and rear points Pa and Pc are 1, the values of the points Pa, Pb, and Pc are 1, -1, 1 and vibrate. In the adder 315, a large vibration (amplitude) is generated by adding each of the first threshold intersections as a center. Originally, one large peak should appear at the position indicating the motion vector position. A large value is also shown in a portion other than the original peak position.

  However, when the smoothing process similar to that shown in FIG. 16 or 17 is executed for the variation pattern shown in FIG. 18, there is a problem that the feature of the projection data cannot be accurately reflected in the n-value conversion process.

  That is, as shown in FIG. 18, when the value b of the point Pb is smaller than the second threshold value −α2 and the point Pa and the value of the point Pc are larger than 0, the point Pa is the point Pa It is considered that a part of the former fluctuation pattern constituted by the earlier points Pq, Pr,... (<Pr <Pq <Pa) is constituted and the point Pb also constitutes the former fluctuation pattern. It is done.

  Further, the point Pc constitutes a part of the subsequent fluctuation pattern constituted by the points Ps, Pt,... (Pc <Ps <Pt <...) After the point Pc, and the point Pb is also in the subsequent stage. It is conceivable to construct a variation pattern.

  In such a case, if the smoothing process is performed in which the value b of the point Pb is simply set to the average value of the value a and the value c, the characteristics of the fluctuation pattern in the preceding stage and the fluctuation pattern in the subsequent stage are modified. Thus, the features of both patterns cannot be accurately reflected in the n-value conversion process.

  Therefore, in the variation pattern as shown in FIG. 18, a smoothing process is performed in which the value b of the point Pb is changed to a value (−β) that is compressed to “0” when the n-value conversion process is performed. . Here, a value compressed to “0” is referred to as a compressed value, and the compressed value is set to a fixed value that is greater than the second threshold value −α2 and equal to or less than 0. By this processing, it is possible to accurately reflect the features of both patterns in the n-value conversion processing without impairing the features of the preceding and following variation patterns.

  FIG. 19 shows the smoothing process when the variation pattern of the projection data is “convex upward, a <0, b> 0, c <0”.

  In this case, b> α2, and if the processing proceeds as it is, the value b is set to 1 based on the second threshold value α2 during the n-value conversion processing (in this example, ternarization) by the n-value converter 314. When the values of the front and rear points Pa and Pc are -1, the values of the points Pa, Pb, and Pc become -1, 1, -1, and vibrate. In the adder 315, a large vibration (amplitude) is generated by adding each of the first threshold intersections as a center. Originally, one large peak should appear at the position indicating the motion vector position. A large value is also shown in a portion other than the original peak position.

  However, when the smoothing process similar to that shown in FIG. 16 or 17 is executed for the variation pattern shown in FIG.

  That is, as shown in FIG. 19, when the value b of the point Pb is larger than the second threshold value α2 and the values of the points Pa and Pc are smaller than 0, the point Pa is larger than the point Pa. It is conceivable that a part of the former fluctuation pattern constituted by the previous points Pq, Pr,... (. .

  Further, the point Pc constitutes a part of the subsequent fluctuation pattern constituted by the points Ps, Pt,... (Pc <Ps <Pt <...) After the point Pc, and the point Pb is also in the subsequent stage. It is conceivable to construct a variation pattern.

  In such a case, if the smoothing process is performed in which the value b of the point Pb is simply set to the average value of the value a and the value c, the characteristics of the fluctuation pattern in the preceding stage and the fluctuation pattern in the subsequent stage are modified. Thus, the features of both patterns cannot be accurately reflected in the n-value conversion process.

  Therefore, in the variation pattern as shown in FIG. 19, a smoothing process is performed in which the value b of the point Pb is changed to a value (β) that is compressed to “0” when the n-value conversion process is performed. Here, a value compressed to “0” is referred to as a compressed value, and the compressed value is set to a fixed value that is smaller than the second threshold value α2 and equal to or greater than zero. By this processing, it is possible to accurately reflect the features of both patterns in the n-value conversion processing without impairing the features of the preceding and following variation patterns.

  Note that when the compressed value is set to 0, if the compression value at the point Pb is used for smoothing the subsequent fluctuation pattern, the characteristics of the subsequent fluctuation pattern will slightly change. The compressed value may be set to 0.

  Next, the effect of the smoothing process in the vertical direction block projection data smoothing unit 34 will be described using specific examples of the variation pattern of the projection data shown in FIGS.

  FIG. 20 shows a part of the waveform of the vertical block projection data when the smoothing process is not performed. As shown in FIG. 20, the waveform is a waveform that vibrates up and down at each adjacent point.

  The cause of this vibration is not only due to various signal processing in the previous circuit, but also in the vertical direction, for example, when the signal level slightly varies from line to line in the camera signal etc. It is conceivable that the addition (emphasis) is made remarkable each time processing is performed.

  FIG. 21 is a diagram schematically showing a state in which threshold values α and −α are set for the projection data shown in FIG. 20 when the ternarization process is performed. The threshold values are increased and decreased at each point in a circled area. I know that Therefore, if the ternarization process is performed as it is, the ternarization result is greatly affected.

  Here, FIG. 22 shows the result of applying the smoothing processing described with reference to FIGS. 16 to 19 to the projection data shown in FIG.

  In FIG. 22, while maintaining the overall characteristics of the waveform of the original vertical block projection data, the waveform superimposed thereon is removed.

  For example, the circled regions R11 to R15 are regions obtained by smoothing the data included in the region where the threshold value is raised or lowered in FIG. 21, and these are the data of the adjacent data described with reference to FIGS. A smoothing process using an average value is performed.

  Further, the region R10 is large such that the value of the point 23 is larger than the threshold value α and the values of the point 22 and the point 24 are smaller than 0, as indicated by the points 22, 23 and 24 in FIG. These are areas having fluctuations, and these are values that are compressed to “0” when the value of the central point among the three points described with reference to FIGS. 18 and 19 is subjected to n-value conversion processing. The smoothing process of changing to is performed.

  By performing the smoothing process as described above, vertical block projection data that does not significantly affect the ternary value due to the vibration in the vicinity of the intersection with the threshold value can be obtained.

  FIG. 23 shows all first threshold value intersections detected by the first threshold value intersection search unit 29 by the adder 315 after the first vertical block projection data is n-valued by the n-value generator 314. FIG. 10 illustrates an example of vertical block n-valued data added in a motion vector detection range (range of ± V described above) centered on each first threshold intersection. Here, V is 31 and the horizontal scale 32 indicates the position of the horizontal motion vector 0.

  The data indicated by the filled diamonds is “no smoothing” data, and the data indicated by the filled squares is “smoothed” data.

  Of the waveforms represented by both data, the waveform of “no smoothing” is oscillating greatly, and is convex upward and downward in various places. On the other hand, the waveform with “smoothing process” is a waveform that does not have extra unevenness while retaining the characteristics of the original waveform.

  In FIG. 23, both the “no smoothing” waveform and the “smoothing” waveform have a maximum peak at the position of the scale 30, so the peak detector 317 uses the maximum peak of the entire motion vector detection range. When the position is determined as a horizontal motion vector, no erroneous determination occurs.

  However, for example, by calculating the peak position for each “upward convex block” whose value changes from negative to positive to negative, and evaluating the magnitude of the peak value for each “block”, the movement of the periodic image When a reliability evaluation is performed to eliminate the vector calculation result and a method that further increases the calculation accuracy is adopted, the waveform of “no smoothing process” in this example is evaluated as “no reliability”. For the waveform “with smoothing”, a correct horizontal motion vector can be calculated.

  That is, in the waveform of “without smoothing”, in addition to the scale 30, in addition to the scale 28 and the scale 32, “lumps” are formed at a large number of positions, and each has an added value of 40, 30, and 26. is doing. Originally, it should have only one large peak other than a small peak. Thus, in the case of having a large peak value equivalent at many positions, the result is not reliable.

  On the other hand, the waveform of “with smoothing process” has a peak of the addition value 21 at the scale 30, the addition value 5 at the scale 42, and the addition value 3 at the scale 5, but the peak of the scale 30 can be determined as the only large peak. Therefore, the scale 30, that is, the vector-2 can be determined as a horizontal motion vector.

  The horizontal motion vector calculated for each vertical block by the peak detector 317 is sequentially input to the horizontal motion vector determination unit 32 via the output terminal 318.

  In the horizontal direction motion vector determination unit 32, the horizontal direction motion vector of the entire image is determined based on the horizontal direction motion vector of each vertical direction block sequentially output from the vertical direction block horizontal direction motion vector calculation unit 31. Specifically, a histogram is created for the horizontal motion vector of each vertical block, and the horizontal motion vector having the largest histogram value is used as the horizontal motion vector for the entire screen.

  As described above, the vertical block projection data smoothing unit 34 can remove the fluctuation noise superimposed on the vertical block projection data without impairing the large tendency (feature) of the vertical block projection data. Since an extra peak is not generated as a result of addition by the adder 315, erroneous determination and a decrease in reliability in the peak detector 317 can be reduced. As a result, a reliable vertical block horizontal motion vector can be calculated.

<Operation of Vertical Motion Vector Detection Unit 4>
Next, the operation of the vertical motion vector detection unit 4 that detects the vertical motion vector of the image in the motion vector detection device 1 will be described.

  As shown in FIG. 2, when a frame image read out by repeating the pixel scanning of the horizontal line in order in the vertical direction is input to the input terminal 40 of the vertical direction motion vector detection unit 4 shown in FIG. In 41, the blocks are horizontally divided. That is, the horizontal image dividing unit 41 sets a plurality of image regions (horizontal blocks) obtained by dividing the frame image in the horizontal direction. As a result, in the subsequent processing, processing and management are performed for each horizontal block.

  In the horizontal image dividing unit 41, for example, as shown in FIG. 13A, image data of 640 pixels × 480 pixels is divided into 10 vertical blocks hb0 to hb0 having a width (divided width) of 64 pixels in the horizontal direction. Divided into hb9. Note that the division width and the number of divisions shown in FIG. 13A are not essential, and the number of divisions may be set to 1, for example.

  The image divided by the horizontal image dividing unit 41 into 10 horizontal blocks hb0 to hb9 as shown in FIG. 13A is extracted, in other words, edge components extending in the horizontal direction by the horizontal edge extraction filtering unit 42. For example, a filtering process for emphasizing an image portion that changes sharply in the vertical direction is performed.

  As a filter used in this filtering processing, a (1, -1,) 2-tap filter that simply takes a difference from a pixel adjacent in the horizontal direction, or (-1, 2, -1) corresponding to a second order differential. It is possible to use a 3-tap filter. It is not essential to use such a filter, and any filter may be used as long as the output value increases in an image portion where the luminance change is large in the vertical direction.

  The image data in which the horizontal edge is enhanced for each horizontal block (image region) hb0 to hb9 by the horizontal edge extraction filtering unit 42 is input to the horizontal block projection unit 43, and the horizontal block projection unit 43 outputs the image data. Projected horizontally. This projection can reduce the noise component in the horizontal direction (between lines) and can further enhance the edge component in the horizontal direction, so that the horizontal edge corresponding to the feature point can be emphasized and the motion vector detection accuracy can be improved. It will be. The operation of the horizontal block projection unit 43 will be described with reference to the conceptual diagram of FIG.

  In the horizontal block projection unit 43, when the input of one horizontal line in each of the horizontal blocks hb0 to hb9 shown in FIG. 13A is completed, the horizontal projection data hn0 having the data array Mh for one vertical line. One array element of hn9 (FIG. 13B) is generated. Therefore, all the array elements are arranged when the input of the final horizontal line of each horizontal block hb0 to hb9 is completed. The data array Mh has array elements of the total number of pixels of the vertical line (for example, 480), but in a 2-tap filter such as (1, -1), the effective element is the total number of pixels of the vertical line − In a 3-tap filter such as 1 (for example, 479) and (-1, 2, -1), the effective elements are the total number of pixels in the vertical line -2 (for example, 478).

  Specifically, as shown in FIG. 7, the image data (image data subjected to edge enhancement in the horizontal direction) of the horizontal blocks hb0 to hb9 input via the input terminal 431 is input to the adder 432. Is done. The adder 432 first writes the data in the horizontal projection temporary storage memory 433 without reading the data in the horizontal projection temporary storage memory 433 at the head of one horizontal line of the input horizontal block. Next, the adder 432 reads the addition result up to the previous pixel of one horizontal line stored in the horizontal projection temporary storage memory 433, and the current pixel of one horizontal line of the horizontal block input from the input terminal 431. And the addition result is written back to the horizontal projection temporary storage memory 433. When the final pixel of one horizontal line in the horizontal block is input to the adder 432, the addition result up to the previous pixel stored in the horizontal projection temporary storage memory 433 is read and the horizontal input from the input terminal 431 is read. Addition with the final pixel of one horizontal line of the direction block, and output the addition result, that is, data obtained by adding all the pixels of one horizontal line of the horizontal block as projection data for one horizontal line of the horizontal block The data is output from the terminal 434 to the first horizontal block projection line memory 44 and the first horizontal block projection data maximum value storage unit 45. The projection data obtained by adding all the pixels of one horizontal line of the horizontal block in the adder 432 is temporarily stored in the horizontal projection temporary storage memory 433, and the first pixel of one horizontal line of the next horizontal block is input. In this case, the projection data in the horizontal projection temporary storage memory 433 may be read out and output from the output terminal 434.

  The first horizontal block projection line memory 44 stores the horizontal block projection data of the current frame input from the horizontal block projection unit 43 for each horizontal block as first horizontal block projection data.

  The horizontal block projection data stored in the first horizontal block projection line memory 44 is read out by the horizontal block projection data smoothing unit 53 when necessary data is prepared, and smoothed. Thereafter, the data is written back to the first horizontal block projection line memory 44. For example, when the smoothing process is performed using three consecutive points Pa, Pb, and Pc in the vertical direction, “when all necessary data is aligned” means when three points are aligned in the vertical direction. At this time, it is possible to smooth the center point Pb. Details of the smoothing processing in the horizontal block projection data smoothing unit 53 will be described later.

  The first horizontal block projection data maximum value storage unit 45 calculates the maximum value related to the projection data of the horizontal block input from the horizontal block projection unit 43, and the first horizontal block projection data maximum value (current Frame horizontal block projection data maximum value).

  The second horizontal block projection line memory 46 stores the horizontal block projection data read from the first horizontal block projection line memory 44 as second horizontal block projection data related to the previous frame. Similarly, in the second horizontal block projection data maximum value storage unit 47, the first horizontal block projection data maximum value output from the first horizontal block projection data maximum value storage unit 45 is related to the previous frame. It is stored as the second horizontal block projection data maximum value (horizontal block projection data maximum value of the previous frame). In other words, the horizontal block projection data maximum value of the current frame stored in the first horizontal block projection data maximum value storage unit 45 is the second horizontal block projection data maximum value storage unit in the next frame. 47 is stored as the horizontal block projection data maximum value of the previous frame.

  The second horizontal block projection data maximum value storage unit 47 calculates a fourth threshold value based on the calculated horizontal block projection data maximum value of the previous frame. For example, if the maximum horizontal block projection data value of the previous frame is P4max, the fourth threshold value α4 is calculated by the following equation (3).

α4 = P4max × k4 (3)
Here, k4 is a predetermined coefficient, and 0 <k4 <1.

  Note that the calculation of the fourth threshold value α4 is not limited to the above equation (3), and it is sufficient if the fourth threshold value α4 increases as the horizontal block projection data maximum value P4max of the previous frame increases. Alternatively, a conversion table may be used. For the fourth threshold α4, it is not essential to use the second horizontal block projection data maximum value for each horizontal block. For example, the maximum value of the horizontal block projection data for the entire image of the previous frame is used. The fourth threshold value may be calculated by using the same fourth threshold value for the entire image.

  The second horizontal block projection data maximum value storage unit 47 sets a third threshold value (predetermined constant value) based on the second horizontal block projection data maximum value for the previous frame. For example, if the maximum horizontal block projection data value of the previous frame is P3max, the third threshold value α3 is calculated by the following equation (4).

α3 = P3max × k3 (4)
Here, k3 is a predetermined coefficient, and 0 <k3 <1.

  Note that the calculation of the third threshold value α3 is not limited to the above equation (4), and it is sufficient if the third threshold value α3 increases as the horizontal block projection data maximum value P3max of the previous frame increases. Alternatively, a conversion table may be used. For the third threshold α3, it is not essential to use the second horizontal block projection data maximum value for each horizontal block. For example, the maximum value of the horizontal block projection data for the entire image of the previous frame is used. May be used to calculate the third threshold value, and the same third threshold value may be adopted for the entire image.

  The third threshold intersection search unit 48 outputs the second horizontal block projection data read from the second horizontal block projection line memory 46 and the second horizontal block projection data maximum value storage unit 47. The intersection with the third threshold value (third threshold value intersection) is searched in the vertical direction. The information on the third threshold intersection obtained by the third threshold intersection searching unit 48 is input to the horizontal block projection data reading unit 49 and the horizontal block vertical motion vector calculating unit 50.

  The horizontal block projection data reading unit 49 outputs the (first) horizontal block projection of the current frame corresponding to the motion vector detection range centered on the third threshold intersection output from the third threshold intersection search unit 48. Data is read from the first horizontal block projection line memory 44. That is, the third threshold intersection is B (i) (where i = 1, 2,... Q, q is the total number of detected third threshold intersections), and the motion vector detection range is the third threshold. Assuming the range from (−U) to (+ U) (where U is a positive integer) centered on the intersection, the projection data read from the first horizontal block projection line memory 44 is (B (i ) −U) to (B (i) + U), which is partial data of the first horizontal block projection data.

  The first horizontal block projection data read from the first horizontal block projection line memory 44 by the horizontal block projection data reading unit 49 is output to the horizontal block vertical motion vector calculation unit 50.

  The first horizontal block projection data output from the horizontal block projection data reading unit 49 is input to the n-value converter 504 via the input terminal 501 of the horizontal block vertical motion vector calculation unit 50 shown in FIG. Is done. The n-value converter 504 outputs the first horizontal block projection data based on the fourth threshold value output from the second horizontal block projection data maximum value storage unit 47 and input via the input terminal 502. n-valued. That is, the n-value converter 504 compresses the data length of each array element of the data array Mh (FIG. 13B) regarding the projection data of the current frame extracted by the horizontal block projection data reading unit 49. About the process of this n value digitizer 504, the process similar to the n value digitizer 314 mentioned above is performed, for example, a 3 value process is performed. That is, in the ternarization processing in the n-value quantizer 504, the horizontal block projection is performed with respect to the fourth threshold value α4 (α4> 0) set based on the maximum value of the array element of the data array Mh in the projection data of the previous frame. Three values in three stages when the value of the array element relating to the projection data of the current frame extracted by the data reading unit 49 is smaller than (−α4), (−α4) or more and α4 or less, and greater than α4. Is done. Note that the n-value generator 504 does not necessarily have the same characteristics as the n-value generator 314, and may have different characteristics from the n-value generator 504.

  The operations of the third threshold value intersection search unit 48, the horizontal block projection data read unit 49, and the horizontal block vertical motion vector calculation unit 50 described above are illustrated in FIGS. This will be specifically described with reference to (c). 14A and 14B indicate the positions of the array elements of the projection data array Mh (FIG. 13B). In FIG. 14A, the third threshold value intersections B (1) to B (6) are indicated by circles.

  In the third threshold intersection search unit 48, the second horizontal block projection data maximum value storage unit 47 outputs the waveform of the second vertical block projection data W4 for the previous frame as shown in FIG. The third threshold value intersections B (1) to B (6) that intersect with the third threshold value α3 are obtained. That is, the third threshold value intersection search unit 48 sets the array element values in the order of the elements in the projection data array Mh (FIG. 13B) with respect to the projection data obtained by the horizontal block projection unit 43 with respect to the previous frame. The position of the array element at each third threshold crossing point where the waveform W4 and the line of the linear function whose array element value is the third threshold value (predetermined constant value) α3 intersects is specified.

  Next, in the horizontal block projection data reading unit 49, a first horizontal vector is detected for a predetermined motion vector detection range centered on each of the third threshold intersections B (1) to B (6) shown in FIG. The horizontal block projection data of the current frame is read from the direction block projection line memory 44. That is, the horizontal block projection data reading unit 49 obtains a data array in a predetermined range centered on the position of the array element of the projection data (data array) at each third threshold intersection at the horizontal block projection unit 43. Extracted from the projection data of the current frame (back frame). For example, with respect to the third threshold value intersection B (4), as shown in FIG. 14B, the third threshold value intersection B (4) is centered on (B (4) -U) to (B (4) + U). The first horizontal block projection data W3 corresponding to the range up to (the waveform portion within the rectangular broken line) is read out.

  The first horizontal block projection data read out by the horizontal block projection data reading unit 49 uses the fourth threshold value α4 output from the second horizontal block projection data maximum value storage unit 47. The n-value unit 504 of the horizontal block vertical motion vector calculation unit 50 performs n-value processing. For example, for the third threshold value intersection B (4), the third threshold value is based on the magnitude relationship of the first horizontal block projection data W3 with respect to the fourth threshold values α4 and (−α4) shown in FIG. For the range from (B (4) -U) to (B (4) + U) centering on the intersection B (4) (the waveform portion within the rectangular broken line), the first horizontal as shown in FIG. The direction block projection data is ternarized. As a result, the horizontal block projection data of the current frame is represented by “−1”, “0”, or “1”.

  In the n-value converter 504, n-value conversion processing as shown in FIG. 14C is performed on all the third threshold value intersections output from the third threshold value intersection search unit 48.

  Next, the adder 505 (FIG. 8) is n-valued by the n-value generator 504 around the third threshold value intersection output from the third threshold value intersection search unit 48 via the input terminal 503. The first horizontal block projection data is added to the addition value of the first horizontal block projection data converted to n-values read from the vertical motion vector addition memory 506, and the addition result Are stored in the vertical motion vector addition memory 506 again. Then, the adder 505 adds the n-valued first horizontal block projection data regarding the last one of all the third threshold intersections detected by the third threshold intersection search unit 48 to obtain one When the addition process for the horizontal block is finished, the addition result is output to the peak detector 507. In addition, after the addition process in the adder 505 is completed, the peak detection unit 507 may read out the addition result from the vertical motion vector addition memory 506 storing the addition results regarding all the third threshold intersections. good.

  The peak detector 507 includes a motion vector detection range (the above-described ± U range) centered on each third threshold intersection for all the third threshold intersections detected by the third threshold intersection search unit 48. Data obtained by adding the corresponding n-valued first horizontal block projection data (hereinafter also referred to as “horizontal block n-valued data”) is input. The value is detected by peak detector 507. The peak position of the horizontal block n-valued data detected by the peak detector 507 becomes the vertical motion vector obtained from the horizontal block, and is output to the vertical motion determination unit 51 via the output terminal 508. .

  The operations of the adder 505 and the peak detector 507 described above will be specifically described with reference to FIGS. 15 (a) and 15 (b). In FIG. 15A, the first horizontal direction in the motion vector detection range (± U range) centering on the third threshold intersections B (1) to B (6) shown in FIG. The block projection data is conceptually shown as three values.

  When the adder 505 sequentially inputs the ternary data around the third threshold intersections B (1) to B (6) shown in FIG. 15A, these data are added together, and FIG. Horizontal block n-valued data as shown in b) is generated. Specifically, the adder 505 extracts the current block projection data (data array Mh (FIG. 13B)) from the horizontal block projection data reading unit 49 and the n-value unit 504 compresses the data length. For each data array in the motion vector detection range (predetermined range) related to the frame, values of array elements having the same relative position with respect to the position of the array element at each third threshold intersection are added.

  When the horizontal block n-valued data generated by the adder 505 is input to the peak detector 507, the vertical peak position vv shown in FIG. 15B is detected as the vertical motion vector of the horizontal block. Is done. That is, the peak detector 507 detects a motion vector in the vertical direction of the frame image based on the addition result added by the adder 505.

  Next, the smoothing process in the horizontal block projection data smoothing unit 53 will be described in detail with reference to FIGS. The horizontal block projection data stored in the first horizontal block projection line memory 44 is read out by the horizontal block projection data smoothing unit 53, and after smoothing processing is performed, the first horizontal block projection data is read out. It is written back to the line memory 44. In FIGS. 16 to 19, an example will be described in which smoothing processing is performed using three-point array elements that are continuous in the vertical direction, such as Pa, Pb, and Pc. Note that a, b, and c are values at points Pa, Pb, and Pc, respectively.

  FIG. 16 shows a smoothing process when the variation pattern of the projection data is “convex downward, a> 0, b ≧ 0, c> 0”.

  For detection of motion vectors, it is important to gradually increase and decrease by a plurality of consecutive points in the projection data, and unevenness due to a small number of points causes a drop in accuracy due to the relationship with the threshold value, so it is desirable to remove them. Therefore, in the pattern shown in FIG. 16, the downwardly convex shape is eliminated by changing the value b of the central point Pb to the average value of the values a and c.

  Similarly, FIG. 17 shows a smoothing process when the variation pattern of the projection data is “convex upward, a <0, b ≦ 0, c <0”. Also in this case, by changing the value b of the center point Pb to the average value of the values a and c, the shape that is convex upward is eliminated.

  On the other hand, FIG. 18 shows a smoothing process when the variation pattern of the projection data is “convex downward, a> 0, b <0, c> 0”.

  In this case, b <−α2, and if the processing proceeds as it is, the value b is based on the fourth threshold value −α4 during the n-value conversion processing (in this example, ternarization) by the n-value converter 504. When the values of the front and rear points Pa and Pc are 1, the values of the points Pa, Pb, and Pc are 1, -1, 1 and vibrate. The adder 505 adds a large vibration (amplitude) by adding each of the third threshold intersections as the center, and this vibration causes a place where one large peak should be originally placed at the position indicating the motion vector position. A large value is also shown in a portion other than the original peak position.

  However, when the smoothing process similar to that shown in FIG. 16 or 17 is executed for the variation pattern shown in FIG. 18, there is a problem that the feature of the projection data cannot be accurately reflected in the n-value conversion process.

  That is, as shown in FIG. 18, when the value b of the point Pb is smaller than the fourth threshold value −α4 and the point Pa and the value of the point Pc are larger than 0, the point Pa is the point Pa It is considered that a part of the former fluctuation pattern constituted by the earlier points Pq, Pr,... (<Pr <Pq <Pa) is constituted and the point Pb also constitutes the former fluctuation pattern. It is done.

  Further, the point Pc constitutes a part of the subsequent fluctuation pattern constituted by the points Ps, Pt,... (Pc <Ps <Pt <...) After the point Pc, and the point Pb is also in the subsequent stage. It is conceivable to construct a variation pattern.

  In such a case, if the smoothing process is performed in which the value b of the point Pb is simply set to the average value of the value a and the value c, the characteristics of the fluctuation pattern in the preceding stage and the fluctuation pattern in the subsequent stage are modified. Thus, the features of both patterns cannot be accurately reflected in the n-value conversion process.

  Therefore, in the variation pattern as shown in FIG. 18, a smoothing process is performed in which the value b of the point Pb is changed to a value (−β) that is compressed to “0” when the n-value conversion process is performed. . Here, a value compressed to “0” is referred to as a compressed value, and the compressed value is set to a fixed value that is greater than the second threshold value −α4 and equal to or less than 0. By this processing, it is possible to accurately reflect the features of both patterns in the n-value conversion processing without impairing the features of the preceding and following variation patterns.

  FIG. 19 shows the smoothing process when the variation pattern of the projection data is “convex upward, a <0, b> 0, c <0”.

  In this case, b> α4, and if the processing proceeds as it is, the value b is set to 1 based on the fourth threshold value α4 during the n-value conversion processing (in this example, ternarization) by the n-value converter 504. When the values of the front and rear points Pa and Pc are -1, the values of the points Pa, Pb, and Pc become -1, 1, -1, and vibrate. The adder 505 adds a large vibration (amplitude) by adding each of the third threshold intersections as the center, and this vibration causes a place where one large peak should be originally placed at the position indicating the motion vector position. A large value is also shown in a portion other than the original peak position.

  However, when the smoothing process similar to that shown in FIG. 16 or 17 is executed for the variation pattern shown in FIG.

  That is, as shown in FIG. 19, when the value b of the point Pb is larger than the fourth threshold value α4 and the point Pa and the value of the point Pc are smaller than 0, the point Pa is larger than the point Pa. It is conceivable that a part of the former fluctuation pattern constituted by the previous points Pq, Pr,... (... <Pr <Pq <Pa) is constituted, and the point Pb also constitutes the former fluctuation pattern. .

  Further, the point Pc constitutes a part of the subsequent fluctuation pattern constituted by the points Ps, Pt,... (Pc <Ps <Pt <...) After the point Pc, and the point Pb is also in the subsequent stage. It is conceivable to construct a variation pattern.

  In such a case, if the smoothing process is performed in which the value b of the point Pb is simply set to the average value of the value a and the value c, the characteristics of the fluctuation pattern in the preceding stage and the fluctuation pattern in the subsequent stage are modified. Thus, it cannot be accurately reflected in the feature n-value conversion processing of both patterns.

  Therefore, in the variation pattern as shown in FIG. 19, a smoothing process is performed in which the value b of the point Pb is changed to a value (β) that is compressed to “0” when the n-value conversion process is performed. Here, a value compressed to “0” is referred to as a compressed value, and the compressed value is set to a fixed value that is smaller than the fourth threshold value α4 and equal to or greater than zero. By this processing, it is possible to accurately reflect the features of both patterns in the n-value conversion processing without impairing the features of the preceding and following variation patterns.

  Note that when the compressed value is set to 0, if the compressed value at the point Pb is used for smoothing the subsequent fluctuation pattern, the characteristics of the subsequent fluctuation pattern will slightly change, but this is not a problem. For example, the compressed value may be set to 0.

  Next, the effect of the smoothing process in the horizontal block projection data smoothing unit 53 will be described using specific examples of the variation pattern of the projection data shown in FIGS.

  FIG. 20 shows a part of the waveform of horizontal block projection data when smoothing processing is not performed. As shown in FIG. 20, the waveform is a waveform that vibrates up and down at each adjacent point.

  The cause of this vibration is not only due to various signal processing in the previous stage circuit, but also in the horizontal direction, for example, when the signal level slightly varies from line to line in the camera signal etc. It is conceivable that the addition (emphasis) is made remarkable each time processing is performed.

  FIG. 21 is a diagram schematically showing a state in which threshold values α and −α are set for the projection data shown in FIG. 20 when the ternarization process is performed. The threshold values are increased and decreased at each point in a circled area. I know that Therefore, if the ternarization process is performed as it is, the ternarization result is greatly affected.

  Here, FIG. 22 shows the result of applying the smoothing processing described with reference to FIGS. 16 to 19 to the projection data shown in FIG.

  In FIG. 22, while maintaining the overall characteristics of the waveform of the original vertical block projection data, the waveform superimposed thereon is removed.

  For example, the circled regions R11 to R15 are regions obtained by smoothing the data included in the region where the threshold value is raised or lowered in FIG. 21, and these are the data of the adjacent data described with reference to FIGS. A smoothing process using an average value is performed.

  Further, the region R10 is large such that the value of the point 23 is larger than the threshold value α and the values of the point 22 and the point 24 are smaller than 0, as indicated by the points 22, 23 and 24 in FIG. These are areas having fluctuations, and these are values that are compressed to “0” when the value of the central point among the three points described with reference to FIGS. 18 and 19 is subjected to n-value conversion processing. The smoothing process of changing to is performed.

  By performing the smoothing process as described above, horizontal block projection data that does not significantly affect the ternary value due to vibration near the intersection with the threshold value can be obtained.

  FIG. 23 shows all third threshold intersections detected by the first threshold intersection search unit 48 by the adder 505 after the n-value conversion processing by the n-value conversion unit 504 is performed on the first horizontal block projection data. FIG. 10 illustrates an example of horizontal block n-valued data added in a motion vector detection range (range of ± U described above) centered on each third threshold intersection. Here, U is 31 and the scale 32 on the horizontal axis indicates the position of the vertical motion vector 0.

  The data indicated by the filled diamonds is “no smoothing” data, and the data indicated by the filled squares is “smoothed” data.

  Of the waveforms represented by both data, the waveform of “no smoothing” is oscillating greatly, and is convex upward and downward in various places. On the other hand, the waveform with “smoothing process” is a waveform that does not have extra unevenness while retaining the characteristics of the original waveform.

  In FIG. 23, both the “no smoothing” waveform and the “smoothing” waveform have a maximum peak at the position of the scale 30, so the peak detector 507 uses the maximum peak of the entire motion vector detection range. When the position is determined as a horizontal motion vector, no erroneous determination occurs.

  However, for example, by calculating the peak position for each “upward convex block” whose value changes from negative to positive to negative, and evaluating the magnitude of the peak value for each “block”, the movement of the periodic image When a reliability evaluation is performed to eliminate the vector calculation result and a method that further increases the calculation accuracy is adopted, the waveform of “no smoothing process” in this example is evaluated as “no reliability”. For the waveform “with smoothing”, a correct vertical motion vector can be calculated.

  That is, in the waveform of “without smoothing”, in addition to the scale 30, in addition to the scale 28 and the scale 32, “lumps” are formed at a large number of positions, and each has an added value of 40, 30, and 26. is doing. Originally, it should have only one large peak other than a small peak. Thus, in the case of having a large peak value equivalent at many positions, the result is not reliable.

  On the other hand, the waveform of “with smoothing process” has a peak of the addition value 21 at the scale 30, the addition value 5 at the scale 42, and the addition value 3 at the scale 5, but the peak of the scale 30 can be determined as the only large peak. Therefore, the scale 30, that is, the vector-2 can be determined as the vertical motion vector.

  The vertical motion vector calculated for each horizontal block by the peak detector 507 is sequentially input to the vertical motion vector determination unit 51 via the output terminal 508.

  In the vertical direction motion vector determination unit 51 (FIG. 6), the vertical direction motion vector of the entire image is determined based on the vertical direction motion vector of each horizontal direction block sequentially output from the horizontal direction block vertical direction motion vector calculation unit 50. Is done. Specifically, a histogram is created for the vertical motion vector of each horizontal block, and the vertical motion vector having the largest histogram value is used as the vertical motion vector for the entire screen.

  As described above, the horizontal block projection data smoothing unit 53 can remove the fluctuation noise superimposed on the horizontal block projection data without impairing a large tendency (feature) of the horizontal block projection data. Since an extra peak is not generated as an addition result in the adder 505, an erroneous determination and a decrease in reliability in the peak detector 507 can be reduced. As a result, a reliable horizontal block vertical motion vector can be calculated.

<Effect>
As described above, in the horizontal direction motion vector detection unit 2 of the motion vector detection device 1 according to the first embodiment, the vertical direction block projection data smoothing unit 34 performs a general trend of the vertical direction block projection data. Since the superimposed local fluctuation noise can be removed without impairing (features), a reliable vertical block horizontal motion vector can be detected. As a result, the horizontal motion vector of the entire image can be determined based on the highly reliable vertical block horizontal motion vector.

  In the vertical direction motion vector detection unit 4, the horizontal direction block projection data smoothing unit 53 generates superimposed local fluctuation noise without impairing the general tendency (feature) of the horizontal direction block projection data. Since it can be removed, a reliable horizontal block vertical motion vector can be detected. As a result, the vertical motion vector of the entire image can be determined based on the highly reliable horizontal block vertical motion vector.

  In addition, the bit number reduction unit 25 of the motion vector detection device 1 performs vertical projection of the current frame based on the maximum value of the vertical projection data of the current frame obtained by the first vertical block projection data maximum value storage unit 24. Since the number of data bits is reduced, the amount of calculation in the subsequent processing can be suppressed, and the motion vector can be detected more quickly.

  Further, in the motion vector detection device 1, an appropriate first threshold value intersection is set in order to obtain the maximum value of the second vertical block projection data regarding the previous frame and to set the first threshold value α1 based on this maximum value. it can. Similarly, since the maximum value of the second horizontal block projection data regarding the previous frame is obtained and the third threshold value α3 is set based on this maximum value, an appropriate third threshold value intersection can be set.

  Further, in the motion vector detection device 1, the vertical image is divided by the vertical image dividing unit 21 and then the vertical edge of each divided image (vertical block) is emphasized. Therefore, the vertical edge is emphasized. The divided image can be easily generated. Similarly, after the frame image is divided in the horizontal direction by the horizontal image dividing unit 41, the horizontal edge of each divided image (horizontal block) is emphasized. Therefore, the divided image in which the horizontal edge is emphasized is simplified. Can be generated.

  Further, in the motion vector detection device 1, the n-value generator 314 of the vertical block horizontal motion vector calculation unit 31 converts the projection data read by the vertical block projection data reading unit 30 into n-values. The amount of calculation can be reduced, and motion vectors can be detected more quickly. In particular, since the n-value converter 314 performs ternarization based on the second threshold value α2 set based on the maximum value of the vertical block projection data of the current frame, real-time processing is performed while ensuring a relatively large dynamic range. Possible motion vector detection can be realized. Also, the memory can be greatly reduced.

  Similarly, since the projection data read out by the horizontal block projection data reading unit 49 is converted into n values in the n-value converter 504 of the horizontal block vertical motion vector calculation unit 50, the subsequent calculation amount can be reduced. The motion vector can be detected more quickly. In particular, since the n-value converter 504 performs ternarization based on the fourth threshold value α4 set based on the maximum value of the horizontal block projection data of the previous frame, an appropriate fourth threshold value α4 can be set, and dynamic Motion vector detection capable of real-time processing can be realized without greatly degrading the range. Also, the memory can be greatly reduced.

<Embodiment 2>
<Configuration of motion vector detection device>
The motion vector detection device 1A according to the second embodiment of the present invention has a configuration similar to that of the motion vector detection device 1 according to the first embodiment shown in FIG. 1, but is perpendicular to the horizontal motion vector detection unit. The configuration differs from that of the directional motion vector detection unit. The configuration of the horizontal direction motion vector detection unit 2A and the vertical direction motion vector detection unit 4A in the motion vector detection device 1A will be described with reference to FIGS.

  FIG. 24 is a block diagram showing a main configuration of the horizontal direction motion vector detection unit 2A. In FIG. 24, parts having the same functions as those in the first embodiment are denoted by the same reference numerals.

  In the horizontal direction motion vector detection unit 2A, the arrangement of the vertical direction image division unit 21 and the vertical direction edge extraction filtering unit 22 is reversed with respect to the horizontal direction motion vector detection unit 2 of the first embodiment. Below, operation | movement of this horizontal direction motion vector detection part 2A is demonstrated.

  As shown in FIG. 2, when a frame image read by sequentially repeating pixel scanning of the horizontal line in the vertical direction is input to the input terminal 20 of the horizontal motion vector detection unit 2A, the vertical edge extraction filtering unit 22 Extraction of edge components extending in the vertical direction, in other words, filtering processing for emphasizing image portions that change sharply in the horizontal direction is performed.

  When the frame image in which the vertical edge is emphasized by the vertical edge extraction filtering unit 22 is input to the vertical image dividing unit 21, it is divided into blocks in the vertical direction. That is, the vertical image dividing unit 21 sets a plurality of image regions (vertical blocks) obtained by dividing the frame image subjected to edge enhancement in the vertical direction. Thereby, in the subsequent processing, processing and management are performed for each vertical block.

  The image data divided by the vertical image dividing unit 21 is input to the vertical block projecting unit 23. The processes after the vertical block projecting unit 23 are the horizontal direction motion vector detecting unit 2 of the first embodiment. The same processing is performed.

  FIG. 25 is a block diagram showing a main configuration of the vertical motion vector detection unit 4A. In FIG. 25, parts having the same functions as those in the first embodiment are denoted by the same reference numerals.

  In the vertical direction motion vector detection unit 4A, the arrangement of the horizontal direction image division unit 41 and the horizontal direction edge extraction filtering unit 42 is reversed with respect to the vertical direction motion vector detection unit 4 of the first embodiment. Hereinafter, the operation of the vertical motion vector detection unit 4A will be described.

  As shown in FIG. 2, when a frame image read out by repeating the pixel scanning of the horizontal line in order in the vertical direction is input to the input terminal 40 of the vertical direction motion vector detection unit 4A, the horizontal direction edge extraction filtering unit 42 Extraction of the edge component extending in the horizontal direction, in other words, filtering processing for emphasizing an image portion that changes sharply in the vertical direction is performed.

  When the frame image in which the edge in the horizontal direction is emphasized by the horizontal edge extraction filtering unit 42 is input to the horizontal direction image dividing unit 41, it is divided into blocks in the horizontal direction. That is, the horizontal image dividing unit 41 sets a plurality of image regions (horizontal blocks) obtained by dividing the frame image subjected to edge enhancement in the horizontal direction. Thereby, in subsequent processing, processing and management are performed for each horizontal block.

  The image data divided by the horizontal image dividing unit 41 is input to the horizontal block projecting unit 43. The processing after the horizontal block projecting unit 43 is the vertical motion vector detecting unit 4 of the first embodiment. The same processing is performed.

  The operation of the motion vector detection device 1A described above provides the same effects as those of the first embodiment.

  Then, in the motion vector detection device 1A, the vertical edge extraction filtering unit 22 emphasizes the edge in the vertical direction with respect to the frame image, and then the vertical image dividing unit 21 divides the frame image subjected to edge enhancement in the vertical direction. A divided image with emphasized vertical edges can be easily generated.

  Similarly, the horizontal edge extraction filtering unit 42 emphasizes the horizontal edge of the frame image, and then the edge-enhanced frame image is divided in the horizontal direction by the horizontal image dividing unit 41. Therefore, the horizontal edge is emphasized. The divided image can be easily generated.

  In the motion vector detection device 1A, it is essential to arrange the vertical edge extraction filtering unit 22 and the horizontal edge extraction filtering unit 42 separately in the horizontal motion vector detection unit 2A and the vertical motion vector detection unit 4A. In addition, a horizontal / vertical edge extraction filtering unit in which the functions of both the vertical edge extraction filtering unit 22 and the horizontal edge extraction filtering unit 42 are integrated is provided, and the output is provided to the vertical image dividing unit 21 and the horizontal direction. Each of the image division units 41 may be input.

<Modification>
For the vertical direction edge extraction filtering unit and the horizontal direction edge extraction filtering unit in each of the above embodiments, after the filtering process for performing edge extraction, for example, “with edge” and “ The binarization of “no edge” or the ternarization of “with positive edge”, “without edge”, and “with negative edge” may be performed. By performing such threshold processing, it is possible to treat edges having a luminance change (luminance gradient) equal to or greater than a predetermined threshold in the same way.

  In each of the above-described embodiments, the data output from the bit number reduction unit 25 is converted into the first vertical block projection line memory 26 and the second vertical block projection line memory 27 by, for example, switch switching for each frame. You may make it input alternately.

  Further, the data output from the horizontal block projection unit 43 is alternately input to the first horizontal block projection line memory 44 and the second horizontal block projection line memory 46 by, for example, switch switching for each frame. Anyway.

  Similarly, the data output from the vertical (horizontal) direction block projection unit is converted into, for example, a first vertical (horizontal) direction block projection data maximum value storage unit and a second vertical (horizontal) direction block by switching switches for each frame. Alternatively, the data may be alternately input to the projection data maximum value storage unit.

  The “previous frame” in the present invention is not limited to the previous frame, but includes two or more previous frames. For example, when a frame in the middle is skipped by the frame thinning process, the frame before the skipped frame is included.

It is a block diagram which shows the principal part structure of the motion vector detection apparatus 1 (1A) which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the pixel scanning direction in a frame image. 3 is a block diagram showing a main configuration of a horizontal motion vector detection unit 2. FIG. FIG. 4 is a block diagram showing a main configuration of a vertical block projection unit 23. FIG. 6 is a block diagram showing a main configuration of a vertical block horizontal motion vector calculation unit 31. FIG. 3 is a block diagram illustrating a main configuration of a vertical motion vector detection unit 4. FIG. 4 is a block diagram illustrating a main configuration of a horizontal block projection unit 43. FIG. 6 is a block diagram showing a main configuration of a horizontal block vertical motion vector calculation unit 50. It is a figure for demonstrating operation | movement of the vertical direction image division part 21 and the vertical direction block projection part 23. FIG. It is a figure for demonstrating the n-value conversion process in the n-value conversion device 314 (504). It is a figure for demonstrating operation | movement of the 1st threshold value intersection search part 29, the vertical direction block projection data reading part 30, and the vertical direction block horizontal direction motion vector calculation part 31. FIG. 6 is a diagram for explaining operations of an adder 315 and a peak detector 317. FIG. It is a figure for demonstrating operation | movement of the horizontal direction image division part 41 and the horizontal direction block projection part 43. FIG. It is a figure for demonstrating operation | movement of the 3rd threshold value intersection search part 48, the horizontal direction block projection data reading part 49, and the horizontal direction block vertical direction motion vector calculation part 50. FIG. FIG. 10 is a diagram for explaining operations of an adder 505 and a peak detection unit 507. It is a figure for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). It is a figure for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). It is a figure for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). It is a figure for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). It is a figure which shows an example of the projection data waveform for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). It is a figure which shows an example of the projection data waveform for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). It is a figure which shows an example of the projection data waveform for demonstrating the smoothing process in the vertical direction block projection data smoothing part 34 (horizontal direction block projection data smoothing part 53). Vertical block n-valued data (horizontal block n-valued data) when smoothing processing is performed in the vertical block projection data smoothing unit 34 (horizontal block projection data smoothing unit 53) It is a figure which shows an example of this waveform. It is a block diagram which shows the principal part structure of 2 A of horizontal direction motion vector detection parts in 1 A of motion vector detection apparatuses which concern on Embodiment 2 of this invention. It is a block diagram which shows the principal part structure of 4 A of vertical direction motion vector detection parts in 1 A of motion vector detection apparatuses.

Explanation of symbols

  1, 1A motion vector detection device, hb0 to hb9 horizontal block, hn0 to hn9 horizontal block projection data, vb0 to vb7 vertical block, vn0 to vn6 vertical block projection data, A (1) to A (8) 1 threshold intersection, α1 first threshold, α2 second threshold, B (1) to B (6) third threshold intersection, α3 third threshold, α4 fourth threshold.

Claims (12)

A motion vector detection device that detects a motion vector between frame images related to a preceding frame and a following frame that are in a time series front-rear relationship,
With respect to a frame image read out by repeating pixel scanning of a horizontal line in order in the vertical direction, edge enhancement means for enhancing a vertical edge for a predetermined image region in the frame image;
Projecting means for projecting the image in which the edge is enhanced by the edge enhancement means in the vertical direction and generating projection data having a data array for one horizontal line;
Smoothing means for obtaining smoothed projection data by removing fluctuation noise superimposed on the projection data;
With respect to the smoothed projection data with respect to the previous frame, an array of intersections at which a waveform obtained by graphing array element values in the order of the elements of the data array intersects with a straight line where the array element values are a predetermined constant value. A identifying means for locating the element;
Extraction means for extracting a predetermined range of data array centered on the position of the array element at each intersection from the smoothed projection data for the subsequent frame;
For each data array in a predetermined range extracted by the extracting means, an adding means for adding values of array elements having the same relative position to the position of the array element at each intersection;
A motion vector detection device comprising: detection means for detecting a horizontal motion vector for a predetermined image area of the frame image based on the addition result added by the addition means. A motion vector detection device that detects a motion vector between frame images related to a preceding frame and a following frame that are in a time series front-rear relationship,
With respect to a frame image that is read out by repeating pixel scanning of a horizontal line in order in the vertical direction, edge enhancement means that enhances a horizontal edge for a predetermined image region in the frame image;
Projecting means for projecting in the horizontal direction the image whose edges are emphasized by the edge enhancement means and generating projection data having a data array for one vertical line;
Smoothing means for obtaining smoothed projection data by removing fluctuation noise superimposed on the projection data;
An array element at each intersection where a waveform obtained by graphing array element values in the order of elements of the data array with respect to the smoothed projection data with respect to the previous frame intersects with a straight line where the array element value is a predetermined constant value A specifying means for specifying the position of
Extraction means for extracting a predetermined range of data array centered on the position of the array element at each intersection from the smoothed projection data for the subsequent frame;
For each data array in a predetermined range extracted by the extracting means, an adding means for adding values of array elements having the same relative position to the position of the array element at each intersection;
A motion vector detection device comprising: detection means for detecting a vertical motion vector for a predetermined image region of the frame image based on the addition result added by the addition means. Maximum value specifying means for obtaining a maximum value in an array element of projection data relating to the previous frame,
The motion vector detection device according to claim 1, wherein the predetermined constant value is set based on a maximum value obtained by the maximum value specifying unit. Compression means for compressing the data length of each array element relating to the data array in a predetermined range extracted by the extraction means,
The adding means includes
The means for adding the values of array elements having the same relative position to the position of the array element at each intersection for each data array in a predetermined range whose data length is compressed by the compression means. 3. The motion vector detection device according to 2. The compression means includes
Means for ternarizing the value of each array element relating to the data array in a predetermined range extracted by the extraction means;
In the ternary processing, the projection extracted by the extraction unit with respect to the projection data of the subsequent frame or the threshold Th (Th> 0) set based on the maximum value of the array element in the projection data of the previous frame. The motion vector according to claim 4, wherein three-level ternarization is performed when a value of an array element related to data is smaller than (−Th), when it is greater than (−Th) and smaller than or equal to Th, and larger than Th. Detection device. The smoothing means includes
For a waveform obtained by graphing array element values in the order of elements of the data array with respect to the projection data, a smoothing process is executed based on three consecutive array elements,
The smoothing process is
When the three array elements are all positive values and form a downwardly convex pattern, the value of the center array element is changed to the average value of the preceding and following array elements,
Including a process of changing the value of the central array element to the average value of the preceding and following array elements when the three array elements are all negative values and form an upwardly convex pattern, The motion vector detection apparatus according to claim 1 or 2. The smoothing process is
When the three array elements are positive, zero, and positive values and form a downward convex pattern, the value of the central array element is changed to the average value of the array elements before and after,
When the three array elements are negative, zero, and negative values and form an upwardly convex pattern, the process of changing the value of the central array element to the average value of the preceding and following array elements The motion vector detection device according to claim 1, comprising: The smoothing means includes
For a waveform obtained by graphing array element values in the order of elements of the data array with respect to the projection data, a smoothing process is executed based on three consecutive array elements,
The smoothing process is
When the three array elements are positive, negative, and positive values and form a downwardly convex pattern, the value of the central array element is set to a value that is compressed to zero by the compression means. change,
When the three array elements are negative, positive, and negative values and form an upwardly convex pattern, the value of the central array element is set to a value that is compressed to zero by the compression means. The motion vector detection device according to claim 5, comprising a process of changing. The edge enhancement means includes
Means for setting a plurality of image areas obtained by dividing the frame image in the vertical direction as the predetermined image areas;
The motion vector detection apparatus according to claim 1, further comprising: a unit that emphasizes a vertical edge for each of the plurality of image regions. The edge enhancement means includes
Emphasis means for emphasizing vertical edges of the frame image;
The motion vector detection apparatus according to claim 1, further comprising: a plurality of image regions obtained by dividing a frame image whose edges are enhanced by the enhancement unit in the vertical direction as the predetermined image regions. The edge enhancement means includes
Means for setting a plurality of image areas obtained by dividing the frame image in the horizontal direction as the predetermined image areas;
The motion vector detection apparatus according to claim 2, further comprising a unit that emphasizes a horizontal edge for each of the plurality of image regions. The edge enhancement means includes
Enhancing means for enhancing a horizontal edge of the frame image;
The motion vector detection device according to claim 2, further comprising: a plurality of image regions obtained by dividing a frame image whose edges are enhanced by the enhancement unit in the horizontal direction as the predetermined image regions.

Images