JPH05110933A - Motion vector detector for image - Google Patents

Abstract

PURPOSE:To provide the motion vector detector for image capable of detecting the motion vectors of images with high accuracy. CONSTITUTION:Respective subtraction circuits SUB1-SUBn detect difference between the image data of respective data picture elements in the block of a current frame and the image data of representative point picture elements in preceding fields and the from 1 to (n) read from respective representative point memories FM1-FMn, and the motion vectors of images among the fields from 1 to (n) are detected by the respective motion vector detecting circuits SUB1-SUBn and outputted through a selector SEL. An acceleration judging circuit ACJ judges acceleration for the motion of the image based on the motion vectors of the images among the fields from 1 to (n) and controls the selection of the selector SEL.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image motion vector detection device, and an image motion correction for detecting and correcting an image movement amount due to a so-called camera shake included in video data, such as an image pickup output of a handy type video camera. It is applied to devices.

[0002]

2. Description of the Related Art Generally, in a handy type video camera, camera shake during shooting, that is, camera vibration, appears as image vibration. Therefore, as an image motion correction device that corrects the image vibration due to such camera shake, for example, as disclosed in Japanese Patent Laid-Open No. 63-166370, a motion vector of an image is detected and based on this motion vector. , Which correct the video data stored in the image memory have been proposed.

A block matching method, for example, is employed to detect the motion vector of the image. In the detection of the motion vector of the image by this block matching method, the screen is divided into a number of areas (referred to as blocks), and the representative point pixel of the previous field located at the center of each block and each pixel in the block of the current field are divided. The absolute value of the field difference from the image data is calculated, the absolute value of the field difference of each block is integrated for each corresponding pixel to obtain the correlation integral value, and the correlation integration having the coordinates corresponding to the pixel array for one block is calculated. Form a value table. Then, the motion vector of the entire screen is determined using the coordinate value of the minimum value of the correlation integrated value in the correlation integrated value table as the coordinate value of the motion vector of the image.

Then, the image motion correction device converts the detected motion vector into a correction signal and performs correction for moving the current image by this correction signal. The correction accuracy in such an image motion correction device depends on the detection accuracy of the motion vector of the image.

[0005]

By the way, as described above, the motion vector between the fields of the entire screen is determined by using the coordinate value of the minimum value of the correlation integrated value in the correlation integrated value table as the coordinate value of the motion vector of the image. In the conventional motion vector detecting apparatus for image by the block matching method described above, high detection accuracy cannot be obtained because the detection time interval is short in one field period, and moreover, the above correlation integral value is discrete corresponding to each pixel. Since the motion vector of the image can be obtained only by the integer coordinate values in the above correlation integrated value table, the detection accuracy further decreases. Further, in a video camera that outputs a video signal complying with a standard television system such as the NTSC system that employs interlaced scanning, high detection accuracy cannot be obtained because there are no corresponding pixels in space.

Therefore, when the image stabilization vector is formed by the image motion vector detected by the conventional image motion vector detection apparatus and the image stabilization is performed, sufficient correction accuracy cannot be obtained, which causes unnaturalness. It becomes an image.

The present invention has been made in view of such circumstances, and an object thereof is to enable high-performance image stabilization in a handy type video camera or the like,
Provided is an image motion vector detection device capable of detecting an image motion vector with high accuracy.

[0008]

In order to solve the above-mentioned problems, an image motion vector detecting apparatus according to the present invention is a representative of each block obtained by dividing an image of one field composed of an input video signal into a plurality of blocks. Cascaded representative point memories that store image data of point pixels for one field respectively, image data of each pixel of a block of the current frame, and blocks 1 to n fields before read from the representative point memories. N difference detecting means for detecting the difference from the image data of the representative point pixel, and n motion vector detecting means based on each difference value between 1 to n fields detected by the difference detecting means. Motion vector detecting means, and acceleration determining means for determining the motion acceleration of the image based on each motion vector obtained by the motion vector detecting means. Motion vector selecting means for selectively outputting each motion vector obtained by the motion vector detecting means in accordance with the judgment output by the acceleration judging means, and the larger the fluctuation of the motion vector for one field, the shorter the field The motion vector based on the difference value of is selected and output by the motion vector selection means.

[0009]

In the image motion vector detecting device according to the invention,
Image data of representative point pixels of each block obtained by dividing an image of one field composed of an input video signal into a plurality of blocks are stored in n representative point memories connected in series for one field, respectively, and detected by a difference detecting means. Based on each difference value between the image data of each pixel of the block of the current frame and the image data of the representative point pixel of the block 1 to n fields before, the vector detecting means detects n types of motion vectors. Then, the acceleration of the motion of the image is judged by the acceleration judging means based on each motion vector obtained by the motion vector detecting means, and as the fluctuation of the motion vector for one field is larger, the motion based on the difference value between the shorter fields is calculated. A vector is selected by the motion vector selection means and output.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image motion vector detecting device according to the present invention will be described in detail below with reference to the drawings. An image motion vector detecting device according to the present invention is shown in FIG.
It is configured as shown in.

The image motion vector detection apparatus 10 shown in FIG. 1 is an application of the present invention to a camera shake correction apparatus that corrects image movement due to camera shake in a handy type video camera. Correction unit 30
Together with this, it constitutes a camera shake correction device. In FIG. 1, a signal input terminal 1 is supplied with input video data obtained by digitizing a video signal obtained as an imaging output by an imaging unit (not shown) of the video camera.

The motion vector detecting device 10 includes n representative point memories FM _{1 to} FM _n connected in cascade, and image data of representative point pixels of each field read from these representative point memories FM _{1 to} FM _n. Of _n subtraction circuits SUB _{1 to} SUB _n , and these subtraction circuits SUB
₁ and n pieces of the motion vector detecting circuit DET ₁ ～DET _n each subtraction output data by ～SUB _n is supplied, the acceleration is the motion vector detected by these motion vector detection circuit DET ₁ ～DET _n supplied Judgment circuit AC
Comprising a J and the selector SEL, the input video data are supplied to the subtraction circuit SUB ₁ ～SUB _n of the first representative point memory FM ₁ and the first to n-th through the signal input terminal 1 ..

The first representative point memory FM ₁ stores image data of representative point pixels for each block obtained by dividing the image of one field constituted by the input video data into a plurality of blocks. Specifically, for example, as shown in FIG. 2, a screen of one field is divided into blocks of m pixels × n lines,
As shown in FIG. 3, a pixel at the center of each block is used as a representative point, and image data of each representative point pixel is stored in the representative point memory FM.
_One field is stored for one field. The above representative points are
It is scattered evenly on the screen. Then, the first image data of each representative point pixels of one field before from the representative point memory FM ₁ is read out is supplied to the second representative point memory FM ₂ is supplied to the subtraction circuit SUB ₁ It

The second representative point memory FM ₂ stores the image data of each representative point pixel one field before read from the first representative point memory FM ₁ for one field period. Then, from the two representative point memories FM ₂ to 2
The image data of each representative point pixel before the field is read out, supplied to the _second subtraction circuit SUB ₂ and supplied to the third representative point memory FM ₃ .

[0015] Similarly, the representative point memory FM ₃ ～FM _n the third to n-th front of the representative point memory FM ₂ ~FM
The image data of the representative point pixels 2 to n-1 fields before, which is supplied from _n-1, is stored for one field period. Then, these third to the representative point memory FM ₃ ～FM _n of the n
The image data of each representative point pixel three to n fields before is read out, and the third to nth subtraction circuit SUB ₃ is read.
~ SUB _n .

The first subtraction circuit SUB ₁ sets m for each block for the input video data supplied via the signal input terminal 1, that is, the image data of the current field.
The difference between the image data of each of the n pixels and the image data of the representative point pixel of the corresponding block read one field before and read from the first representative point memory FM _1, that is, each difference between one fields is detected.

Further, the second subtraction circuit SUB ₂ receives the image data of m × n pixels for each block for the input video data supplied through the signal input terminal 1, that is, the image data of the current field. And the image data of the representative point pixel of the corresponding block two fields before, which is read from the second representative point memory FM _2, that is, each difference between the two fields is detected.

Similarly, the third to n-th subtraction circuits SUB _{3 to} SUB _n have m × m × blocks for the input video data supplied through the signal input terminal 1, that is, the image data of the current field. n keypoints memory FM ₃ ～FM the image data and the third to n of each pixel
detecting each difference between the difference i.e. 1 to n fields of the 3 to n previous field of the image data of the representative point pixels of the corresponding block is read from the _n.

The first to n-th subtraction circuits SU
₁ obtained as subtraction output data by B _{1 to} SUB _n
The difference value data between the nth field to the nth field are supplied to the _{first to nth} motion vector detection circuits DET _{1 to} DET _n .

The first motion vector detection circuit DET ₁
Is the sum of the difference values between the fields of each block obtained by the _first subtraction circuit SUB ₁ for each corresponding pixel over one field period, and m × corresponding to the pixel array for one block. A correlation integrated value table having n integer coordinates is formed, and the minimum coordinate P (x, y) of the correlation integrated values is detected as a motion vector coordinate. Here, the correlation integrated value table shows the distribution of the integrated value of the difference value between one field, and the correlation integrated value of the coordinates corresponding to the pixel having the strongest one-field correlation with respect to the representative point pixel is the minimum value. Become. Therefore, the coordinate P (x, y) of the minimum correlation integrated value with respect to the coordinate P (0, 0) corresponding to the representative point pixel represents the motion vector. Then, the first motion vector detection circuit DET ₁ detects the coordinate P of the minimum correlation integrated value.
A motion vector is calculated from (x, y) and output.

The second motion vector detecting circuit D
ET ₂ forms a correlation integrated value table by integrating the difference value between the two fields of each block obtained by the _second subtraction circuit SUB ₂ for each corresponding pixel over one field period, and forming the correlation integrated value table. The coordinate P (x, y) of the minimum integrated value is detected as the motion vector coordinate. Then, the second motion vector detection circuit DET ₂ calculates and outputs a motion vector obtained by converting the coordinate P (x, y) of the minimum correlation integrated value into a motion vector for one field. That is,
The motion vector is calculated by converting the coordinates P (x, y) of the minimum correlation integrated value into the motion vector coordinates P (x / 2, y / 2) in one field.

Similarly, the third to n-th motion vector detection circuits DET _{3 to} DET _n are the blocks 3 to 3 of the blocks obtained by the third to _n-th subtraction circuits SUB _{3 to} SUB _n. The difference values between n fields are integrated for each corresponding pixel over one field period to form a correlation integrated value table, and the coordinates P of the minimum value of the correlation integrated values are formed.
(X, y) is detected as a motion vector coordinate. Then, the third to nth motion vector detection circuits DET ₃
~ DET _n is the minimum coordinate P (x,
Each motion vector converted from y) into a motion vector for one field is calculated and output.

In the first to n-th motion vector detection circuits DET _{1 to} DET _n , if the motion of the image is linear, the detection accuracy of the motion vector becomes higher as the detection period becomes longer, and DET ₁ <DET ₂ <... <DET _n-1 <DET
Each motion vector with _n detection accuracy is obtained.

Then, each motion vector based on the correlation integrated value between 1 to n fields obtained by the first to _nth motion vector detection circuits DET _{1 to} DET _n is supplied to the acceleration determination circuit ACJ and the selector SEL. To be done.

The acceleration judging circuit ACJ judges the acceleration of the motion of the image on the basis of the motion vectors obtained by the _{first to nth} motion vector detecting circuits DET _{1 to} DET _n . Then, this acceleration determination circuit ACJ
Is the larger the one-field variation of the motion vector,
The motion vector is selected based on the difference value between the short fields, and the smaller the fluctuation of the motion vector for one field is,
The selector SEL is controlled so as to select and output the motion vector based on the difference value between the long fields. That is, when the motion of the image is linear, the first
To the n-th motion vector detection circuit DET _{1 to} DET _n , the temporal change of each motion vector decreases, and the smaller the temporal change of each motion vector, the higher the accuracy based on the difference value between long fields. The selector SEL is controlled so as to select and output the motion vector of.

For example, as shown in FIG. 4, when the motion of the image is non-linear and the motion vector fluctuates for each field, the second to nth motion vector detection circuits DET are used.
_{2 to} DET _n include a large amount of error in the motion vector, so that the motion vector of one field detected by the first motion vector detection circuit DET ₁ is selected to When it is linear, the second to nth motion vector detection circuits DET _{2 to} DE
The selector S is arranged to select and output one of the motion vectors detected by T _n , which has less temporal fluctuation.
Control EL. That is, for example, as shown in FIG.
In the case of an image motion in which the point corresponding to the pixel S _A of the image of the m-th field in the interlaced scan moves to the position corresponding to the pixel S _B of the image of the m + 4 th field, the third vector detection circuit described above. Since the motion vector between the three fields detected by DET ₃ indicates the motion of the image with high accuracy, the motion vector between the one field obtained by converting the motion vector between the three fields is selected and output. Controls the selector SEL.

Then, the motion vector detected by the motion vector detecting device 10 is supplied to the correction amount generating section 20.

The correction amount generation section 20 forms a camera shake correction signal using the motion vector detected by the motion vector detection device 10 as a camera shake vector, and supplies this camera shake correction signal to the correction section 30.

Further, the correction section 30, as shown in FIG. 6, for example, has an address control circuit 31 and a select signal generation circuit 32 to which a camera shake correction signal is supplied from the correction amount generation section 20, and the address control circuit 31. A frame memory 33 and a peripheral memory 34 in which video data is written / read according to an address signal supplied from
And the video data read from the frame memory 33 and the peripheral memory 34, the select signal generating circuit 3
2 and a selector 35 that selectively outputs according to a select signal supplied from 2.

Input video data supplied through the signal input terminal 1 is sequentially written in the frame memory 32. The read address of the frame memory 32 is controlled by the camera shake correction signal according to the camera shake vector. As a result, from the frame memory 32, video data obtained by moving one frame of input video data according to the shake vector is obtained. Then, the video data read from the frame memory 32 and the peripheral video data read from the peripheral memory 33 are combined by the selection by the selector 35 and output from the signal output terminal 2 as video data subjected to the image stabilization process.

In the peripheral memory 33, the video data of the peripheral portion corresponding to the image correction range of the image data subjected to the image stabilization processing output via the selector 35 is sequentially written as peripheral video data.

As described above, the motion vector detecting device 1
Since a motion vector of an image can be detected with high accuracy by setting 0, a camera shake correction device that performs camera shake correction based on this motion vector can secure high camera shake correction accuracy and can obtain a natural image output. ..

[0033]

As is apparent from the above description, in the image motion vector detecting device according to the present invention, the representative point pixel of each block obtained by dividing the image of one field composed of the input video signal into a plurality of blocks is selected. Image data is stored in each of n representative point memories connected in cascade for one field,
N types detected by the vector detection means based on each difference value between the image data of each pixel of the block of the current frame detected by the difference detection means and the image data of the representative point pixel of the block 1 to n fields before Of the motion vector is detected in advance, and the acceleration of the motion of the image is judged by the acceleration judging means based on each motion vector obtained by the above-mentioned motion vector detecting means. Since the motion vector based on the difference value between the motion vectors is selected and output by the motion vector selection means, it is possible to output the motion vector with high accuracy with respect to the linear motion of the image.

Therefore, according to the present invention, it is possible to provide an image motion vector detection device capable of detecting an image motion vector with high accuracy, and to perform high-performance image stabilization in a handy type video camera or the like. You can enable it.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a camera shake correction device provided with an image motion vector detection device according to the present invention.

FIG. 2 is a diagram showing a state of block division of a screen in the motion vector detecting device.

FIG. 3 is a diagram showing a structure of one block of the screen divided into blocks.

FIG. 4 is a diagram showing a motion vector detected by the motion vector detecting device.

FIG. 5 is a diagram for explaining an example of a motion vector detected by the motion vector detecting device.

FIG. 6 is a block diagram showing a configuration of a correction unit of a camera shake correction device provided with the motion vector detection device.

[Explanation of symbols]

10 .......... motion vector detecting device FM ₁ ~FM _n ····· representative point memory SUB ₁ ~SUB _n ··· subtracting circuit DET ₁ ~DET _n ··· motion vector detecting circuit ACJ ... ・ ・ ・ Acceleration judgment circuit SEL ・ ・ ・ ・ ・ ・ ・ ・ Selector

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Ken Horishi 6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Tsukasa Hashino 6-35 Kita-Shinagawa, Shinagawa-ku, Tokyo No. Sony Corporation

Claims (1)

[Claims] 1. A series of n representative point memories connected in cascade, each storing one field of image data of a representative point pixel of each block obtained by dividing an image of one field composed of an input video signal into a plurality of blocks. A difference n between the image data of each pixel of the block of the current frame and the image data of the representative point pixel of the block 1 to n fields before read from each representative point memory is detected.
N difference detection means and n for detecting a motion vector based on each difference value between 1 to n fields detected by each difference detection means
Motion vector detecting means, acceleration determining means for determining the acceleration of the motion of the image based on each motion vector obtained by the motion vector detecting means, and each motion vector obtained by the motion vector detecting means for the acceleration. And a motion vector selecting unit that selectively outputs the motion vector according to the judgment output by the judging unit. As the variation of the motion vector for one field is larger, the motion vector based on the difference value between the shorter fields is calculated by the motion vector selecting unit. An image motion vector detection device characterized by selecting and outputting.