JP2810814B2 - Motion vector detection device and camera-integrated camera shake correction device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting device capable of detecting image signal fluctuation and a camera-integrated camera shake correction device provided with the motion vector detecting device capable of correcting image fluctuation.

[0002]

2. Description of the Related Art A conventional motion vector detecting apparatus divides a supplied image into blocks of a predetermined size, and moves in each direction in each block by a correlation between a previous field image and a current field image. A motion vector extraction method representing a motion is used.

The motion vector extraction method used in the above-described conventional motion vector detection apparatus includes a gradient method obtained from a spatial gradient and a temporal gradient of a luminance value, and a phase correlation obtained from a ratio of a phase term of a Fourier transform coefficient. And a representative point matching method in which the representative point of one image and the other image are obtained from the accumulated value of the absolute value of the inter-field difference. Among them, the hardware scale can be relatively small,
This is a representative point matching method used in the MUSE encoder.

Referring to FIG. 29, a description will be given of a representative point matching method for obtaining a motion vector of the entire supplied field image. First, b × c = p from the entire image
Number of representative points are uniformly selected, and the absolute value of the difference between the fields in the area of the search range of m pixels × n pixels is expressed by the following equation (1).

[0005]

(Equation 1)

[0006] is obtained in multiple values. Where β ⁿ _{d, e} (i,
j) represents the luminance value of the current field image and β
^n-1 _{d, e} (0,0) represents the luminance value of the previous field image. A cumulative function representing the cumulative sum of α _{d, e} (i, j) for each representative point is

[0007]

(Equation 2)

[0008] The displacement amount (i,
A method of using j) as a motion vector is a representative point matching method. The cumulative function α (i, j) represented by the equation (2) has a “mortar” shape centered on the position (i, j) of the motion vector.

[0009]

However, in the above-described conventional motion vector detecting device, the size of the input image is finite, and the input image has passed through a low-pass filter necessary when sampling as a representative point. The subsequent image signal is an image that is affected outside the image area.
In a TV image signal input in real time, the horizontal synchronizing signal and the vertical synchronizing signal have a large influence because the luminance value is at the black level. In a device that detects a motion vector from image correlation, by passing through a low-pass filter,
In an image region where an excessive state in which the image rises from the black level to the image brightness level occurs, as shown in FIG. 30, the shape has a substantially constant rising shape. There is a problem that there is a correlation, and the detected vector moves in a direction toward zero.

The first aspect of the present invention provides a motion vector detecting device capable of accurately detecting that an image is oscillating in view of the above-mentioned problems in the conventional motion vector detecting device. A second aspect of the present invention provides a camera-integrated image stabilization device that includes a motion vector detection device and that can correct swinging of an image.

[0011]

According to a first aspect of the present invention, there is provided an arithmetic unit for calculating a correlation value of an image based on an absolute difference value between a previous field image and a current field image, and a vertical and horizontal brand.
Image area located at a predetermined distance from the King period
A timing signal for generating a timing signal for determining whether to accumulate a correlation value corresponding to an address assigned in a predetermined direction of each pixel by delaying a synchronization signal by a certain period in order to accumulate only correlation values in Means, storage means for accumulating and storing the correlation value calculated by the calculation means based on the timing signal output from the timing signal generation means, and corresponding to a position having an extreme value of the accumulated value stored in the storage means. And a detecting means for detecting a moving vector to be detected.

According to a second aspect of the present invention, a correlation value of an image is calculated by an arithmetic means based on an absolute difference value between a previous field image and a current field image, and the timing signal is generated by a timing signal generating means .
A distance away from the direct and horizontal blanking periods by a predetermined distance
In order to accumulate only the correlation value in the image area at the position, a timing signal for determining whether or not to accumulate the correlation value corresponding to the address assigned in a predetermined direction of each pixel by delaying the synchronization signal by a certain period is provided. occurs, with an extreme value of the timing signal cumulatively stores the correlation values calculated by the calculating means on the basis of a timing signal output from the generating means, the cumulative value stored in the storage means by the detection means in the storage means A motion vector detecting device for detecting a motion vector corresponding to a position, and a correction vector in the vertical and horizontal directions based on the motion vector detected by the motion vector detecting device.
This is achieved by a camera-integrated image stabilization device having a motion vector processing means for outputting a motion vector.

In the motion vector detecting device according to the first aspect of the present invention, the calculating means calculates the correlation value of the image based on the absolute difference value between the previous field image and the current field image, and the timing signal generating means calculates the vertical and horizontal blanking periods. Distance from
Accumulates only correlation values in image areas that are separated by a distance
In order to determine whether or not to accumulate the correlation value corresponding to the address assigned to each pixel in a predetermined direction by delaying the synchronization signal by a certain period, a storage means outputs the timing signal from the timing signal generation means. The correlation value calculated by the calculating means based on the timing signal is accumulated and stored, and the detecting means detects a motion vector corresponding to a position having an extreme value of the accumulated value stored in the storing means.

In the camera-integrated image stabilizing apparatus according to the second aspect of the present invention, the motion vector detecting apparatus calculates the correlation value of the image based on the absolute difference value between the previous field image and the current field image by the calculating means, and generates the timing signal generating means. by,
A certain distance away from the vertical and horizontal blanking periods
In order to accumulate only the correlation value in the image area at the position, the timing signal for determining whether or not to accumulate the correlation value corresponding to the address assigned in the predetermined direction of each pixel by delaying the synchronization signal by a certain period is set. occurs, with an extreme value of the timing signal cumulatively stores the correlation values calculated by the calculating means on the basis of a timing signal output from the generating means, the cumulative value stored in the storage means by the detection means in the storage means A motion vector corresponding to the position is detected, and the motion vector processing means performs correction vectors in the vertical and horizontal directions based on the motion vector detected by the motion vector detection device.
Output file .

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a camera-integrated image stabilizing apparatus provided with a motion vector detecting device according to the first invention and a motion vector detecting device according to the second invention will be described below with reference to the drawings.

FIG. 1 shows a configuration of a main part of an embodiment of a motion vector detecting device according to the first invention.

First, the configuration of the main part in the motion vector detecting device shown in FIG. 1 will be described.

As shown in FIG. 1, the main parts of the motion vector detecting device are a timing signal generating unit 10 as a timing signal generating unit, an adding unit 11 as an arithmetic unit, an accumulative addition memory 12 as a storage unit, and a detecting unit. And the detecting unit 13.

The timing signal generator 10 receives the clock signal CK, the horizontal synchronizing signal HD and the vertical synchronizing signal VD, and generates and outputs each timing signal used for controlling the apparatus.

The timing signal generator 10 outputs a horizontal block address signal XBLK, a vertical block address signal YBLK, a read / write (R / W) signal, a horizontal address signal XADR, and a vertical address signal Y.
The ADR and the block timing signal are output as timing signals, respectively.

The adding section 11 inputs the absolute value output from the absolute value section (described later) to the cumulative addition memory 12.

The cumulative addition memory 12 is connected to the timing signal generator 10 and the adder 11, respectively. The R / W signal, the horizontal address signal XADR, and the vertical address signal output from the timing signal generator 10 are provided. YAD
The absolute value output from the adder 11 is cumulatively added and stored based on the R and the block timing signal, and the stored absolute value (absolute cumulative value) is output to the adder 11 and the detector Output to 13.

The detecting section 13 is connected to the accumulative addition memory 12, receives the accumulative addition value output from the accumulative addition memory 12, and receives the horizontal address signal XADR, Based on the vertical address signal YADR and the block timing signal, the cumulative addition value output from the cumulative addition memory 12 is input to detect and output a motion vector.

Next, each of the above components will be described in detail.

The timing signal generator (hereinafter, referred to as a timing generator) 10 includes a delay unit 14, an address generator 15, and a timing adjuster 16, as shown in FIG.

The delay section 14 constituting the timing generation section 10 will be described in detail with reference to FIGS. 3, 4, and 5.

FIG. 3 shows a circuit configuration of the delay unit 14 configured to delay the horizontal synchronizing signal HD and the vertical synchronizing signal VD. FIGS. 4 and 5 show timing charts for an odd field and an even field, respectively.

A signal defined by the American Television Standards Committee (hereinafter referred to as NTSC signal) and a signal based on the color television broadcasting system developed in West Germany and the United Kingdom (hereinafter referred to as PAL signal). As shown in FIG. 6, the phase of the vertical synchronizing signal VD coincides with the phase of the horizontal synchronizing signal HD in the odd field, but only 0.5 line in the even field, as shown in FIG. It is off. for that reason,
If the address counter is simply reset at the rise of the horizontal synchronizing signal HD and the vertical synchronizing signal VD, the position of the address of the odd field and the position of the address of the even field are shifted by 0.5 line.

From the falling of the horizontal synchronizing signal HD and the vertical synchronizing signal VD, blanking without an image starts.

Horizontal blanking is performed by the Electronic Industries Association (Electronic Industries Association), EIA.
According to RS-170A, the horizontal blanking period is 78 pixels if one line is 455 pixels. That is, when assigning addresses in the horizontal and vertical directions to individual pixels, in order to set the address of the start position of the image to (0, 0), the horizontal synchronizing signal HD and the vertical synchronizing signal VD are set only for the blanking period. Need to be delayed.

The delay circuit shown in FIG. 3 includes counters 17 and 18, flip-flops (hereinafter referred to as FFs) 19 to 21, a logic circuit AND22, a logic circuit NOR23, a logic circuit NAND24, a logic circuit NOT25, and logic circuits OR26 and OR27. It is configured.

The horizontal synchronizing signal HD is a pulse for one clock of negative polarity as shown in FIG. 4, and is connected to the terminal CLR (synchronous clear) of the counter 18 as shown in FIG. Each time this pulse is input, the terminal Q of the counter 18 is reset to "0".

The terminal Q of the counter 18 has 7 bits and N
The input terminal of AND24 has several inverter functions, and NAND24 is a decoder configured to output "0" only at 75 (4BH).

Terminal E of counter 18 (enable terminal)
There When rising signal to the terminal CK when "1" is input, the counter 18 counts up.

When the terminal Q of the counter 18 becomes "0", N
The output of the AND 24 becomes "1", and the counter 18 is enabled.

The terminal Q of the counter 18 counts up by the clock signal CK successively input. When the terminal Q becomes exactly 75, the output of the NAND 24 becomes "0".
, The enable of the terminal E of the counter 18 is inhibited, and the count does not increase any more.

The fall of the output of the NAND 24 is determined by the FF 20 and the O
An edge is detected by R27. This is delayed by one clock by the FF 21 at the next stage, and an HDP signal delayed by 77 clocks from the horizontal synchronization signal HD is obtained.

The vertical synchronizing signal VD is connected to the clear terminal CLR of the counter 17, and the terminal Q of the counter 17 is reset each time the vertical synchronizing signal VD becomes "0".

The AND22 has several inverter functions at its input end, and is "1" when it is exactly 21 (15H).
Is output.

When the terminal Q of the counter 17 becomes "0", A
The output of ND22 becomes "0", and the other input terminal of NOR23 receives the horizontal synchronizing signal HD delayed by 76 clocks.
Only when the value of the terminal 17 becomes "0", the terminal E of the counter 17 is set to "1" to enable it, and the terminal Q is counted up. That is, it counts up every time a pulse delayed by 76 clocks of the horizontal synchronizing signal HD enters. When the value of the terminal Q of the counter 17 becomes 21, the output of the AND 22 changes from "0" to "1".

When the output of AND22 becomes "1", NOR
The output of 23 is always "0" and no longer counts up.

At the same time, the output of NOT 25 changes from “1” to “0”.
Therefore, the pulse is detected by the FF 19 and the OR 26, and as a result, a delay of 77 pulses and a delay of 20 horizontal synchronizing signals HD with respect to the horizontal synchronizing signal HD are obtained.

In the above-mentioned delay unit, the following equation is used.

[0044]

(Equation 3)

The address signal VDP is arranged at the position of the mark in FIG.

When an arbitrary amount of delay is desired, it can be realized by changing the input value of the AND22 and NAND24 decoders.

Assuming that the values of the decoders (AND22 and NAND24) are HN and VN, a delay (75 → HN, 21 → VN) is obtained in the above equation.

The delayed signal HDP and signal VDP
Is input to an address generation circuit described below.

Next, FIG. 8 shows a circuit configuration of the address generator 15 which forms a part of the timing generator. FIGS. 9 and 10 are timing charts of the operation of the address generator 15, respectively, and FIG. 11 is an address mapping diagram.

The address generator 15 shown in FIG.
31, logic circuits AND32, AND33, logic circuit NAND3
4, NAND35, logic circuit NOT36, NOT37, NOT3
8 and a logic circuit OR39.

When the signal HDP output from the delay unit (see FIG. 3) changes from "1" to "0", the output of the AND 32 changes to "0", the terminal Q of the counter 28 is reset, and the address signal is reset. XADR (see FIG. 1) becomes “0”.

The NAND 34 is a decoder having several inverter functions at the input terminal .
It becomes "0" only in the case of F).

Since the clock signal CK is successively input to the counter 28, the counter 28 counts up according to the rising edge of the clock signal CK.

When the count of the counter 28 reaches 23, N
AND34 becomes "0", and the terminal Q of the counter 28 is reset again. At the same time, the output of NOT 36 becomes "1", so that the counter 29 is enabled and counts up.

The counter 29 is reset when the next signal HDP is input. Thus, the counter 28 in the X direction
Is assigned an address in pixel units as the address signal XADR, and the process from 0 to 23 is repeated.

When the count reaches 23, the counter 29 counts up and becomes the block address in the X direction as the block address signal XBLK.

In the Y direction, the signal VDP causes a counter
30 and the counter 31 are reset and counted up by the signal HDP. A pixel address and a block address are assigned by the address signal XADR, the block address signal XBLK, the address signal YADR, and the block address signal YBLK.

Next, the timing adjuster 16 for setting the timing of whether or not to perform accumulation will be described in detail with reference to FIG.

The timing adjustment unit 16 in FIG. 12 includes comparators (hereinafter referred to as CMP) 40 to 43 and logic circuits OR 44 to 4
6. It is composed of logic circuits NAND 47 and NAND 48.

In the timing adjustment section 16 in FIG. 12, when the value in parentheses () in the figure indicates the timing at which the inclined section in FIG. 13 does not accumulate, the output of the OR 46 at that time is “1”.
It is. For other values, the output of OR46 is "0".
, Which indicates the timing at which accumulation is performed.

The position where the accumulation is performed is determined by the input value at each terminal B of the four CMPs 40 to 43 in FIG. The values shown by the arrows in the figure correspond to the timing shown in FIG.

The timing adjustment unit 16 shown in FIG.
Only the values of MP40 to MP43 are block units (in this case, 24
However, if the values of the delay units AND22 and NAND24 (see FIG. 3) for determining the amount of delay of the horizontal / vertical synchronization pulse are changed, fine adjustment on a pixel-by-pixel or line-by-line basis becomes possible. .

The NAND 47 in FIG. 12 is a timing signal CLR having a timing for clearing the contents of the accumulation memory.
B is generated, and when it is "0", the accumulated memory is cleared. The timing signal CLRB includes address signals XADR and YADR.
The clear operation is performed over several lines including all the addresses of the ADR accumulation memory (corresponding to the mesh portions in FIGS. 13 and 14).

When the motion vector detection timing (corresponding to the mesh portion in FIGS. 13 and 14) is "0", the NAND 48 outputs the address signals XADR and YADR corresponding to the smallest value among all the addresses of the accumulation memory. It is detected as a motion vector.

Next, with reference to FIG. 15, the configuration of the portion formed by the adder 11, the cumulative addition memory 12, and the detector 13 shown in FIG. 1 will be described in detail. The operation of the portion of FIG. 15 will be described with reference to the timing chart of FIG.

FIG. 15 shows an adder 11, an accumulative adder memory 12, a detector 13, a logic circuit AND70, flip-flops (FF) 71 and 72, a logic circuit NOT73, and a logic circuit OR.
74 and a comparator (CMP) 75. The detection unit 13 includes flip-flops (FFs) 76 and 77.

The clock signal CK is always supplied, and a write pulse is applied to the terminal R / W of the accumulative addition memory 12 only at the timing signals CLBK (clear timing) and ENACC (accumulated valid timing). When the timing signal CLBK = 0, the output of the adder 11 for performing the cumulative addition is inhibited by the AND 70, and all the addresses of the cumulative memory 12 are cleared by the write pulse.

When the timing signal ENACC = 0,
The sum of the output of the accumulation memory 12 and the output from the absolute value section (described later) is latched by the FF 71 at the falling edge of the clock signal CK, and the content is written into the accumulation memory 12 again.

When the timing signal CLBK = 0, the FFs 72 are all set to 1, and when the signal DETV = 0, the operation of the CMP 75 is reflected on the clock enable terminal CE of the FF 72. If the value of cumulative memory 12
If the value read from is smaller than the clock
And the content of the accumulation memory 12 at that time is written to the FF 72.

At the same time, the FFs 76,
At 77, the address signals XADR and YADR at that time are latched. Finally, the address signals XADR and YADR corresponding to the smallest accumulated value are latched by the FFs 76 and 77.

FIG. 17 shows address signals XADR, Y
FIG. 18 shows the address mapping of ADR and address block signals XBLK and YBLK.
The R / W generated from R and the address signal YADR and the reset timing are shown.

Next, with reference to FIG. 19, a description will be given of the configuration of an embodiment of the motion vector detecting device according to the present invention having the main parts of FIG.

The motion vector detecting device shown in FIG. 19 includes a timing generator 10 (see FIG. 1), an analog / digital (A
/ D) conversion section 78, line interpolation section 79, low-pass filter (LPF) 80, representative point memory 81, representative point line memory 8
2, subtraction unit 83, absolute value unit 84, addition unit 11, cumulative addition memory 1
2 and a detection unit 13.

The operation of each component other than that shown in FIG. 1 will be described below.

The A / D converter 78 converts the image signal input from the input terminal 85 from analog to digital. The line interpolator 79 converts the image signal digitally converted by the A / D converter 78 into a field image. Are linearly interpolated based on the field odd / even judgment signal supplied from the outside to generate a line.

The low-pass filter 80 reduces the influence of noise on the input image signal.

The representative point memory 81 extracts and stores the representative point from the current field signal, and the representative point line memory 82 outputs the representative point of the previous field.

The subtraction unit 83 and the absolute value unit 84 each calculate the correlation between the two fields, and the addition unit 11 and the accumulation memory 12 accumulatively add the accumulation function ρ _{d, e} (i, j).

The detector 13 detects a motion vector from the data in the accumulation memory 12.

Next, the operation of the motion vector detecting device shown in FIG. 19 will be described.

First, the analog image signal input from the input terminal 85 is quantized by the A / D converter 78 by 8 bits, and the line interpolator 79 changes the positional relationship with the odd field up and down by one line (that is, one line). , 0.5 pixels), as shown in FIG. 20, one line of the one-sided field image is generated by linear interpolation processing for two lines. At this time, the interpolation may be performed only on one side field image, and it is determined whether or not to perform the interpolation based on a field odd / even determination signal supplied from the outside.

[0082] LPF80, the luminance value a ⁿ _d to remove aliasing distortion and unwanted noise generated when sampling the representative _points, and outputs the _e (i, j). Denoised image signal at a timing generated by the timing generator 10, is temporarily stored in the representative point memory 81 as representative point a ⁿ _d in the current field _{image, e} (0,0), the luminance of the representative point The value is transferred to the representative point memory 82 in the next vertical synchronization period, and is output to the subtraction unit 83 as the representative point a ^n-1 _{d, e} (0,0) of the previous field image.

At this time, the representative point memory 81 and the representative point memory 81
Is designated by the timing generator 10. Luminance signal to the subtraction unit 83 In this way _{^{a n d, e (i,}} j) and _{^{a n-1 d, e (}} 0,0)
Is input, the difference is calculated, and the absolute value part 84 calculates the absolute difference value ρ _{d, e} (i, j) = | ad ⁿ _{, e} (i, j) −a
^n-1 _{d, e} (0,0) | is calculated. The correlation value calculated by the absolute value unit 84 is cumulatively added to the accumulation memory 12 by the adding unit 11.

For each representative point of the image for one field,
When these series of calculation processes are completed, a cumulative function ρ ⁿ (i, j) corresponding to the address is obtained in the cumulative memory 12, and
The detector 13 calculates the cumulative function ρ ⁿ (i,
An address corresponding to (i, j) at which j) becomes the maximum or minimum value is detected and output as a motion vector.

At this time, the timing of the write and read control signals for the adder 11, the accumulative adder memory 12, and the detector 13 is controlled by the timing generator 10.

FIG. 21 is a diagram for explaining the function of the timing generator 10 described above.

As shown in FIG. 21, a timing generation circuit
Reference numeral 10 denotes counters 86 to 89, an R / W timing extracting unit 90, a reset timing extracting unit 91, a double / single-time timing extracting unit 92, a line memory transfer timing unit 93, a cumulative memory clear block extracting unit 94, and a cumulative invalidation valid. Block extractor 95
And the motion vector detection block extraction unit 96.

A reference clock signal CK is a counter
The signal is input to the terminal CKT 86 and also input to the counter 88. The horizontal synchronization signal HD generated by dividing the frequency of the clock signal CK is input to the terminal CLR of the counter 86 and also to the terminal CLR of the counter 87.

The vertical synchronization signal VD generated by dividing the frequency of the clock signal CK is input to the counter 88 and also to the terminal CLR of the counter 88. Counter 86
Uses the horizontal synchronization signal HD as a reset signal, counts with the clock signal CK, generates and outputs a horizontal address signal XADR within the search range.

The horizontal address signal XADR output from the counter 86 is input to the terminal CKT of the counter 87 and output from the timing generator 10.

The counter 87 further counts the carry of the horizontal address signal XADR input from the terminal CKT of the counter 87 to generate a horizontal block address XBLK.

Similarly, the counter 88 outputs the vertical synchronizing signal VD
Is counted as the reset signal by the horizontal synchronizing signal HD to generate and output the vertical address signal YADR within the search range. The vertical address signal YADR output from the counter 88 is input to the terminal CKT of the counter 89 and output from the timing generator 10.

The counter 89 generates a vertical block address signal YBLK by further counting the carry of the vertical address signal YADR input to the terminal CKT of the counter 89.

Among the generated address signals, the address signal XADR and the address signal YAD within the search range are set.
R is input to an R / W timing extracting unit 90, a reset timing extracting unit 91, a double / single-time timing extracting unit 92, and a line memory transfer timing unit 93, respectively. A timing signal such as a transfer timing to the point line memory is generated.

The block address signal XBLK and the block address signal YBLK are input to the cumulative memory clear block extracting unit 94, the cumulative invalid valid block extracting unit 95, and the motion vector detecting block extracting unit 96, respectively. Various block timing signals such as a cumulative memory invalidation valid signal and a motion vector detection timing are generated.

Referring to FIG. 22, a description will be given of the cumulative invalid / effective extracting section 95 of the timing generating section 10.

The cumulative invalid valid extracting section 95 in FIG. 22 is composed of comparators (CMP) 97 to 100 and a logic circuit NOR101.

The horizontal block address signal XBLK is compared with threshold values Xl and Xr, and the vertical block address signal YBLK is compared with threshold values Yu and Yd. And

That is, the correlation calculation between the images is performed only for the block in which the accumulation memory is effective. By setting the period during which the cumulative memory is valid at a position away from the vertical / horizontal synchronization signal as shown in FIG. 23, it is possible to remove the influence outside the image area.

[0100] The detecting unit 13 shown in FIG. 1 is utilized at the same time the maximum value of the cumulative value, minimum value, and detects and outputs the average value, a microcomputer or the like connected to the subsequent stage (not shown) as a parameter You.

Next, the principle of the motion vector detecting device of the present invention will be described with reference to FIGS.

As shown in FIG. 24, the TV image signal is divided into a vertical synchronizing signal portion, a horizontal synchronizing signal portion and an image portion, and is separated from the vertical synchronizing signal by a distance which is not affected by the low-pass filter in the vertical direction. With a margin, similarly, in the horizontal direction, a margin is set by a distance that is not affected by the low-pass filter, and the representative points are uniformly selected in b columns and c rows in the horizontal direction. Pixels are set vertically for n pixels.

Now, the luminance value of the representative point on the d-th column and the e-th row is extracted from the previous field image, and the value is extracted as a ^n-1 _{d, e} (0,
0) is stored in the memory. The search range of pixels around the representative point from the current field image a ⁿ _{d, e (i,}
j), the absolute value ρ _{d, e} (i, j) of the difference value is

[0104]

(Equation 4)

Is represented by

FIG. 25 is a diagram conceptually showing equation (1). As shown in FIG. 25, (i, j) where ρ _{d, e} (i, j) = 0 is a motion vector candidate.
It is represented by

Absolute value after accumulation

[0108]

(Equation 5)

From the above, the most likely motion vector is

[0110]

(Equation 6)

Is obtained.

FIG. 26 shows a first embodiment of the camera-integrated camera shake correction apparatus according to the second invention.

The camera-integrated camera shake correction device shown in FIG.
CCDTV signal processing unit 102, motion vector detecting device 103 of the first invention, field memory for storing the image of one field before or frame memory 104 for storing the image of one frame before, motion vector detection having BPF for determining correction frequency A motion vector signal processing unit 105 for judging the reliability of an image, an interpolation enlargement processing unit 106 for enlarging and interpolating a part of an image stored in a field or a frame memory
It is constituted by.

Next, the operation of the camera-integrated camera shake correction device of FIG. 26 will be described.

The luminance signal output from the CCDTV signal processing unit 102 is input to a motion vector detecting device 103, and a motion vector is detected from two continuous field or frame image signals. Also, CCDTV signal processing unit
The luminance signal, chrominance signal RY, and chrominance signal BY output from 102 are input to and stored in a field or frame memory 104, and after a delay of one field or one frame, the interpolation / enlargement processing unit 106 It is enlarged. At this time, the motion vector detected by the motion vector detecting device 103 is input to the control terminal of the interpolation / enlargement processing unit 106 via the motion vector signal processing unit 105, and the interpolation / enlargement processing is performed by the amount of the motion vector that is the detected hand shake component. Part 10
T that is corrected vertically and horizontally by 6 to suppress camera shake
A V image signal is obtained.

FIG. 27 shows a second embodiment of the camera-integrated camera shake correction apparatus according to the second invention.

The camera-integrated camera shake correction device shown in FIG.
CCDTV signal processing unit 107, variable delay line 108, motion vector detection device 109 of the first invention, motion vector signal processing unit
Achieved by 110.

Next, the operation of the camera-integrated camera shake correction device of FIG. 27 will be described.

The CCDTV signal processing unit 107 controls the number of high-speed discharge clocks of the vertical charge transfer of the CCD, and the variable delay line 108 controls the parallel movement of the image in the vertical and horizontal directions. The luminance signal of the image output from the CCDTV signal processing unit 107 is input to a motion vector detecting device 109 through a variable delay line 108, and a motion vector is detected from the image signal as in the first embodiment. The detected motion vector, having a BPF for determining a corrected frequency
Is the input processing to the motion vector signal processor 110 that the motion base
The vertical direction correction vector output from the vector signal processing unit 110 is feedback-corrected to the CCDTV signal processing unit 107, and the horizontal direction correction vector is feedback-corrected to the variable delay line 108 to correct camera shake as a closed loop servo.

FIG. 28 shows a third embodiment of the camera-integrated camera shake correction apparatus according to the second invention.

The camera-integrated camera shake correction device shown in FIG.
A lens 111, actuators 112 and 113, a CCD TV signal processing unit 114, a motion vector detecting device 115 of the first invention,
It comprises a motion vector signal processing section 116, a gain corrector 117, and a driver 118.

Next, the operation of the camera shake correction device for the camera-integrated VTR shown in FIG. 28 will be described.

The luminance signal output from the CCDTV signal processing unit 114 is used to detect a motion vector from an image signal by the motion vector detecting device 115 in the same manner as in the first and second embodiments. The detected motion vector has a BPF for determining the correction frequency ,
The input processing is performed on a motion vector signal processing unit 116 that outputs a correction vector .

The lens 111 is supported by the actuators 112 and 113, and has a structure capable of moving two-dimensionally in a direction perpendicular to the optical axis of the lens 111. By applying a voltage to the vertical actuator and the horizontal actuator, The lens 111 moves two-dimensionally. The correction vector output from the motion vector signal processing unit 117 is amplified by the driver 118 in the vertical and horizontal directions, respectively.
When the correction voltage is applied to the actuators 112 and 113, the position of the lens 111 is corrected, and a feedback loop is formed as a whole. However, when the lens 111 has a zoom variable function, the camera shake correction loop gain changes according to the zoom magnification. Therefore, the zoom magnification information is input to the gain corrector 117 to perform gain control in inverse proportion to the zoom magnification.

[0125]

According to the motion vector detecting device of the first invention,
Calculating means for calculating a correlation value of the image based on the absolute difference between the previous field image and the current field image, the vertical and
At a position a predetermined distance away from the horizontal blanking period.
In order to accumulate only the correlation values in the image area, a timing signal for determining whether or not to accumulate the correlation value corresponding to the address assigned in a predetermined direction of each pixel by delaying the synchronization signal by a certain period is generated. a timing signal generating means, and storage means for accumulating and storing correlation values calculated by the calculating means on the basis of a timing signal output from the timing signal generating means, the extreme values of the stored accumulated value in the storage means Since the detection means for detecting the motion vector corresponding to the position is provided, it is possible to accurately detect the motion vector indicating the fluctuation of the image.

[0126] In the camera-integrated hand-shake correction apparatus of the second invention, calculates the correlation value of the image based on the absolute difference between the previous field image and the current field image by computing means, the timing signal generating means, the vertical and Horizontal blankin
Within the image area at a position separated by a predetermined distance from the
In order to accumulate only the correlation value, a timing signal for deciding whether or not to accumulate the correlation value corresponding to the address assigned in the predetermined direction of each pixel by delaying the synchronization signal by a certain period is generated, The correlation value calculated by the calculation means based on the timing signal output from the signal generation means is accumulated and stored, and the motion vector corresponding to the position having the extreme value of the accumulated value stored in the storage means is detected by the detection means. a motion vector detection apparatus to be detected, based on the motion vector detected by the motion vector detecting device
And a motion vector processing means for outputting a correction vector in the vertical and horizontal directions, so that the correction vector can be obtained from the detected motion vector to accurately correct the image shaking.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a main part of an embodiment in a motion vector detection device of the first invention.

FIG. 2 is a block diagram showing a configuration of a timing signal generator of FIG. 1;

FIG. 3 is a block diagram illustrating a configuration of a delay unit in FIG. 1;

FIG. 4 is a timing chart in the delay unit of FIG. 3 in an odd field.

FIG. 5 is a timing chart in the delay unit of FIG. 3 for an even field.

FIG. 6 is a characteristic diagram illustrating phases of a vertical synchronization signal and a horizontal synchronization signal in an odd field and an even field.

FIG. 7 is an explanatory diagram of an arrangement position of an address signal.

FIG. 8 is a block diagram showing a configuration of an address generator of FIG. 2;

FIG. 9 is a timing chart at the time of operation of an address generation unit.

FIG. 10 is a timing chart at the time of operation of an address generation unit.

FIG. 11 is a characteristic diagram showing address mapping.

FIG. 12 is a block diagram illustrating a configuration of a timing adjustment unit in FIG. 2;

FIG. 13 is an explanatory diagram of accumulation timing.

FIG. 14 is an explanatory diagram of another accumulation timing.

FIG. 15 is a block diagram showing a configuration of a part formed by an adding unit, a cumulative addition memory, and a detecting unit shown in FIG. 1;

FIG. 16 is a timing chart for explaining the operation of the part shown in FIG. 15;

FIG. 17 is a characteristic diagram showing address mapping of address signals XADR, YADR and address block signals XBLK, YBLK.

FIG. 18 shows an address signal XADR and an address signal YAD.
FIG. 9 is a characteristic diagram showing R / W generated from R and reset timing.

FIG. 19 is a block diagram showing a configuration of one embodiment of a motion vector detecting device according to the present invention including the main part of FIG. 1;

FIG. 20 is an explanatory diagram of a linear interpolation process.

FIG. 21 is an explanatory diagram of the function of the timing generator of FIG. 2;

FIG. 22 is an explanatory diagram of a cumulative invalid / effective extracting unit in FIG. 21;

FIG. 23 is an explanatory diagram of a set position of a cumulative memory valid period.

24 is an explanatory diagram of the principle of the motion vector detection device in FIG.

FIG. 25 is another explanatory diagram of the principle of the motion vector detecting device in FIG. 19;

FIG. 26 is a block diagram showing a configuration of a first embodiment of the camera-integrated camera shake correction apparatus according to the second invention.

FIG. 27 is a block diagram showing a configuration of a second embodiment of the camera-integrated camera shake correction apparatus according to the second invention.

FIG. 28 is a block diagram showing a configuration of a third embodiment of the camera-integrated camera shake correction apparatus according to the second invention.

FIG. 29 is a block diagram for explaining detection of a motion vector by a conventional motion vector detection device.

FIG. 30 is an explanatory diagram of a problem in a conventional motion vector detection device.

[Explanation of symbols]

 10 Timing signal generator 11 Adder 12 Cumulative addition memory 13 Detector

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Yasukun Yamane 22-22 Nagaike-cho, Abeno-ku, Osaka-shi, Osaka Inside Sharp Corporation (58) Field surveyed (Int.Cl. ⁶ , DB name) H04N 5/232

Claims (2)

(57) [Claims] A calculating means for calculating a correlation value of the image based on an absolute difference value between a previous field image and a current field image; and a predetermined distance from a vertical and horizontal blanking period.
Only the correlation values in the image area
Rubeku, a timing signal generating means for generating a timing signal for determining whether to accumulate the correlation values corresponding to the address allocated to the predetermined direction of each pixel delaying the sync signal a predetermined period, said timing signal generator Storage means for accumulating and storing the correlation value calculated by the calculating means based on the timing signal output from the means, and a position corresponding to a position having an extreme value of the accumulated value stored in the storage means A motion vector detecting device, comprising: detecting means for detecting a motion vector. 2. A calculating means calculates a correlation value of the image based on an absolute difference value between a previous field image and a current field image, and a timing signal generating means calculates a vertical and vertical correlation values of the image.
At a position a predetermined distance away from the horizontal blanking period.
In order to accumulate only the correlation values in the image area, a timing signal for determining whether or not to accumulate the correlation value corresponding to the address assigned in a predetermined direction of each pixel by delaying the synchronization signal by a certain period is generated. Storing the correlation value calculated by the calculation means based on the timing signal output from the timing signal generation means by storage means, and storing the accumulated value stored in the storage means by detection means. A motion vector detecting device that detects a motion vector corresponding to a position having an extreme value, and a motion vector detecting device that moves up and down based on the motion vector detected by the motion vector detecting device.
And a motion vector processing means for outputting a correction vector in the horizontal direction .