JPH07264464A - Sensor for image shake - Google Patents

Abstract

PURPOSE:To correct image shake at zooming. CONSTITUTION:A microcomputer 9 allows a memory controller 7 to select control to a representative point memory 3 or a difference memory 5. Thus, a difference between a picture signal of one area in the middle position of an image in a current field via a band limit filter 2 and a picture signal at a representative point of the same area at the middle position of the image in the preceding field having been already stored in the representative point memory 3 is taken and the result is subjected to absolute value processing by an absolute value processing circuit 4 and stored in the difference memory 5. Furthermore, a characteristic quantity arithmetic section 6 calculates a characteristic quantity and a motion vector by using the difference in the difference memory 5 and the accumulated difference resulting from the difference fed to an adder 10 and accumulated thereat and the microcomputer 8 detects a correction quantity of the movement of the image and provides an output of the result.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image shake detecting device for detecting image shake during zooming.

[0002]

2. Description of the Related Art Conventionally, in a handy type image pickup device such as an 8 mm video, that is, a video camera, a subject image is greatly shaken due to some camera shake or vibration, so that an image is difficult to view during reproduction.

In order to eliminate such shaking of the subject image due to camera shake or vibration, a method of detecting and correcting a camera shake component has been proposed. One of the methods is to store the image pickup signal of the screen imaged by the video camera in the field memory, expand and interpolate the image pickup signal of one screen, detect the motion vector from the image pickup signal, and expand according to the motion vector. There is a method of correcting the camera shake component by shifting the read position of the interpolated screen.

A motion vector detecting method based on a representative point matching method is adopted as the above-mentioned motion vector detecting method. In this representative point matching method, a correlation value between a representative point in a block of the current field and a pixel distant from the representative point in the block and the block in the next frame is obtained, and these are cumulatively added to obtain a motion vector. Is to seek.

In the conventional electronic image shake correction system,
The shake of the image is detected using an algorithm that determines the correlation value by performing representative point matching and determines the state of detection of the motion vector based on the state of the correlation value and the detected motion vector. In this case, the area where the representative point matching is performed is the whole screen divided into five areas.

Further, in motion detection by normal image processing, the screen is divided into, for example, 12 regions, and representative point matching is performed in these 12 regions to detect the motion vector of the camera shake component.

[0007]

By the way, in the motion detection by the image shake correction system and the image processing as described above, when the image is enlarged, that is, the zoom operation is performed, the image shake is detected. The motions of the zoom operation are mixed in the motion vectors of the respective regions, and the motions of the respective regions are scattered, so that it becomes impossible to detect only the motion vector caused by the shaking of the image or the hand shake. Specifically, the motion of the zoom operation has different vectors depending on the position of the pixel, and thus it is not possible to detect the shake of the image or the shake component of the image pickup apparatus.

In view of the above situation, the present invention provides an image shake detecting device capable of detecting the image shake during zooming.

[0009]

An image shake detecting apparatus according to the present invention is a zoom information input means for inputting zoom information indicating that a zoom operation has been performed, and the zoom information from the zoom information input means. Depending on, using the motion vector detected for each of the plurality of divided areas using the plurality of divided areas, or switch control whether to use only the motion vector detected from the divided area of the screen center position, The problem described above is solved by having a control unit that detects image shake.

Further, one of the plurality of divided areas is used as the divided area at the center position of the screen.

Furthermore, the motion vector detection for each divided area of the screen is characterized by performing representative point matching.

Here, in the motion vector detection, the first storage means for storing the value of the representative point of the image signal and the first storage means are stored.
Absolute value converting means for converting the difference between the representative point value of the image signal of the previous field and the image signal of the current field stored in the storage means, into the same area output from the absolute value converting means. Second storage means for storing a value obtained by cumulatively adding absolute values; storage control means for controlling the first storage means and the second storage means according to the zoom information;
It is characterized in that it is performed by a feature amount calculation unit that calculates a feature amount and a motion vector from the value stored in the second storage unit.

The zoom information is a signal obtained by operating a zoom button.

[0014]

According to the present invention, one screen is divided into a plurality of areas according to the input zoom information when zooming is not performed, representative point matching is performed for all of these divided areas, and a motion vector is calculated to obtain an image. To detect the shaking of. Also, during zooming, representative point matching is performed on the divided area at the center position of the screen, representative point matching is performed using only the divided area at the center position of the screen, and a motion vector is obtained to detect image shake.

Here, as the divided area at the center position of the screen where the representative point matching is performed at the time of zooming, one area of the plurality of divided areas used for the representative point matching when the zoom is not performed may be used. Alternatively, one divided area set at the center position of the screen may be used.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a schematic configuration of an image shake detecting apparatus according to the present invention.

In the present invention, the representative point matching method is used to detect the motion vector. Here, the representative point matching method will be briefly described. First, one screen displayed on the display device or the like is divided into a plurality of areas, for example, nine areas. These 9 areas are, as shown in FIG.
There are K horizontal points and L vertical points for each region.

In the representative point matching method, the previous field
Representative point in the block and the block above the current field
The correlation value with the pixel away from the representative point in the same block
Ask. Specifically, the above (K ×
Representative included in the (k, l) block in the (L) block
Point value Sⁿ _{(k, l)}(0,0) and (n + 1) fields
(K × L) block in (i,
j) the pixel values S apart^{n + 1} _{(k, l)}Phase with (i, j)
Function p_{(k, l)}Find (i, j). This correlation value p _{(k, l)}
(i, j) is as shown in the following equation (1).

[0019]

[Equation 1]

Furthermore, by cumulatively adding for each deviation over all blocks in the (K × L) block,
The correlation value P (i, j) of the entire divided area is obtained.
The correlation value P (i, j) of the entire region is as follows (2)
Calculated by the formula.

[0021]

[Equation 2]

The motion vector is detected by using the correlation value P (i, j) of the entire area. Further, the motion vector for each area is obtained by performing the above-described operation of obtaining the correlation value P (i, j) for each of the four divided areas.

Next, each structure shown in FIG. 1 will be described. First, from the zoom information input terminal 11 connected to the microcomputer 8, a zoom button ON / OFF switching signal is input as zoom information and sent to the microcomputer 8. This zoom button ON / OF
The F switching signal is a signal sent from the microcomputer of the image pickup apparatus according to whether or not a zoom button provided on the image pickup apparatus (not shown) is pressed. From the microcomputer 8, the zoom button ON / OF
A control signal for switching control of the representative point memory 3 and the difference memory 5 based on the F switching signal is sent to the memory controller 7. Therefore, the memory controller 7
Then, switching control of the area of the data stored in the representative point memory 3 and the difference memory 5 is performed based on the control signal. That is, it is determined whether or not zooming has been performed, and based on the result of the determination, control is performed for the representative point memory 3 to control the storage of the representative point by designating the position of the area where the value of the representative point is obtained. The memory 5 is controlled to store the absolute value and cumulative addition value of the designated area and to output the already stored absolute value and cumulative addition value.

Here, the input circuit 1 inputs an image signal after the signal of the screen imaged by an image pickup device (not shown) is processed, specifically, a luminance signal for television, a control signal, a clock and the like. It is a circuit to do. Input circuit 1
The image signal for one screen input to is sent to the representative point memory 3 which is the first storage means via the band limiting filter 2. When zooming is not performed, the representative point memory 3 sequentially stores representative point values for each of a plurality of divided areas into which the screen has been divided. Further, at the time of zooming, the representative point memory 3 stores the value of the representative point of only one divided area of the screen center position in the field.

The image signal for each field passed through the band limiting filter 2 is also sent to the subtractor 9. When the image signal in one divided area of a certain field is currently sent to the subtractor 9, the value of the representative point in the same divided area as the divided area in the previous field of the field is stored in the memory. The representative point memory 3 is controlled by the controller 7.
Is output to the subtractor 9. The subtractor 9 takes the difference between the sent image signal of the current field and the value of the representative point of the previous field. The difference value thus obtained is sent to the absolute value conversion circuit 4. The absolute value conversion circuit 4 obtains the absolute value of the transmitted difference value. This absolute value is the second
Is sent to and stored in the difference memory 5, which is the storage means of

The absolute value output from the absolute value conversion circuit 4 is also sent to the adder 10. In the adder 10, the absolute value sent from the absolute value conversion circuit 4 and the absolute value sent from the difference memory 5 under the control of the memory controller 7 are cumulatively added for each divided area. The cumulative addition value obtained by this is sent again to the difference memory 5 and stored therein. The cumulative addition value for each divided area stored in the difference memory 5 is sent to the feature amount calculation unit 6.

In the feature quantity computing unit 6, the feature quantity and the motion vector are calculated from the cumulative addition value sent. Here, the feature amount specifically indicates the minimum value, the average value, the neighborhood value of the minimum value, the minimum value, etc. The motion vector is the address of the minimum value. The calculated feature amount and motion vector are sent to the microcomputer 8.
The microcomputer 8 detects the shake component of the image by using the sent feature amount and motion vector. The detected shake component of the image is output as the correction amount of the movement of the captured screen.

Next, the procedure for detecting the shake component of the image according to the zoom information will be described in detail with reference to the flowchart shown in FIG. Here, for simplification of description, it is assumed that the representative point matching is performed for the current field that constitutes one predetermined screen and the previous field of this current field.

First, as shown in step S1, when the microcomputer 8 of FIG. 1 obtains zoom information indicating whether or not the zoom button is pressed, a control signal based on this zoom information is sent to the memory controller 7. To be In this memory controller 7, from the sent control signal,
Switching control is performed as to whether all the divided areas of one screen are used as the divided areas for detecting the motion vector or one divided area of the screen center position is used.

That is, if it is determined that the zoom button has not been pressed, the process proceeds to step S2 and, for example, as shown in FIG.
As shown in a), when the entire screen is divided into, for example, nine areas, all of the nine divided areas are set as areas for detecting motion vectors. However, if it is determined that the zoom button has been pressed and the zoom operation has been performed, step S
3, the one divided area in the center of the screen, which is hardly affected by the movement due to the zoom, is set as the area for detecting the motion vector, as shown in FIG.

When the area for detecting the motion vector is set, the process proceeds to step S4, and the value of the representative point of the image signal in the set area is set to the representative point memory by the switching control of the memory controller 7. 3 is stored. Here, when the zoom button is not pressed, the representative point is stored for each of the divided areas.

Thereafter, in step S5, the image signal currently to be searched within the set divided area,
The difference value from the value of the representative point stored in the representative point memory 3 corresponding to the image signal in the same divided area as the set divided area in the previous field is obtained. This difference value is obtained for all the representative points in the divided area.
Further, when the zoom button is not pressed, the operation for obtaining the difference value is performed in all the set nine divided areas.

Further, in step S6, the difference values in the same divided area thus obtained are integrated, that is, cumulative addition is performed. Here, if the zoom button is not pressed,
The difference values are sequentially integrated with respect to the set 9 divided areas.

Thereafter, in step S7, when the zoom button is pressed, the feature amount and the motion vector of the central area are calculated using the obtained integral value of the central area. When the zoom button is not pressed, the feature value and the motion vector for each divided area are calculated using the integrated value obtained for each divided area.

The characteristic amount and the motion vector are transferred to the microcomputer 8 in step S8. The microcomputer 8 detects the shake component of the image using the sent feature amount and motion vector.

As another method, the representative point memory 3 always stores representative points for all of a plurality of divided areas when zooming is not performed, and when the zoom is not performed, the representative point memory 3 is stored. The representative points of all of the plurality of divided areas stored in are output to the subtractor 9 for each divided area, and at the time of zooming, only the representative points corresponding to one area of the screen center position in the plurality of divided areas are extracted and The memory controller 7 controls so that it is sent to the subtractor 9 at an appropriate timing. Accordingly, at the time of zooming, the cumulative addition value of only one region of the screen center position is obtained in the difference memory 5, and the feature amount calculation unit 6 calculates the feature amount and the motion vector of only one region of the screen center position. Therefore, the microcomputer 8 calculates the shake component of the screen using the feature amount and the motion vector of only one region of the center position of the screen.

Furthermore, the representative point memory 3 always stores representative points for all of the plurality of divided areas when zooming is performed, whether during zooming or not. The memory controller 7 also controls the difference memory 5 so as to store the cumulative addition value for each divided area from the absolute values calculated for all the divided areas. further,
The feature amount calculation unit 6 calculates a feature amount and a motion vector for each divided area from the cumulative addition value for each divided area, and sends the calculated motion amount and motion vector to the microcomputer 8. In the microcomputer 8, it is possible to detect the shake component of the image based on the zoom information from the zoom information input terminal 11 at the time of zooming by using the feature amount and the motion vector of only one divided area of the screen center position. is there.

Further, in the first embodiment described above, during zooming, the motion vector is obtained by directly using one area of a plurality of divided areas which are previously screen-divided for calculating the motion vector when zooming is not performed. However, the divided area when zooming is not performed and the divided area when zooming may be areas having completely different positions or sizes.

That is, for example, as a second embodiment of the present invention, the divided areas when zooming is not performed are 12 areas as shown in FIG. 4A, and when zooming, as shown in FIG. 4B, A new central region is set at a position different from the above-mentioned 12 regions. In this embodiment, the memory controller 7 stores in advance a range of divided areas for detecting a vector that moves when the zoom button is pressed. When the zoom button is pressed, the position of the central area is controlled in the representative point memory 3 so as to store the newly set representative point of the central area.
The difference memory 5 is controlled to store the absolute value based on the obtained representative point and the cumulative addition value of the absolute values. The size of the central region may be the same as the one divided region used when zooming is not performed, or may be a different size.

[0040]

As is apparent from the above description, the image shake detecting apparatus according to the present invention has zoom information input means for inputting zoom information indicating that a zoom operation has been performed, and this zoom information input. Depending on the zoom information from the means, the motion vector detected for each of the plurality of divided areas using the plurality of divided areas is used, or only the motion vector detected from the divided area at the center position of the screen is used. By including the control means for switching whether or not the image is detected and detecting the image shake, the image shake component of the image at the time of zooming can be accurately detected, so that the image shake correction during zooming can be performed.

By using one of the plurality of divided areas as the divided area at the center of the screen,
Since it is not necessary to set a new divided area, the memory control in the control means can be simplified.

Further, the motion vector can be accurately detected by performing the representative point matching for the motion vector detection for each divided area of the screen.

Here, the motion vector detection is performed by the first storage means for storing the value of the representative point of the image signal and the first storage means.
Absolute value converting means for converting the difference between the representative point value of the image signal of the previous field and the image signal of the current field stored in the storage means, into the same area output from the absolute value converting means. Second storage means for storing a value obtained by cumulatively adding absolute values; storage control means for controlling the first storage means and the second storage means according to the zoom information;
By performing the calculation with the feature amount calculation unit that calculates the feature amount and the motion vector from the value stored in the second storage unit, the shake component of the image at the time of zooming can be accurately detected. Image shake correction can be performed.

Since the zoom information is a signal obtained by operating the zoom button, it is possible to accurately obtain information on whether or not zooming has been performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an image shake detection apparatus according to the present invention.

FIG. 2 is a diagram showing a first switching of divided areas of a screen.

FIG. 3 is a flowchart showing a procedure for detecting image shake.

FIG. 4 is a diagram showing a second switching of divided areas of the screen.

[Explanation of symbols]

 1 Input Circuit 2 Band Limit Filter 3 Representative Point Memory 4 Absolute Value Circuit 5 Difference Memory 6 Feature Amount Calculation Section 7 Memory Controller 8 Microcomputer 11 Zoom Information Input Terminal

Claims (5)

[Claims] 1. An image shake detection device for detecting a shake of an image by detecting a motion vector for each divided area of a screen divided into a plurality of areas, wherein zoom information indicates that a zoom operation has been performed. And a motion vector detected for each of the plurality of divided areas in accordance with the zoom information from the zoom information input means and the zoom information from the zoom information input means. An image shake detection apparatus, comprising: a control unit that controls whether to use only a motion vector detected from a divided region at a central position and detects a shake of an image. 2. The image shake detecting apparatus according to claim 1, wherein one of the plurality of divided areas is used as the divided area at the center position of the screen. 3. The image shake detection apparatus according to claim 1, wherein the motion vector detection for each divided area of the screen is performed by representative point matching. 4. The motion vector detection is performed by first storage means for storing the value of the representative point of the image signal, and the value of the representative point of the image signal of the previous field stored in the first storage means and the current value. Absolute value conversion means for converting the difference from the image signal of the field into an absolute value; second storage means for storing a value obtained by cumulatively adding the absolute values in the same region output from the absolute value conversion means; Storage control means for controlling the first storage means and the second storage means according to information, and a feature quantity calculation means for calculating a feature quantity and a motion vector from the values stored in the second storage means. The image shake detecting device according to claim 1, wherein 5. The image shake detection device according to claim 1, wherein the zoom information is a signal obtained by operating a zoom button.