JP2007019743A - Image blur detector and electronic camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image blur detector capable of calculating an accurate motion vector even in the case that an electronic camera is moved at a high speed when imaging or a zoom magnification is changed at a high speed and the size of an object image is changed, and the electronic camera loaded with such an image blur detector. <P>SOLUTION: The image blur detector comprises: a resize circuit 107 for magnifying the prescribed region of image data of a preceding frame so as to be matched with the image size of a region corresponding to the prescribed region of the preceding frame in the image data of a succeeding frame and outputting the magnified image data of the preceding frame and the image data of the succeeding frame; and a motion vector calculation circuit 108 for calculating the motion vector relating to the prescribed region on the basis of the image data relating to the preceding frame and the image data of the succeeding frame outputted from the resize circuit 107. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image blur detection device that detects image blur in an image obtained by imaging and an electronic camera equipped with such an image blur detection device.

  In recent years, electronic cameras have become more sophisticated, and not only a still image shooting function but also a moving image shooting function has become essential. Here, in an electronic camera that has been reduced in size and weight, image blur due to camera shake or the like is likely to occur when shooting by hand.

  In addition, an electronic imaging technique such as an electronic camera is applied to a vehicle-mounted camera, an endoscope, and the like, and an image blur correction function is essential in these fields.

Here, as a method of motion detection when performing conventional image blur correction, there is a known method such as a representative point matching method. Further, for example, Patent Document 1 proposes various motion detection methods so that favorable motion detection can be performed without being affected by the state of the subject. In the method disclosed in Patent Document 1, a motion vector detection device that obtains a motion vector by relatively comparing two fields in a video signal, and representative point control that controls a representative point for obtaining a motion vector based on a representative point input command. A circuit and a representative point setting circuit for setting the representative point in the motion vector detecting device based on the control of the representative point control circuit are provided. In such a configuration, the representative point input command is instructed from the pattern generation circuit based on the pattern of the representative point selected by the photographer, and the representative point control circuit is the representative point selected by the photographer. An effective representative point position for motion vector detection is determined based on the point pattern.
Japanese Patent No. 2892685

  In motion detection as in the method of Patent Document 1, accurate motion vector detection cannot be performed when the size of the subject changes between two consecutive frames. In this case, the image blur cannot be corrected accurately. For example, the size of the subject image may change drastically in the video of a camera mounted on a vehicle moving at high speed or the video obtained when an endoscope is inserted. In this case, there is a risk that the image may be difficult to view without performing appropriate image blur correction.

  The present invention has been made in view of the above circumstances, and an accurate motion vector can be obtained even when the size of a subject image changes due to the electronic camera moving at a high speed during imaging or the zoom magnification changing at a high speed. An object of the present invention is to provide an image blur detection device capable of calculating the above and an electronic camera equipped with such an image blur detection device.

  In order to achieve the above object, an image blur detection apparatus according to the first aspect of the present invention uses a predetermined area of image data of a preceding frame as an image size of an area corresponding to the predetermined area in the image data of a subsequent frame. A resizing circuit that scales to match the image, a scaled preceding frame image data output from the resizing circuit, and a motion for calculating a motion vector related to the predetermined area based on the succeeding frame image data And a vector calculation circuit.

According to the first aspect, the image size of the predetermined area of the image data of the preceding frame is scaled to match the image size of the image data of the subsequent frame, and based on the image data in which these image sizes are combined. A motion vector is calculated.
Thereby, even when there is a change in the size of the subject between two consecutive frames, accurate motion vector calculation can be performed.

  In order to achieve the above object, an electronic camera according to a second aspect of the present invention has a light receiving surface and converts an object image formed on the light receiving surface by an optical system into an electric signal. And an image memory for storing image data from the image sensor, and an image blur detection according to the first aspect for calculating the motion vector using the image data stored in the image memory as image data relating to the preceding frame. And an image blur correction circuit that corrects an image blur based on a motion vector calculated by the image blur detection device.

  According to the second aspect, the image blur can be accurately corrected based on the motion vector accurately calculated by the image blur detection apparatus of the first aspect. Therefore, an image blur can be corrected accurately and a good image can be taken without being affected by the state of the subject image and the shooting situation.

  According to the present invention, even when the electronic camera moves at a high speed during imaging or the zoom magnification changes at a high speed, the motion vector can be accurately calculated even when the size of the subject image changes. A detection device and an electronic camera equipped with such an image blur detection device can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram showing a configuration of an electronic camera according to the first embodiment of the present invention. The electronic camera shown in FIG. 1 is an example mounted on a moving body such as an automobile.

  In FIG. 1, an optical system 101 forms a subject image on a light receiving surface of a solid-state image sensor 102 disposed behind the subject image. 1 shows an example including a zoom lens capable of zooming, the optical system 101 of the first embodiment may not include a zoom lens. The solid-state imaging device 102 converts an object image formed on the light receiving surface via the optical system 101 into an electric signal by photoelectric conversion. The image sensor drive circuit 103 outputs an analog electric signal from the solid-state image sensor 102 at every predetermined image capture timing. The signal processing circuit 113 converts the electrical signal output from the solid-state image sensor 102 into digital image data. The image memory 104 stores the image data output from the signal processing circuit 113.

  A speed sensor 105 serving as a moving speed detection unit detects a relative moving speed between the subject and the solid-state image sensor 102. The speed sensor 105 receives the drive signal output from the image sensor drive circuit 103 in synchronization with the imaging timing of the solid-state image sensor 102 and detects the moving speed. A scaling factor calculation circuit 106 calculates a scaling factor at the time of resizing the image data of the preceding frame in the resizing circuit 107 described later, from the relative speed between the subject detected by the speed sensor 105 and the solid-state imaging device 102. The resizing circuit 107 resizes a predetermined area of the image data of the preceding frame stored in the image memory 104 according to the scaling factor calculated by the scaling factor calculation circuit 106. This resizing will be described in detail later. The motion vector calculation circuit 108 uses the image data of the predetermined area of the preceding frame resized by the resizing circuit 107 and the image data of the temporally subsequent frame output from the signal processing circuit 113 (image data of the subsequent frame). A motion vector is calculated. The motion vector calculation circuit 108 includes a compensation circuit (not shown) for compensating for a delay between the resized image data of the preceding frame and the image data of the subsequent frame. As this compensation circuit, for example, a memory can be considered.

  An image data read control circuit 109 as an image blur correction circuit is a so-called electronic blur correction that controls the reading start position of data from the image memory 104 based on the motion vector calculated by the motion vector calculation circuit 108, or for recording. Various image processing such as compression processing is performed. The display device 110 displays the image data processed in the image data read control circuit 109 as a video. The memory card 111 records the image data processed in the image data read control circuit 109.

Next, the operation of the first embodiment will be described.
The solid-state image sensor 102 converts a subject image formed on the light receiving surface via the optical system 101 into an electrical signal. The image sensor drive circuit 103 inputs a vertical synchronization signal and a horizontal synchronization signal to the solid-state image sensor 102 and outputs an electric signal from the solid-state image sensor 102. The vertical and horizontal synchronization signals are also input to the signal processing circuit 113 and the speed sensor 105 in order to synchronize with the output operation in the solid-state image sensor 102. The electric signal output from the solid-state image sensor 102 is converted into digital image data by the signal processing circuit 113. The image memory 104 temporarily stores the image data input from the signal processing circuit 113. Further, in synchronization with the output operation in the solid-state image sensor 102, the speed sensor 105 detects the relative movement speed of the subject and the moving body. A scaling factor calculation circuit 106 calculates a scaling factor from a moving speed at the time of imaging image data of a subsequent frame among two consecutive frames. Here, for example, when the moving speed is 0 or not so fast, the scaling factor is increased as the moving speed increases in the direction approaching the subject. Further, the moving speed is decreased as the moving speed increases in the direction away from the subject.

  The image data of the preceding frame stored in the image memory 104 is input to the resizing circuit 107, and the image data of the subsequent frame from the signal processing circuit 113 is input to the motion vector calculation circuit 108.

Here, resizing processing in the resizing circuit 107 and motion vector calculation in the motion vector calculating circuit 108 will be described.
FIG. 2 is a diagram showing a state of image comparison when a motion vector is detected by the electronic camera of FIG. In FIG. 2, reference numeral 201 is an image of a preceding frame, and reference numeral 202 is an image of a subsequent frame. Since the electronic camera in FIG. 1 is moving, the size of the subject changes between when the preceding frame image 201 is captured and when the subsequent frame image 202 is captured. In this case, even if a motion vector is detected from these two frames of images, it cannot be detected accurately. Therefore, in the first embodiment, a resize process is performed in which a predetermined area corresponding to the image of the subsequent frame is cut out from the image of the previous frame, and the size of the image in the cut out predetermined area is matched with the size of the image of the subsequent frame. Is performed in the resizing circuit 107.

  For this purpose, first, based on the scaling factor obtained by the scaling factor calculation circuit 106, an image 203 of a predetermined area necessary for creating an image having the same size as the image 202 of the subsequent frame is cut out from the image 201 of the preceding frame. . For example, when the scaling factor is larger than 1, the size of the image 203 in the predetermined area is smaller than the size of the image 201 in the preceding frame, and thus the image 203 in the predetermined area can be cut out from the center of the image 201 in the preceding frame. it can. The resizing circuit 107 generates a scaled image 204 by scaling (enlarging) the cut-out image according to the scaling ratio obtained by the scaling ratio calculating circuit 106.

  When the scaling factor is 1, the image 203 in the predetermined area has almost the same size as the image 201 of the preceding frame, and image clipping and image scaling are not necessary.

  Further, when the scaling factor is smaller than 1, the size of the image 203 in the predetermined area is larger than the size of the image 201 in the preceding frame. In this case, the resizing circuit 107 creates the scaled image 204 using the entire image 201 of the preceding frame. However, in this case, peripheral image data is lost. In this case, the motion vector may be detected using image data near the center including the subject.

  After the resizing process in the resizing circuit 107, the motion vector calculating circuit 108 compares the image 202 of the subsequent frame stored in the image memory 104 with the scaled image 204 generated in the resizing circuit 107 and calculates a motion vector. To do. Note that the motion vector may be calculated using a technique such as representative point matching, or may be calculated using a correlation function.

  The image data read control circuit 109 controls the read position of the image data from the image memory 104 based on the motion vector calculated by the motion vector calculation circuit 108. That is, the image data read control circuit 109 detects the image blur in the image by moving the image data read start position in a direction to cancel the image blur according to the motion vector calculated by the motion vector calculation circuit 108. Correct above.

  By such a method, even if an electronic camera is mounted on a moving body that moves at high speed, a motion vector can be correctly calculated, and appropriate image blur correction can be performed.

  After performing the image blur correction, the image data read control circuit 109 performs various image processes for displaying or recording an image. The display device 110 displays the image data processed in the image data read control circuit 109 as a video. The memory card 111 records the image data processed by the image data read control circuit 109.

  As described above, according to the first embodiment, even when the electronic camera is mounted on the moving body and the size of the subject differs between two consecutive frames due to the movement of the moving body, the electronic camera moves correctly. The vector can be detected, and appropriate image blur correction can be performed.

In the first embodiment, the scaling factor is calculated using the speed sensor 105. However, the scaling factor may be obtained using an acceleration sensor or the like. Further, the capacity of the image memory 104 is at least two frames. As an alternative to the above-described compensation circuit, both the image data of the preceding and succeeding frames may be stored and the image data of the succeeding frame may be output from the image memory 104.

[Second Embodiment]
A second embodiment of the present invention will be described with reference to FIG. FIG. 3 is a block diagram showing a configuration of an electronic camera according to the second embodiment of the present invention.

  3 differs from FIG. 1 in that a counter circuit 112 as a zoom parameter detection unit is provided instead of the speed sensor 105, and that the optical system 101 includes a zoom lens capable of zooming.

  The optical system 101 is lens-driven when the zoom ratio is changed by the user. At this time, the optical system 101 outputs a pulse signal indicating a parameter related to zoom to the counter circuit 112. The counter circuit 112 counts the pulses input from the optical system 101, and synchronizes this count value with the imaging timing of the solid-state imaging device 102 (that is, the timing at which the electrical signal readout instruction signal is output from the imaging device driving circuit 103). And output to the scaling factor calculation circuit 106.

  The scaling factor arithmetic circuit 106 compares the input value from the counter circuit 112 when the preceding frame image is captured with the input value from the counter circuit 112 when the subsequent frame image is captured, and performs resizing processing in the resizing circuit 107. Calculate the scaling factor. The scaling factor can be obtained, for example, from an input value from the counter circuit 112 at the time of capturing an image of the subsequent frame with respect to an input value from the counter circuit 112 at the time of capturing the image of the preceding frame. Here, when the variable magnification exceeds 1, the lens is driven in the tele side direction, and when the variable magnification is smaller than 1, the lens is driven in the wide side direction.

  Note that the resizing process and the image blur correction after the scaling factor is calculated are the same as those in the first embodiment.

  As described above, according to the second embodiment, even when the zoom rate of the optical system 101 changes at high speed, a motion vector can be detected correctly, and appropriate image blur correction can be performed. .

Although the present invention has been described based on the above embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention. For example, the display device 110 may be omitted in the above-described embodiment. The memory card 111 may be another recording medium such as a hard disk.
Furthermore, the scaling factor calculation circuit 106, the resizing circuit 107, the motion vector calculation circuit 108, and the counter circuit 112 may be configured by hardware or software.
Further, the first embodiment and the second embodiment may be used in combination.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be extracted as an invention.

It is a block diagram which shows the structure of the electronic camera which concerns on the 1st Embodiment of this invention. It is a figure which shows the mode of the image comparison at the time of detecting a motion vector with the electronic camera of FIG. It is a block diagram which shows the structure of the electronic camera which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Optical system, 102 ... Solid-state image sensor, 103 ... Image sensor drive circuit, 104 ... Image memory, 105 ... Speed sensor, 106 ... Variable magnification arithmetic circuit, 107 ... Resizing circuit, 108 ... Motion vector calculation circuit, 109 ... Image Data read control circuit, 110 ... display device, 111 ... memory card, 112 ... counter circuit, 113 ... signal processing circuit

Claims (5)

A resizing circuit that scales a predetermined area of the image data of the preceding frame so as to match an image size of an area corresponding to the predetermined area in the image data of the subsequent frame;
A motion vector calculation circuit that calculates a motion vector related to the predetermined region based on the image data related to the preceding frame output from the resizing circuit and the image data of the subsequent frame;
An image blur detection apparatus comprising:   The image blur detection apparatus according to claim 1, further comprising a scaling factor calculation circuit that calculates a scaling factor between a predetermined region of the image data of the preceding frame and a predetermined region of the image data of the subsequent frame. An image sensor that has a light receiving surface and converts an object image formed on the light receiving surface by an optical system into an electrical signal;
An image memory for storing image data from the image sensor;
The image blur detection device according to claim 1, wherein the motion vector is calculated using the image data stored in the image memory as image data related to the preceding frame;
An image blur correction circuit for correcting image blur based on a motion vector calculated in the image blur detection device;
An electronic camera comprising: The image blur detection device includes a movement speed detection unit that detects a movement speed between a subject and the imaging device,
4. The electronic camera according to claim 3, wherein the scaling factor calculation circuit calculates the scaling factor from a detection result of a moving speed detection unit at the time of acquiring image data of the subsequent frame. The image blur detection device includes a zoom parameter detection unit that detects a parameter related to zoom of the optical system in image data,
4. The zoom factor calculation circuit according to claim 3, wherein the zoom factor calculating circuit calculates the zoom factor from a parameter relating to the zoom in the image data of the preceding frame and a parameter relating to the zoom in the image data of the subsequent frame. Electronic camera.

Images