JP2004229084A - Blurring correcting instrument, method therefor and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which has a function to set parameters, such as brightness, white balance, focus distance, or the like, when a recording button is half-pressed, and to form an image when the button is deeply pressed, and also has a blurring correcting function to reproduce the parameters set in a half-pressed state precisely. <P>SOLUTION: The imaging apparatus comprises first and second frame memories in which images are stored, a gain controller connected to the output side of either of the first and second frame memories, a gain control means for controlling brightness levels of respective frame memory images to make them identical, and a controller which extracts partial picture elements out of the first and second frame memory images whose brightness levels are equalized by the gain control means. The apparatus further includes a motion vector detecting means for detecting motion vectors in respective extracted partial picture elements, and a corrective control means for calculating a correction quantity in response to the movement quantity of the motion vector given by the motion vector detecting means and correcting a formed image. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a camera shake correction apparatus, a camera shake correction method, and an image pickup apparatus. The present invention relates to a correction device, a camera shake correction method, and an imaging device.
[0002]
[Prior art]
A recording button (also referred to as a shutter release button) of an imaging device that is a conventional digital camera has two stroke states.
The first is a half-pressed state, in which only parameters such as brightness, white balance, and focal length are determined, and no recording is performed on a recording medium. Second, the captured image is recorded on the recording medium while the parameters determined in the deeply pressed state are maintained.
[0003]
[Patent Document 1]
JP-A-2002-131799 (page 5 FIG. 3)
[0004]
[Problems to be solved by the invention]
However, in the imaging apparatus described in the related art, there is a case where the recording button is shifted from a half-pressed state to a deeply-pressed state by a camera shake or the like due to camera shake or the like. For this reason, even if the brightness, white balance, and focal length can be processed as expected by the photographer, the recorded data when the image is actually photographed is different, so-called image shaking occurs. For this reason, there is a problem that the image cannot be shot as expected with respect to image shaking unless a fixing device such as a tripod is used.
[0005]
Therefore, in an image pickup apparatus having a recording button having half-press and deep-press functions, a camera-shake correction apparatus, a camera-shake correction method, and an image pickup apparatus that can perform recording without causing image shaking when deeply pressed must be solved. Have issues that must be met.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, a camera shake correction apparatus, a camera shake correction method, and an imaging apparatus according to the present invention have the following configurations.
[0007]
(1) The image stabilization apparatus has first and second frame memories and a memory controller that controls the first and second frame memories, and the first and second frame memories are provided at different times. A memory controller for writing a frame image, and a gain controller on one of the output sides of the first and second frame memories, and a luminance level of each frame image stored in the first and second frame memories. And a controller for extracting pixels of a part of each of the frame images stored in the first and second frame memories whose luminance levels are made equal by the gain control means. Motion vector detecting means for detecting a motion vector from each of the extracted partial pixels; Calculates a correction amount in accordance with the moving amount of the obtained motion vector by a correction control means for correcting the captured image obtained when the record button is pressed deep state, is that with the.
(2) The memory control unit writes a frame image in the first frame memory when setting the brightness, white balance, and focal length at least when the recording button is half-pressed, and presses the recording button deeply. The image stabilizing apparatus according to (1), wherein a frame image is written in the second frame memory in the state.
(3) The camera shake correction apparatus according to (1), wherein the motion vector detecting means detects the motion vector by a representative point matching method.
[0008]
(4) In the camera shake correction method, the frame image when the recording button is half-pressed is stored in the first frame memory to set the brightness of the image, and the frame image when the recording button is fully pressed is stored in the second frame memory. The brightness of the image stored in the frame memory is set, the brightness levels of the respective frame images stored in the first and second frame memories are made equal, and the frame stored in the first frame memory is set. A motion vector is detected by comparing a pixel of a part of the image with a pixel of a part of the frame image stored in the second frame memory, and based on information detected by the motion vector, the recording button is pressed deeply. Is to correct the captured image obtained in the case of (1).
[0009]
(5) The imaging apparatus sets at least parameters of brightness, white balance, and focal length when the recording button is half-pressed, and functions to capture an image when the recording button is fully pressed. Storing the frame image when the recording button is half-pressed in the first frame memory and setting the brightness of the image, and storing the frame image when the recording button is fully pressed in the second frame memory A memory controller for setting the brightness of an image, and a gain controller on one of the output sides of the first and second frame memories, each of which is stored in the first and second frame memories. Gain control means for controlling the luminance levels of the frame images to be equal, and the first and second frame memories having the same luminance level by the gain control means. A motion vector detecting means for detecting a motion vector from each of the extracted partial pixels; and a motion vector detecting means for detecting a motion vector from each of the extracted partial pixels. And a correction control means for calculating a correction amount in accordance with the moving amount of the recording button and correcting a captured image obtained when the recording button is in a deeply pressed state.
(6) The imaging device according to (5), wherein the motion vector detection unit detects a motion vector by a representative point matching method.
[0010]
As described above, the frame images at different times, that is, when the recording button is half-pressed, are stored in the first frame memory, and the frame images when the recording button is deeply pressed are stored in the second frame memory. In such a manner, when the state is shifted to the deeply pressed state, the gain control is performed so that the luminance levels of the frame images stored in the first and second frame memories become equal, and the pixels of a part of each frame image are compared. By detecting the motion vector of the frame image when transitioning from the half-pressed state to the deeply-pressed state, based on this motion vector, correcting the captured image obtained in the fully-pressed state, the half-pressed state It is possible to obtain a captured image based on various parameters set in the state.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of a camera shake correction apparatus, a camera shake correction method, and an imaging apparatus according to the present invention will be described below with reference to the drawings.
[0012]
As shown in FIG. 1, a camera shake correction apparatus and an image pickup apparatus capable of realizing a camera shake correction method according to the present invention include an image pickup device 11 for picking up an image of a subject, A / D converter 12 for converting the digital signal into a digital signal, and a microcomputer 20 calculates a gain to obtain an optimal brightness based on information obtained from the luminance detector 15 of the digital signal obtained by the A / D converter 12. A first gain controller 13 for adjusting the gain with the gain; a memory controller 14 for controlling writing and reading of the first frame memory 18 and reading and writing of the second frame memory 19 under the control of the microcomputer 20 with the luminance-adjusted digital signal. A timing generator (TG) 17 for driving the imaging device 11 under the control of the microcomputer 20, and a half of the recording button 23. A first frame memory 18 for storing a frame image at the time of recording, a second frame memory 19 for storing a frame image when the recording button 23 is deeply pressed, and a frame image stored in the second frame memory 19. A second gain controller 21 that adjusts the luminance level of the comparison image to be the same when detecting the motion vector, and detects the motion vector from two frame images of the first frame memory 18 and the second frame memory 19. A motion vector detecting unit 22 that outputs detection information to the microcomputer 20, a luminance detector 15 that outputs luminance information of an image output from the memory controller 14 to the microcomputer 20, and a motion vector obtained by the motion vector detecting unit 22. A corrector 16 for performing correction based on correction information calculated by the microcomputer 20 based on the It is largely constituted by a recording medium 24 for recording the corrected image corrected by righteous 16.
[0013]
As shown in FIG. 2, the motion vector detection unit 22 receives a first input unit 25 for inputting an image of the first frame memory 18 (see FIG. 1) and an image of the second frame memory 19 (see FIG. 1). A second input unit 26, a first filter processing circuit unit 31 for removing a frequency component unnecessary for detecting a motion vector from the image data input by the first input unit 25, and a second input unit A second filter processing circuit 33 for removing frequency components unnecessary for detecting a motion vector from the image data input at 26, and a first frame memory 18 processed at the first filter processing circuit 31. And a representative point extracting circuit unit 32 for performing processing for extracting the representative points from the image and storing the luminance level, and representative points of the image of the first frame memory 18 (see FIG. 1) extracted by the representative point extracting circuit unit 32. Shine An absolute value conversion circuit section 34 for converting a difference between the level and a luminance level of a pixel in a search range of an image of the second frame memory 19 (see FIG. 1) obtained by the second filter processing circuit section 33 into an absolute value; As shown in FIG. 5, the absolute value of the residual obtained in each search range obtained by the value conversion circuit unit 34 is cumulatively added by the cumulative addition circuit unit 35 and the cumulative addition circuit unit 35 for each coordinate. Based on the result, as shown in FIG. 6, a motion vector generating circuit 36 for searching for a point having the highest correlation and generating a motion vector is provided.
[0014]
In the imaging apparatus having such a configuration, the imaging device 11 is driven by a timing generator (TG) controlled by the microcomputer 20. The analog signal output from the imaging device 11 is converted from an analog signal to a digital signal by the A / D converter 12. The digital signal is calculated by the microcomputer 20 based on information obtained from the luminance detector 15 so as to obtain an optimal brightness, and the first gain controller 13 adjusts the gain.
The memory controller 14 is controlled by the microcomputer 20 and performs writing in the first frame memory 18 and reading and writing in the second frame memory 19.
The brightness detector 15 outputs the brightness information of the image output from the memory controller 14 to the microcomputer 20. The corrector 16 performs correction based on the correction information calculated by the microcomputer 20 based on the motion vector obtained by the motion vector detector 22.
The first frame memory 18 stores an image when the recording button 23 is half-pressed. The second frame memory 19 stores an image when the recording button 23 is pressed deeply. The second gain controller 21 is for adjusting the luminance level of the comparison image to be the same when detecting the motion vector. The second gain controller 21 controls the microcomputer 20 based on the luminance information of the image in the half-pressed state and the fully-pressed state. Is controlled by The motion vector detection unit 22 detects a motion vector by comparing pixels extracted from the respective frame images of the first and second frame memories 18 and 19, and outputs the detection information to the microcomputer 20.
[0015]
The motion vector detection unit 22 having the above-described configuration performs camera shake correction using a representative point matching method. The representative point matching method detects a motion vector from images at two different times, past and present. In this case, a method is used to detect which luminance level of a certain pixel (representative point) of the past image is closest to the pixel of which coordinate of the current image.
[0016]
FIG. 3 shows a method of detecting a motion vector by the representative point matching method.
Number 6 indicates the m-th frame (frame image stored in the first frame memory 18), and number 7 indicates the n-th frame (frame image stored in the second frame memory 19). Here, it is necessary that m <n. Number 3 indicates one pixel. Number 1 is a search range in which a motion vector is detected. Number 2 indicates the representative point set in the m-th frame. Number 5 compares the luminance level of each pixel in the search range in the n-th frame with the luminance level of the representative point 2 in the m-th frame. The pixel closest to the luminance level of the representative point 2 among the pixels. The motion vector 4 is detected from the positional relationship between the representative point 2 and the pixel 5 in the m-th frame.
[0017]
FIG. 4 shows a detection area 1 and a plurality of search ranges 2 included in one detection area 1. Number 1 indicates a set of search ranges, and one motion vector is detected from a plurality of search ranges. Number 2 indicates one search range and has one representative point 3 and a pixel for comparison.
[0018]
FIG. 5 shows the absolute value of the residual of each search range obtained by the absolute value conversion circuit unit 34 which converts the difference between the luminance level of the pixel and the brightness level of the pixel in the search range of the frame image in the second frame memory 19 (see FIG. 1). This is for explaining the operation of the cumulative addition circuit unit 35 for cumulatively adding the values for each coordinate, and shows the absolute value of the residual in each search range and its integration.
Number 1 indicates the absolute value of the difference between the representative point in the m-th frame in one search range and each pixel in the n-th frame. The representative point in this search range is different for each coordinate where the search range is arranged.
Numeral 2 indicates the data obtained by integrating the absolute value of the residual in each search range for each same coordinate in the search range. This is to improve the detection accuracy by providing a plurality of search ranges.
[0019]
FIG. 6 illustrates the operation of the motion vector generation circuit 36. Based on the result accumulated by the accumulation circuit 35 shown in FIG. 5, a point having the highest correlation is searched to generate a motion vector. FIG.
That is, the relationship between the shift and the absolute value of the residual is shown, and a motion vector is detected from the position of the point a having the highest correlation (the absolute value of the residual is the smallest) and the representative point.
[0020]
Next, a control flow when the recording button is pressed in the imaging apparatus having the above-described configuration will be described with reference to the block diagrams shown in FIGS. 1 and 2 and the flowcharts shown in FIGS. explain.
[0021]
First, this control flow is called every frame, and it is determined whether or not the recording button 23 is pressed. If the button is pressed, the process proceeds to a process of determining whether the button is half-pressed or deeply pressed. If not, a normal process is performed (steps ST11, ST12, ST13, ST14, ST15).
[0022]
If the recording button 23 is half-pressed in step ST12, the frame image in the half-pressed state is written to the first frame memory 18 as shown in FIG. 8A (step ST31).
Then, the frame image written in the first frame memory 18 is treated as a comparison image in the subsequent motion vector detection.
Next, the same frame image as that written in the first frame memory 18 is input from the memory controller 14 to the luminance detector 13, and the detected value is set as a variable brightA (step ST32).
[0023]
In step ST12 shown in FIG. 7, when the recording button 23 is not in the half-pressed state but in the deep-pressed state, a deep-pressing process is performed (step ST14).
[0024]
In the deep pressing process, as shown in FIG. 8B, first, the frame image in the deep pressing state is written in the second frame memory 19 (step ST21). The frame image written here is treated as a later correction image. Next, the same frame image as the one written in the second frame memory 19 is input from the memory controller 14 to the luminance detector 15, and the detection check here is set to a variable brightB (step ST22).
Next, a gain is determined so that the luminance level of the image in the second frame memory 19 becomes the same as the luminance level of the image in the first frame memory 18 (step ST23).
gain = brightB / brightA
Next, the variable gain determined above is set in the second gain controller 21 (step ST24). This processing is necessary for equalizing the levels of both images for comparing the luminance levels in the representative point matching.
Next, the microcomputer 20 calculates a correction amount from the frame image output from the second gain controller 21 and the second frame memory 19 based on the information detected by the motion vector detection unit 22 (step ST25).
Next, the frame image in the second frame memory 19 is read from the memory controller 14, and correction (partial extraction) is performed by the corrector 16 based on the correction information calculated by the microcomputer 20 (step ST26).
[0025]
FIG. 9 shows the timing of writing to the first and second frame memories 18 and 19 and the timing of writing to the recording medium 24 with time. When the recording button 23 changes to a half-pressed state at time 5, the imaging device The eleventh frame image is written to the first frame memory 18. In the half-pressed state, processing for determining parameters such as brightness, white balance, and focal length is simultaneously performed.
When the recording button 23 changes to a deeply pressed state at time 15, the frame image of the imaging device 11 is written to the second frame memory 19. At time 16, the frame image read from the first frame memory 18 is adjusted by the second gain controller 21 and then input to the motion vector detecting unit 22. At the same time, the frame image read from the second frame memory 19 is input to the motion vector detection unit 22. At time 17, the frame image read from the second frame memory 19 is input to the corrector 16.
Then, the microcomputer 20 corrects the input image by the corrector 16 based on the motion vector information detected at the time 16, and records the corrected image on the recording medium 24.
[0026]
【The invention's effect】
As described above, the camera shake correction apparatus, the camera shake correction method, and the imaging apparatus according to the present invention store the frame image when the recording button is half-pressed in the first frame memory and store the frame image when the recording button is deeply pressed. Are stored in the second frame memory, and when the display is shifted to the deeply pressed state, gain control is performed so that the luminance levels of the frame images stored in the first frame memory become equal. By detecting a motion vector from a frame image when transitioning from the pressed state to the fully pressed state and correcting the captured image obtained in the fully pressed state, it is possible to correct the image shake in the image in the fully pressed state. Thus, there is an effect that an image as expected by the photographer can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram of an imaging apparatus having a camera shake correction function according to the present invention.
FIG. 2 is a block diagram of a motion vector detection unit.
FIG. 3 is an explanatory diagram showing a method for detecting a motion vector by a representative point matching method.
FIG. 4 is an explanatory diagram showing a detection region and a plurality of search ranges included in one detection region.
FIG. 5 is an explanatory diagram showing an absolute value of a residual in each search range and its integration.
FIG. 6 is an explanatory diagram showing the relationship between the shift and the absolute value of the residual, and showing how a motion vector is detected from the position of the point a having the highest correlation and the position of the representative point.
FIG. 7 is a flowchart showing a control flow.
FIG. 8 is a flowchart showing a half-pressed state and a deeply-pressed state.
FIG. 9 is a timing chart showing writing to the first and second frame memories.
[Explanation of symbols]
11; imaging device, 12; A / D converter, 13; first gain controller, 14; memory controller, 15; luminance detector, 16; corrector, 17; timing generator, 18; 2nd frame memory, 20; microcomputer, 21; second gain controller, 22; motion vector detection unit, 23; recording button, 24; recording medium, 25; first input unit, 26; second input unit, 31; 1 filter processing circuit section, 32; representative point extraction circuit section, 33; second filter processing circuit section, 34; absolute value conversion circuit section, 35; cumulative addition circuit section, 36; motion vector generation circuit section.

Claims (6)

A memory control means having first and second frame memories and a memory controller for controlling the first and second frame memories, and writing frame images at different times in the first and second frame memories; ,
A gain controller having a gain controller on one of the output sides of the first and second frame memories and controlling the luminance levels of the respective frame images stored in the first and second frame memories to be equal. Means,
A controller for extracting a part of pixels of each of the frame images stored in the first and second frame memories whose luminance levels are made equal by the gain control means, and calculating a motion vector from each of the extracted parts of the pixels. Motion vector detecting means for detecting,
Correction control means for calculating a correction amount according to the movement amount of the motion vector obtained by the motion vector detection means, and correcting a captured image obtained when the recording button is in a deeply pressed state,
A camera shake correction device comprising: The memory control unit writes a frame image in the first frame memory when setting the brightness, white balance, and focal length at least when the recording button is half-pressed, and when the recording button is fully pressed. 2. The image stabilizing apparatus according to claim 1, wherein a frame image is written in the second frame memory. The apparatus according to claim 1, wherein the motion vector detecting means detects the motion vector by a representative point matching method. The frame image when the recording button is half-pressed is stored in the first frame memory to set the brightness of the image, and the frame image when the recording button is fully pressed is stored in the second frame memory to store the image. The brightness is set, the luminance levels of the respective frame images stored in the first and second frame memories are made equal, and a pixel of a part of the frame image stored in the first frame memory is A motion vector is detected by comparing with a part of the pixel of the frame image stored in the two-frame memory. Based on the information detected by the motion vector, a captured image obtained when the recording button is pressed deeply is displayed. A camera shake correction method characterized by correcting. When the recording button is half-pressed, at least, brightness, white balance, set the parameters of the focal length, an imaging device that functions to take an image when the recording button is fully pressed,
The frame image when the recording button is half-pressed is stored in the first frame memory to set the brightness of the image, and the frame image when the recording button is fully pressed is stored in the second frame memory to store the image. Memory control means for setting brightness;
A gain controller having a gain controller on one of the output sides of the first and second frame memories and controlling the luminance levels of the respective frame images stored in the first and second frame memories to be equal. Means,
A controller for extracting a part of pixels of each of the frame images stored in the first and second frame memories whose luminance levels are made equal by the gain control means, and calculating a motion vector from each of the extracted parts of the pixels. Motion vector detecting means for detecting,
Correction control means for calculating a correction amount according to the movement amount of the motion vector obtained by the motion vector detection means, and correcting a captured image obtained when the recording button is in a deeply pressed state,
An imaging device comprising: The imaging apparatus according to claim 5, wherein the motion vector detection unit detects a motion vector by a representative point matching method.

Images