JP2008245146A - Imaging apparatus, and control method of same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging technique by which consecutive photographing can be performed without being affected by recovery processing of blurring for a photographed image. <P>SOLUTION: In accordance with the amount of blurring just after an image is photographed, no recovery A, simplified recovery processing B, full-scale recovery processing, or recovery disability D is discriminated (steps S202-S204) and each of processing is carried out. When the next release operation (photographing start) is detected during the execution of the full-scale recovery processing C that takes a long processing time (steps S209-S212), the recovery processing of blurring is stopped (steps S213, S214) and shifted to the next photographing operation. The stopped recovery processing is executed in following reproduction as needed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging technique, and for example, relates to a technique effective when applied to an imaging apparatus, an image restoration method, and the like that detect camera shake during imaging and restore a blurred image to an image without blur.

An imaging apparatus such as a digital camera or a video camera is required to have a camera shake correction function that corrects a deteriorated image caused by camera shake during imaging and restores an image close to the original image.
For example, in a digital camera (hereinafter abbreviated as “camera” as appropriate), camera shake correction for a still image or the like is performed by detecting a camera shake locus using an angular velocity sensor or the like and performing predetermined imaging after imaging based on the detected shake locus. There is known a technique for performing the image restoration calculation (such as a method using a point spread function (PSF)) (for example, Patent Document 1).

As a process for restoring the blurred image based on a method such as PSF as described above, the method of automatically and continuously executing it immediately after shooting is the simplest, and is suitable for an inexpensive popular price range camera.
However, in the case of such a popular price range camera, if processing is performed with a processing circuit that is relatively inexpensive and has a low processing capacity that can be mounted on the camera from the viewpoint of cost, it takes a processing time of several seconds and continues to shoot. If it is desired, it is difficult to perform shooting in parallel with the restoration process.

As a result, there is a technical problem that there is a concern that the next photo opportunity may be removed due to the waiting time until the restoration process is completed.
JP 2006-229392 A

An object of the present invention is to provide an imaging technique capable of continuous shooting without being affected by a blur restoration process for a shot image.
Another object of the present invention is to provide an imaging technique in which image restoration of a shake is performed simply and automatically and does not miss a photo opportunity.

  Another object of the present invention is to provide a technique for improving the operability of an imaging apparatus having a function of restoring motion of a captured image.

According to a first aspect of the present invention, an image sensor that acquires image data of a subject,
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to acquire the image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Including
When the control unit detects an exposure start operation of the image sensor during execution of the restoration operation by the restoration unit, the control unit stops the restoration operation and executes an exposure operation of the image sensor. provide.

According to a second aspect of the present invention, in the imaging device according to the first aspect,
The control logic of the control means provides an imaging apparatus that stores the blur locus information together with the image data before restoration in the storage medium when the restoration operation is stopped and an exposure operation is performed.

According to a third aspect of the present invention, in the imaging device according to the first aspect,
The control logic of the control means reads the image data whose restoration operation has been stopped from the storage medium and the blur locus information related to the image data, restores the image data by the restoration means, and then stores the image data in the storage medium. Provided is an image pickup apparatus that performs an operation of recording again at a predetermined timing after the exposure operation of the image pickup element is completed.

According to a fourth aspect of the present invention, there is provided an image sensor that acquires image data of a subject,
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration by executing at least one of a first process and a second process having different required times based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to obtain the image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Including
The control unit stops the second process by the restoration unit and executes the first process when detecting an exposure start operation to the image sensor during the restoration operation using the second process by the restoration unit. Provided is an imaging device provided with control logic.

According to a fifth aspect of the present invention, in the imaging device according to the fourth aspect,
The time required for the restoration operation of the restoration process of the image data is provided by the imaging apparatus in which the execution time of the first process is shorter than the execution time of the second process.

According to a sixth aspect of the present invention, in the imaging device according to the fourth aspect,
In the restoration operation using the first process, edge enhancement processing of the image data is performed, and in the restoration operation using the second process, an inverse function of the point spread function is calculated based on the blur locus information. Thus, an image pickup apparatus that performs the restoration process of the image data is provided.

According to a seventh aspect of the present invention, there is provided an image sensor that acquires image data of a subject,
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration by executing at least one of a first process and a second process having different required times based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to obtain image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Including
When the control unit detects an exposure start operation to the image sensor during the execution of the restoration operation using the second process by the restoration unit, the elapsed time of the restoration operation being performed has passed a predetermined time. Control logic that causes the restoring means to continue the restoring operation being executed if it is present, and to stop the restoring operation being executed if the elapsed time is less than a predetermined time and to execute the restoring operation using the first process An imaging apparatus including the above is provided.

An eighth aspect of the present invention is the imaging device according to the seventh aspect,
The time required for the restoration operation of the restoration unit by the restoration unit is to provide an imaging apparatus in which the execution time of the first process is shorter than the execution time of the second process.

According to a ninth aspect of the present invention, in the imaging device according to the seventh aspect,
In the restoration operation using the first process, edge enhancement processing of the image data is performed, and in the restoration operation using the second process, an inverse function of the point spread function is calculated based on the blur locus information. Thus, an imaging apparatus in which the image data is restored is provided.

A tenth aspect of the present invention is an image sensor that acquires image data of a subject,
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to obtain image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Reproducing means for reading out and reproducing image data from the storage medium;
Have
When the control unit detects an exposure start operation of the image sensor during execution of the restoration operation by the restoration unit, the control unit stops the restoration operation, and the blur locus information together with the image data that has not been restored is stored in the storage medium. After recording, the image sensor is exposed,
When the reproduction unit reads the image data that has not been restored from the recording medium, the reproduction unit causes the restoration unit to perform a restoration operation of the image data based on the blur locus information of the image data, and the restored image data An imaging apparatus for recording on the recording medium is provided again.

An eleventh aspect of the present invention is a first step of continuously obtaining image data from a subject and restoring the blur in the image data;
A second step of stopping the restoration process and proceeding to the next exposure operation when an exposure start operation of the next image data is detected during the restoration process;
A method for controlling an imaging apparatus including the above is provided.

According to a twelfth aspect of the present invention, in the control method for an imaging apparatus according to the tenth aspect,
In the second step, when the restoration operation is stopped and the next exposure operation is performed, an image pickup apparatus that stores, in the storage medium, blur locus information used in the restoration operation together with the image data before the restoration. A control method is provided.

According to a thirteenth aspect of the present invention, in the control method for an imaging device according to the twelfth aspect,
Further, the operation of reading out the image data whose restoration operation has been stopped from the storage medium and the blur locus information related to the image data, restoring the image data by the restoration means, and then recording the image data again on the storage medium, Provided is a method for controlling an imaging apparatus including a third step that is performed at a predetermined timing after the end of an exposure operation.

A fourteenth aspect of the present invention is a first step in which image data acquisition from a subject and blur recovery processing in the image data are continuously performed;
A second step of switching the restoration process between a first process and a second process different from each other when an exposure start operation of the next image data is detected during the restoration process;
A method for controlling an imaging apparatus including the above is provided.

According to a fifteenth aspect of the present invention, in the control method for an imaging apparatus according to the fourteenth aspect,
The execution time of the image data restoration operation by the first process is shorter than the execution time of the image data restoration operation by the second process,
In the second step, there is provided a control method of the imaging apparatus for switching from the second process having a long execution time to the first process having a short real time.

According to a sixteenth aspect of the present invention, in the control method for an imaging apparatus according to the fourteenth aspect,
In the restoration operation using the first process, edge enhancement processing of the image data is performed, and in the restoration operation using the second process, an inverse function of the point spread function is calculated based on the blur locus information. Thus, a method for controlling the imaging apparatus in which the image data is restored is provided.

According to the image pickup apparatus and the control method thereof according to the present invention described above, it is possible to deal with a release during the restoration of a blurred image, and a photo opportunity is not missed.
Further, the previous image can be restored without missing a photo opportunity.

In addition, since the image restoration itself after shooting is performed in a short time, it is not necessary to miss the photo opportunity and perform the process again later.
In addition, even when the user does not perform any operation, the restoration process can be executed during playback without affecting the release time lag, and the image can be obtained in which the camera shake is automatically removed before the image is confirmed while reducing the release time lag. .

According to the present invention, it is possible to provide an imaging technique capable of continuous imaging without being affected by the blur restoration process for a captured image.
In addition, it is possible to provide an imaging technique in which blurring image restoration is performed simply and automatically and does not miss a photo opportunity.

  In addition, it is possible to provide a technique for improving the operability of an imaging apparatus having a function for restoring motion of a captured image.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<First embodiment>
FIG. 1A is a front perspective view of a digital camera as an imaging apparatus according to an embodiment of the present invention, and FIG. 1B is a rear perspective view thereof.

  The digital camera 10 of this embodiment includes a camera body 1, a lens barrel 2, a release button 3, a zoom switch 4, a power switch 5, a built-in flash 6, a rear LCD panel 7, and a cross key 8.

A zoomable lens barrel 2 is provided on the front side of the camera body 1, and a back LCD panel 7 is provided on the back side.
The release button 3 disposed on the upper part of the camera body 1 is a two-stage switch that can detect half-pressing and full-pressing. When the half-pressing is performed, the half-pressing switch 3a (RSW1) is turned ON, and when fully pressing, the full-pressing switch 3b is pressed. (RSW2) turns ON.

  The zoom switch 4 includes a zoom switch / tele4-1 and a zoom switch / wide 4-2. When the TSW of the zoom switch / tele 4-1 is pressed, the lens is electrically driven to shift to the tele (telephoto) state, and when the zoom switch / wide 4-2 is pressed, the lens is shifted to the wide (wide angle) state. .

The power switch 5 switches the power of the digital camera 10 on / off.
The built-in flash 6 is arranged on the front side of the camera body 1 and can emit light as auxiliary illumination for a subject (not shown).

The rear LCD panel 7 displays a through image seen from the lens barrel 2, displays a captured image, and displays various menus set by operating the cross key 8.
The cross key 8 is arranged on the back side of the camera body 1 in the vicinity of the back LCD panel 7. The cross key 8 is composed of five keys, an up / down / left / right arrow key and a center determination key. For example, after the menu is opened with the center determination key, various settings are made by operating the arrow keys. It is possible.

  An angular velocity sensor 101 and an angular velocity sensor 102 are mounted inside the camera body 1. The angular velocity sensor 101 and the angular velocity sensor 102 detect angular velocities around the X axis and the Y axis of the camera body 1 (lens barrel 2). That is, the angular velocity sensor 101 and the angular velocity sensor 102 are means for detecting camera shake.

FIG. 2 is a block diagram illustrating an example of a configuration of a control circuit unit of the digital camera 10 as the imaging apparatus according to the present embodiment.
The control circuit unit of the digital camera 10 according to the present embodiment includes an analog processing circuit 103, an AD conversion circuit 104, a basic trajectory calculation circuit 105, a trajectory memory circuit 106, a camera shake restoration function calculation circuit 107, a camera shake restoration processing circuit 108, and an image sensor. 109, image sensor output processing circuit 110, image memory 111, image processing circuit 112, image compression / decompression circuit 113, image recording medium 114, sequence controller 115, LCD driver 116, battery 118, power supply circuit 119, motor driver circuit 120, zoom A motor 121, a focus motor 122, a shutter motor 123, and an aperture motor 124 are included.

The outputs of the angular velocity sensor 101 and the angular velocity sensor 102 are amplified by the analog processing circuit 103 and converted into a digital signal by the AD conversion circuit 104.
The basic trajectory calculation circuit 105 calculates a blur trajectory on the image sensor 109 based on the output of the AD conversion circuit 104 within the shutter opening time and the data indicating the zoom state. This is temporarily stored in the trajectory memory circuit 106.

  The image sensor 109 is an image sensor such as a MOS sensor or a CCD sensor. Image data acquired by the image sensor 109 is processed as an image signal by the image sensor output processing circuit 110 and then stored in the image memory 111. The image data stored in the image memory 111 is in an uncompressed state, and is suitable for an operation for restoring image deterioration due to camera shake.

  The camera shake restoration function calculation circuit 107 calculates a point spread function (PSF) based on the data in the trajectory memory circuit 106, and calculates an inverse function of the point spread function for restoring an image. .

The camera shake restoration processing circuit 108 performs the camera shake restoration processing by applying the restoration function calculated by the camera shake restoration function calculating circuit 107 to the photographed image data in the image memory 111.
The image processing circuit 112 performs a process of optimizing with parameters set for the image.

  The image compression / decompression circuit 113 compresses the image optimized by the image processing circuit 112 into a form suitable for storage in the image recording medium 114, and develops and re-reads the image read from the image recording medium 114. Perform processing to enable processing.

The image recording medium 114 is an image recording medium such as a flash memory card.
The sequence controller 115 is composed of, for example, a microprocessor, grasps the state of each part of the digital camera 10 of the present embodiment, and issues an operation instruction to each circuit. In addition, the output of the AD conversion circuit 104 can be directly read, and the amount of blurring when the release button 3 is pressed can be detected.

  The sequence controller 115 also includes a release button 3 (half-press switch 3a, full-press switch 3b), zoom switch 4 (zoom switch / tele 4-1, zoom switch / wide 4-2), power switch 5, and cross key. 8 and the like, operation signals such as ON / OFF of each switch and key are input.

  In this case, the sequence controller 115 performs the exposure based on the exposure time of the image sensor 109 set based on a predetermined program line, which will be described later, and the shake information detected by the basic trajectory calculation circuit 105 before the image sensor 109 is exposed. A prediction function that predicts the amount of shake that occurs when exposure is performed in time, and a change function that changes the exposure time set based on the program line when the predicted amount of shake of this prediction function is greater than or equal to a predetermined value; The control logic 115a which implement | achieves is provided.

  Further, when the control logic 115a detects an exposure start operation of the image sensor 109 during the execution of the restoration operation by the restoration means such as the camera shake restoration function calculation circuit 107 and the camera shake restoration processing circuit 108, the control logic 115a stops the restoration operation and performs imaging. A function of executing the exposure operation of the element 109 is realized.

  Further, when the restoration operation is stopped and the exposure operation of the image sensor 109 is performed, the control logic 115 a performs an operation of storing the blur locus information in the image memory 111 and the image recording medium 114 together with the image data before the restoration. .

  Further, the control logic 115a reads the image data for which the restoration operation has been stopped from the image recording medium 114 and the shake locus information relating to the image data, and restores the image data by the shake restoration function calculation circuit 107 or the shake restoration processing circuit 108. After that, the image recording medium 114 is again recorded at a predetermined timing after the exposure operation of the image sensor 109 is completed.

The control logic 115a is realized by software or hardware included in the sequence controller 115.
The LCD driver 116 outputs the image data to the rear LCD panel 7 for display.

The battery 118 is a power source such as a battery. The power supply circuit 119 converts the power from the battery 118 into a power source suitable for each unit and supplies the power.
The motor driver circuit 120 is controlled by the sequence controller 115 and supplies operating power to, for example, the zoom motor 121, the focus motor 122, the shutter motor 123, and the aperture motor 124.

Next, the operation at the time of shooting of the digital camera 10 of the present embodiment will be described.
FIG. 3 is a flowchart illustrating an example of image restoration processing of the digital camera 10 according to the present embodiment.

  This image restoration is executed after being shot by the digital camera 10 and before being recorded on the recording medium. Alternatively, it can be executed after the image recorded on the image recording medium 114 is read together with the blurring locus data once and expanded.

  When the image restoration process is started (step S201), first, it is determined whether or not the absolute value of the blur locus is equal to or lower than the threshold value P0 (step S202). If the blur is less than or equal to the threshold value P0, it is determined that the blur does not affect the captured image, and the restoration process is not executed (no restoration A) (step S207).

  If the blur is greater than or equal to the threshold value P0 in step S202, it is checked in step S203 whether or not it is greater than or equal to the threshold value P1. Here, when the shake is equal to or less than the threshold value P1, the simple restoration process (simple restoration process B) (first process) is executed (step S208). This simple restoration process B is not a full-scale restoration operation but a process centering on edge enhancement processing, and realizes high speed and prevention of side effects due to the restoration processing.

  If the threshold value P1 is greater than or equal to the threshold value P1 in step S203, it is determined whether or not the amount of shake exceeds the threshold value P3 (step S204). Here, the threshold value P3 is a limit value that can be handled by the restoration operation. If the threshold value P3 is exceeded, no improvement of the image can be expected even if the restoration process is executed. Execution is not performed and restoration processing is not performed (restoration impossible D) (step S205).

  On the other hand, if it is equal to or smaller than the threshold value P3 in step S204, full-scale restoration calculation processing (full-scale restoration processing C) (second processing) based on PSF (Point Spread Function) is started (step S209).

Here, the effect of the restoration calculation process and the processing time will be described.
With reference to FIG. 4, the effects of the above-described four types of restoration processing, that is, no restoration A, simple restoration processing B, full-scale restoration processing C, and restoration impossible D will be described.

FIG. 4 is an example of the digital camera 10 using the image sensor 109 having 7 million pixels, for example.
The amounts of the threshold value P0, the threshold value P1, the threshold value P2, and the threshold value P3 are, for example, 3, 6, 20, and 30 pixels, respectively. This numerical value is appropriately set depending on the size of the image sensor 109, the number of pixels, and the processing circuit capability.

FIG. 4 is a line showing the absolute value of the blur locus on the horizontal axis and the ratio of image quality (OK / NG) judgment (sensory evaluation) based on human senses when the vertical axis is viewed as a 2L size print. FIG.
The restoration calculation of the blurred image by the PSF works well up to a predetermined amount as shown in FIG. 4, but when the amount exceeds a certain amount (in this case, near the threshold value P2), the effect is rapidly reduced. This is because, as the amount of blur increases, the effects of blur detection and computation errors increase, and not only cannot be accurately restored, but the side effects of computation due to restoration processing are conspicuous, and the image appears to deteriorate when viewed sensorially. It is. For this reason, for example, if the captured image itself is suppressed to a blur amount that is equal to or smaller than the threshold value P2, the blurred image restoration works effectively.

FIG. 5 is a diagram showing a processing time after shooting including the image restoration.
In the case of A without restoration in which no restoration processing is performed, the time required for compression and writing to the image recording medium 114 is the shortest time, and thus is the shortest. The simple restoration process B also takes a short time.

  On the other hand, in the case of the full-scale restoration process C, the processing time becomes long. In addition, if the amount of blur of the image to be processed is large, the calculation becomes complicated, so a longer time is required. Even for the same full-scale restoration process C, the processing time further increases when the threshold value P2 is exceeded.

Returning again to the flowchart of FIG. 3, the full-scale restoration process C started in step S209 will be described.
While continuing the full restoration process C in step S210, during this process, it is monitored in step S211 whether or not the operation of the release button 3 for the next shooting (2nd release) has been performed. If the operation of the release button 3 is detected, the process proceeds to step S213 to stop the image restoration process, clear the process and intermediate data being processed (step S214), and end (step S206).

  If there is no release in step S211, it is checked in step S212 whether or not the image restoration process has been completed. If not, the process returns to step S210, and if completed, the image restoration process is performed. The process ends (step S206).

FIG. 6 is a flowchart showing an example of processing at the time of release of the digital camera 10 of the present embodiment.
When the release button 3 is pressed halfway and the half-press switch 3a (RSW1) is turned on (step S301), automatic focus control (AF) and automatic exposure control (AE) are executed (step S302), and the focus is fixed. Exposure is provisionally determined (step S303). Here, in the program diagram for determining exposure, the ISO sensitivity, aperture, and shutter speed of the image sensor 109 are determined along the basic lines. Each constant is determined.

  FIG. 7 shows an example of this program diagram. For example, in the case of a camera equipped with this program diagram, in the telephoto state (zoom switch / tele 4-1 is ON), the program line L 1 indicated by tele and the wide-angle end (zoom switch / wide 4- 2 is ON), the exposure setting is determined by the program line L2 indicated by wide.

  Returning to FIG. 6, after the exposure is provisionally determined in step S303, the sequence controller 115 repeatedly monitors the amount of blur (step S304) and waits for the release button 3 to be fully pressed (step S305). When it is detected that the release button 3 is fully pressed and the full-press switch 3b (RSW2) is turned on, the angular velocity of the previous blur, the zoom state, and the exposure shutter speed T0 temporarily determined in step S303 are set. The predicted amount of shake within the time T0 is calculated using (Step S306).

If the predicted blur amount is equal to or smaller than the threshold value P2 in step S306 (step S307), exposure is executed at the same shutter speed (step S309).
On the other hand, if it is predicted in step S307 that the blur exceeds the threshold value P2, by increasing the shooting sensitivity so that the shutter can be shot at a high speed with the same subject brightness as that used in step S303, A shutter speed T1 that is equal to or lower than the threshold value P2 is calculated.

  The program diagram in step S308 is described in FIG. With the tele H-limit (program line L3) and the wide H-limit (program line L4) in FIG. 7 as the upper limit, the shutter speed T1 at which the blur amount is less than or equal to the threshold P2 is determined by increasing the ISO sensitivity of the image sensor 109. If the threshold value P2 does not fall below the threshold value P2 even if the upper limit is increased, the shutter speed T1 is determined with the upper limit ISO sensitivity.

By determining the shutter speed T1 as described above, the amount of blurring that can exhibit the effect of electronic camera shake correction is suppressed.
Returning to the flowchart of FIG. 6 again, since the exposure value is finally determined as described above, the exposure operation of the image sensor 109 is performed (step S309), and the image is read from the image sensor 109 (step S310).

Thereafter, the process proceeds to an image restoration calculation (step S311), and the image restoration process with the contents described in the flowchart of FIG. 3 is executed.
When the image restoration process in step S311 is exited, in step S312, it is confirmed whether the restoration process is canceled by a release operation during the restoration operation and the image restoration process is exited.

  When the restoration process is interrupted, the image compression process is performed based on the image data before restoration (step S321), the restoration calculation suspension flag is set, and the image data and the set are written to the image recording medium 114 (step S321). In step S322, the blur locus data is written on the image recording medium 114 (step S323), and the image data is written on the image recording medium 114 (step S324). Thereafter, the process immediately proceeds to the next shooting operation (step S325). Specifically, in this step S325, the process proceeds to the above-described step S301 and subsequent steps.

  If it is determined in step S312 that the restoration process is not stopped during the restoration operation, the restored image is subjected to image compression processing (step S313) and written to the image recording medium 114 (step S314).

  Next, with reference to the flowchart of FIG. 8, the operation of reproducing image data in the digital camera 10 of the present embodiment, that is, the operation of displaying an image on the rear LCD panel 7 will be described.

  For example, when a reproduction button (not shown) appropriately assigned to the cross key 8 or the like is pressed to enter the reproduction mode (step S401), first, whether or not the image read from the image recording medium 114 is an image whose restoration has been stopped halfway. Is confirmed (step S402). This can be done by checking whether the restoration calculation stop flag described in step S322 of FIG. 3 is written accompanying the image.

If it is not a restoration stop image, the image is displayed as it is on the rear LCD panel 7 (step S407).
On the other hand, if the image is a restoration stop image in step S403, a blur image restoration calculation is executed based on the blur locus data recorded together (step S403), and the image is reproduced and displayed on the rear LCD panel 7. After (step S404), the restored image is overwritten and recorded on the image recording medium 114 (step S405). Then, the restoration calculation stop flag recorded in the set with the image is cleared (step S406).

  As a modification of the first embodiment, the restoration calculation stop flag described above is checked for all the images recorded on the image recording medium 114, and when an arbitrary image is reproduced and displayed on the rear LCD panel 7, the reproduction is stopped. A method is conceivable in which the sequence controller 115 executes the restoration process of the image in the background if there is an image for which the restoration calculation is stopped even if it is other than the relevant image.

According to this method, restoration processing can be performed at the time of reproduction even for images other than the image to be reproduced, and the restoration processing is executed even when the image is not reproduced.
According to the digital camera 10 of the first embodiment, even if a release operation occurs during the image restoration calculation, the next shooting opportunity is not missed due to the waiting time of the image restoration calculation, so that the next shooting can be performed immediately. It is possible to migrate.

That is, continuous shooting is possible without being affected by the blur restoration process for the shot image.
In addition, since the data being restored is cleared and the process proceeds to the next shooting, memory resources such as the image memory 111 for processing in the digital camera 10 are released, and it is easy to capture and process the next shot image. .

  Further, since the restoration calculation is executed at the time of reproduction to the rear LCD panel 7 or the like that does not require immediacy such as a photo opportunity based on the image before the restoration calculation, the blur locus data, and the restoration calculation stop flag, the camera shake restoration processing is performed. However, since it is possible to execute the follow-up, and the follow-up restoration process is automatically performed, the process can be easily performed as compared with the case where the operation is performed by the user.

That is, it is possible to provide an imaging technique in which image restoration of blurring is performed easily and automatically and does not miss a photo opportunity.
Further, in the above-described modification of the first embodiment, if any of the plurality of images recorded on the image recording medium 114 has stopped the restoration operation, the restoration operation is automatically performed on the image. This is more convenient than the case of performing only the reproduced image displayed on the rear LCD panel 7 or the like.

In other words, it is possible to improve the operability of the imaging apparatus having a function for restoring the shake of the captured image.
<Second Embodiment>
Next, a digital camera that is a second embodiment of the present invention will be described.

  In the second embodiment, the configuration and main operation of the digital camera 10 are the same as those in the first embodiment. However, if there is a release operation during the blur image restoration calculation, is it faster to continue as it is? The difference is that the control logic 115a of the sequence controller 115 determines whether it is faster to switch to the simple processing that requires a short time, and executes one of them to reduce the time lag in continuous shooting.

FIG. 9 is a flowchart showing an example of image restoration processing of the digital camera 10 according to the second embodiment.
The same processes as those in the flowchart of FIG. 3 described above will be denoted by the same reference numerals, and redundant description will be omitted, and different parts will be described.

  When the full-scale restoration process C is started in step S209 and this process is continued (step S210), if a release operation is detected again (step S211), the progress of the restoration process should be observed to continue the process. It is determined whether or not to proceed to the simple restoration process B that can be performed in a short time by discarding the process up to that point (step S221). In other words, when the elapsed time of the current full-scale restoration process C is equal to or less than the predetermined value, the process is executed when it is determined that the current full-scale restoration process C will end sooner.

  More specifically, as an example, the remaining required time trC of the full-scale restoration process C currently being executed (that is, the time from the current time to the completion of the full-scale restoration process C) and the estimated required time tB of the simple restoration process B If trC ≦ tB, that is, if it is determined that continuing the current full restoration process C will end earlier, the process is executed.

  On the other hand, if it is determined in step S221 that it is faster to switch the process (trC> tB), the current full restoration process C is canceled to clear the data being processed (step S222), and the simple restoration process B Then, the process is started again (step S208).

  FIG. 10 is a flowchart illustrating an example of processing during the release operation of the digital camera 10 according to the second embodiment. Note that the same processes as those in FIG. 6 described above are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 10 is similar to the flowchart of FIG. 6 described above, but either the simple restoration process B or the full restoration process C is already executed even when a release operation occurs during the image restoration process. Therefore, the difference is that the restoration calculation stop flag and the blur locus data are not written on the recording medium.

That is, in the flowchart of FIG. 10, the processes of steps S312 and steps S321 to S325 in the flowchart of FIG. 6 described above are omitted.
According to the digital camera 10 of the second embodiment, even if a release operation occurs during image restoration, any one of the image restoration processing is executed, so that it is not necessary to take time and time to process the image later. is there.

In particular, there is an advantage that the burden at the time of reproduction is less when the response at the time of reproduction to the rear LCD panel 7 is more important than the first embodiment.
In each of the above-described embodiments, the exposure / shutter speed is provisionally determined in response to the half-press switch 3a (RSW1) of the release button 3, and then the full-press switch 3b (RSW2) is turned on. The operation to be changed again according to the state of shaking (processing in steps S304 to S308 in the flowcharts of FIGS. 6 and 10) is not essential, and when a processor with low processing capability is used as the sequence controller 115, the operation is omitted. May be.

  Needless to say, the present invention is not limited to the configuration exemplified in the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

1 is a perspective view of a front side of a digital camera as an imaging apparatus according to an embodiment of the present invention. It is a perspective view of the back side. It is a block diagram which shows an example of a structure of the control circuit part of the digital camera as an imaging device which is one embodiment of this invention. It is a flowchart which shows an example of the image restoration process of the digital camera which is one embodiment of this invention. It is a diagram showing an example of the relationship between the amount of image blur and the effect of the restoration process. It is a diagram which shows an example of the relationship between the blurring amount of an image, and the required time of a decompression | restoration process. It is a flowchart which shows an example of the process at the time of release of the digital camera which is one embodiment of this invention. It is a program diagram with which the digital camera which is one embodiment of this invention was equipped. 6 is a flowchart illustrating an example of an image data reproduction operation in the digital camera according to the embodiment of the present invention. It is a flowchart which shows an example of the image restoration process of the digital camera which is other embodiment of this invention. It is a flowchart which shows an example of the process at the time of release operation of the digital camera which is other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera body 2 Lens barrel 3 Release button 3a Half push switch 3b Full push switch 4 Zoom switch 4-1 Zoom switch / tele 4-2 Zoom switch / wide 5 Power switch 6 Built-in flash 7 Rear LCD panel 8 Cross key 10 Digital Camera 101 Angular velocity sensor 102 Angular velocity sensor 103 Analog processing circuit 104 AD conversion circuit 105 Basic trajectory calculation circuit 106 Trajectory memory circuit 107 Camera shake restoration function calculation circuit 108 Camera shake restoration processing circuit 109 Image sensor 110 Image sensor output processing circuit 111 Image memory 112 Image processing Circuit 113 image compression / decompression circuit 114 image recording medium 115 sequence controller 115a control logic 116 LCD driver 118 battery 119 power supply circuit 120 motor driver circuit 121 zoom Motor 122 Focus motor 123 Shutter motor 124 Aperture motor A No restoration B Simple restoration process C Full restoration process D No restoration L1 Program line L2 Program line L3 Program line L4 Program line P0 Threshold P1 Threshold P2 Threshold P3 Threshold tB Simple restoration process B Required time trC The remaining required time for the full-scale restoration process C

Claims (16)

An image sensor for acquiring image data of a subject;
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to acquire the image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Including
The control means includes control logic that, when detecting an exposure start operation of the image sensor during execution of the restoration operation by the restoration means, stops the restoration operation and executes the exposure operation of the image sensor. An imaging device. The imaging device according to claim 1,
The image pickup apparatus, wherein the control logic of the control means stores the blur locus information together with the image data before restoration in the storage medium when the restoration operation is stopped and an exposure operation is performed. The imaging device according to claim 1,
The control logic of the control means reads the image data whose restoration operation has been stopped from the storage medium and the blur locus information related to the image data, restores the image data by the restoration means, and then stores the image data in the storage medium. An image pickup apparatus, wherein the recording operation is performed again at a predetermined timing after the exposure operation of the image pickup device is completed. An image sensor for acquiring image data of a subject;
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration by executing at least one of a first process and a second process having different required times based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to obtain the image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Including
The control unit stops the second process by the restoration unit and executes the first process when detecting an exposure start operation to the image sensor during the restoration operation using the second process by the restoration unit. An imaging apparatus comprising control logic for causing the The imaging apparatus according to claim 4.
The time required for the restoration operation of the image data by the restoration unit is shorter than the execution time of the second process. The imaging apparatus according to claim 4.
In the restoration operation using the first process, edge enhancement processing of the image data is performed, and in the restoration operation using the second process, an inverse function of the point spread function is calculated based on the blur locus information. Then, the image data restoration process is performed. An image sensor for acquiring image data of a subject;
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration by executing at least one of a first process and a second process having different required times based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to obtain image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Including
When the control unit detects an exposure start operation to the image sensor during the execution of the restoration operation using the second process by the restoration unit, the elapsed time of the restoration operation being performed has passed a predetermined time. Control logic that causes the restoring means to continue the restoring operation being executed if it is present, and to stop the restoring operation being executed if the elapsed time is less than a predetermined time and to execute the restoring operation using the first process An imaging apparatus comprising: The imaging device according to claim 7.
The time required for the restoration operation of the restoration unit by the restoration unit is shorter than the execution time of the second process. The imaging device according to claim 7.
In the restoration operation using the first process, edge enhancement processing of the image data is performed, and in the restoration operation using the second process, an inverse function of the point spread function is calculated based on the blur locus information. Then, the image data is restored. An image sensor for acquiring image data of a subject;
Blur detection means;
Computing means for obtaining blur locus information relating to the blur locus during the exposure operation of the image sensor based on the output of the blur detection means;
Restoring means for restoring the image data acquired by the imaging device to image data without deterioration based on the blur locus information;
A storage medium for storing the image data restored by the restoration means;
Control means for causing the image sensor to perform an exposure operation to obtain image data, restoring the image data by the restoration means, and storing the image data in the storage medium;
Reproducing means for reading out and reproducing image data from the storage medium;
Have
When the control unit detects an exposure start operation of the image sensor during execution of the restoration operation by the restoration unit, the control unit stops the restoration operation, and the blur locus information together with the image data that has not been restored is stored in the storage medium. After recording, the image sensor is exposed,
When the reproduction unit reads the image data that has not been restored from the recording medium, the reproduction unit causes the restoration unit to perform a restoration operation of the image data based on the blur locus information of the image data, and the restored image data An image pickup apparatus which records again on the recording medium. A first step for continuously obtaining image data from a subject and restoring the blur in the image data;
A second step of stopping the restoration process and proceeding to the next exposure operation when an exposure start operation of the next image data is detected during the restoration process;
A method for controlling an imaging apparatus, comprising: In the control method of the imaging device according to claim 10,
In the second step, when the restoration operation is stopped and the next exposure operation is performed, the blur locus information used in the restoration operation is stored in the storage medium together with the image data before the restoration. A control method for an imaging apparatus. In the control method of the imaging device according to claim 12,
Further, the operation of reading out the image data whose restoration operation has been stopped from the storage medium and the blur locus information related to the image data, restoring the image data by the restoration means, and then recording the image data again on the storage medium, A method for controlling an imaging apparatus, comprising a third step that is performed at a predetermined timing after the end of an exposure operation. A first step for continuously obtaining image data from a subject and restoring the blur in the image data;
A second step of switching the restoration process between a first process and a second process different from each other when an exposure start operation of the next image data is detected during the restoration process;
A method for controlling an imaging apparatus, comprising: The method of controlling an imaging apparatus according to claim 14,
The execution time of the image data restoration operation by the first process is shorter than the execution time of the image data restoration operation by the second process,
In the second step, the imaging apparatus control method is characterized in that the second process having a long execution time is switched to the first process having a short real time. The method of controlling an imaging apparatus according to claim 14,
In the restoration operation using the first process, edge enhancement processing of the image data is performed, and in the restoration operation using the second process, an inverse function of the point spread function is calculated based on the blur locus information. Then, the image data is restored, and a method for controlling the image pickup apparatus.