JP2008058546A - Detection device, correction device, camera and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a detection device, a correction device, a camera and a program by which image blur is suitably corrected. <P>SOLUTION: The detection device is provided with point spread function arithmetic calculation parts (14A, S30 and S50) arithmetically calculating point spread functions for a first area including an object brought into focus in a photographic image and a second area different from the first area in the photographic image, and component arithmetic calculation parts (14A and S60) detecting the state of the divergence of focus in the photographic image by using the point spread functions for the first area and the second area, and arithmetically calculating blur components (A3, B3 and C3) included in the photographic image different from the divergence of focus. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a detection device, a correction device, a camera, and a program related to image shooting.

2. Description of the Related Art A camera equipped with an optical blur correction device that reduces image blur due to camera shake during shooting is known.
However, the optical blur correction device cannot expect a correction effect for image blur when the camera rotates around the optical axis of the photographing optical system.
On the other hand, Patent Document 1 discloses a technique for correcting image blur when a camera rotates around the optical axis of a photographing optical system using a point spread function (point spread function).
However, the technique described in Patent Document 1 has a problem that the quality of the image after correction is greatly influenced by the position where the point spread function is selected, and the reliability is low.
JP 2000-298300 A

  An object of the present invention is to provide a detection device, a correction device, a camera, and a program that can appropriately correct image blur.

The present invention solves the above problems by the following means. In addition, in order to make an understanding easy, although the code | symbol corresponding to embodiment of this invention is attached | subjected and demonstrated, it is not limited to this.
According to the first aspect of the present invention, a point spread function is calculated for a first region including a focused subject in a captured image and a second region different from the first region in the captured image. And a point image distribution function calculation unit (14A, S30, S50) that performs the detection and detects the state of out-of-focus in the captured image using the point image distribution functions of the first area and the second area. And a component calculation unit (14A, S60) for calculating a blur component (A3, B3, C3) included in the photographed image different from the out-of-focus state.
A second aspect of the present invention is the detection apparatus according to the first aspect, wherein the blur component (A3, B3, C3) includes a rotational blur around a photographing optical axis. .
A third aspect of the present invention is the detection apparatus according to the first or second aspect, wherein the component calculation unit (14A, S60) is the point image of the first region and the second region. In the detection apparatus, the blur component is calculated after removing the in-focus shift using a distribution function.
According to a fourth aspect of the present invention, there is provided the detection apparatus according to any one of the first to third aspects, and an image restoration calculation unit (16, S80) for removing the blur component detected by the detection apparatus. It is a correction apparatus characterized by including these.
According to a fifth aspect of the present invention, in the correction apparatus according to the fourth aspect, a rotational component computing unit (14B, 14) for computing the rotational blur component from the point spread function of the first region and the second region. S60), and using the rotational blur component calculated by the rotational component calculation unit, the point spread function is converted into a point spread function of the region other than the first region and the second region in the captured image. A conversion unit (14B, S70) that converts the image into an image, and an image calculation unit (16 that performs image restoration by removing the rotational blur component included in the photographed image using the point spread function converted by the conversion unit. , S80).
According to a sixth aspect of the present invention, in the correction device according to any one of the first to fifth aspects, the point spread function calculation unit (14A, S30, S50) sets the second region to 2. It is a correction device characterized in that the calculation is performed using a place.
A seventh aspect of the present invention is a camera (10) characterized by including the correction device according to any one of the fourth to sixth aspects.
The invention according to claim 8 is the camera according to claim 7, wherein the point spread function calculation unit (14A, S30, S50) corresponds to the focus detection region (21) set in the photographing screen. The camera (10) is characterized in that the area to be operated is the first area.
The invention according to claim 9 is an input unit to which information relating to a captured image and a first region in the captured image is input, the first region, and the first region in the captured image. Using a point spread function calculation unit for calculating a point spread function and the point spread function of the first area and the second area for a second area different from each other. A correction apparatus comprising: a component calculation unit that detects a defocus state and calculates a blur component included in the photographed image that is different from the focus shift; and an image restoration calculation unit that removes the component (100).
A tenth aspect of the present invention is the correction apparatus according to the ninth aspect, wherein the blur component includes a rotation blur around a photographing optical axis.
According to the eleventh aspect of the present invention, the computer (100) is configured to input a photographed image and information relating to the first region in the photographed image, the first region, and the image in the photographed image. Point spread function calculation means (S30, S50) for calculating a point spread function for a second area different from the first area, and the point spread of the first area and the second area. A function calculation unit (S60) that detects a state of in-focus deviation in the captured image using a function, calculates a blur component included in the captured image that is different from the in-focus error, and removes the component. This is a program for functioning as the image restoration calculation means (S80).
A twelfth aspect of the present invention is the program according to the eleventh aspect, wherein the blur component includes a rotational blur around a photographing optical axis.
In addition, you may improve suitably the structure which attached | subjected the code | symbol. In addition, at least a part may be replaced with another component, and the configuration requirements that are not particularly limited with respect to the configuration are not limited to the configurations disclosed in the embodiments.

  According to the present invention, it is possible to provide a detection device, a correction device, a camera, and a program that can appropriately correct image blur.

  Hereinafter, embodiments of the present invention will be described in more detail with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram illustrating an optical blur correction mechanism of the camera 10 according to the first embodiment.
FIG. 2 is a block diagram illustrating an outline of the camera 10 according to the first embodiment.
1 and 2, the camera 10 according to the first embodiment includes a lens 11, a shake correction actuator 2 (2p, 2y), an optical shake correction control unit 3, a lens position detection sensor 4 (4p, 4y), a release switch 5, and the like. Shake sensor 9 (9p, 9y), image sensor 12, AF area information acquisition unit 13, PSF calculation unit 14, captured image acquisition unit 15, deconvolution execution unit 16, recovery image acquisition unit 17, data storage unit 18, input unit 19 etc.

The lens 11 forms a part of the photographing optical system, and is provided so as to be movable with respect to an image sensor 12 described later within a plane substantially orthogonal to the optical axis Z. The lens 11 is a blur correction optical system that corrects image blur in the image sensor 12 by moving in a direction that cancels image blur of the subject image due to camera shake.
The blur correction actuator 2 is a driving unit that generates a driving force for moving the lens 11, and a voice coil motor or the like is used. The blur correction actuator 2 is provided with a blur correction actuator 2p that is driven at the time of image blur correction in the pitching direction and a blur correction actuator 2y that is driven at the time of image blur correction in the yawing direction.

The optical shake correction control unit 3 is a circuit that drives the shake correction actuator 2 and drives the shake correction actuator 2 according to a drive target calculated by a shake correction calculation circuit (not shown).
The lens position detection sensor 4 is a position sensor that detects the position of the lens 11, and a lens position detection sensor 4 p that detects the position of the lens 11 during image blur correction in the pitching direction, and a yawing direction. A lens position detection sensor 4Y that detects the position of the lens 11 at the time of image blur correction is provided.
The shake sensor 9 is an angular velocity sensor that detects the angular velocity of the camera 10, and is provided with a shake sensor 9p that detects pitching shake and a shake sensor 9y that detects yawing shake.
The camera 10 according to the present embodiment includes two systems of blur correction actuators 2, pitching and yawing, a lens position detection sensor 4, a shake sensor 9, and other control systems, which are caused by pitching and yawing. Image blur is optically corrected by driving the lens 11.
However, with such an optical blur correction mechanism alone, it is not possible to correct rotational blur caused by rolling in which the camera 10 rotates about the optical axis Z. Also, complete translation (translational movement) without rotation cannot be detected by the shake sensor 9 and cannot be corrected.
Therefore, the camera 10 of the present embodiment is capable of performing blur correction by image restoration in addition to the optical blur correction mechanism, correcting rotational blur due to rolling, and translational motion. The image blur due to the image can be corrected.

  In FIG. 2, an image pickup device 12 is a device that picks up an image formed on the image pickup surface by the lens 11, and in this embodiment, a CCD (Charge Coupled Devices) is used. The image sensor is not limited to the CCD, and other types of image sensors such as CMOS (Complementary Metal Oxide Semiconductor) may be used.

The AF area information acquisition unit 13 acquires information on the AF area used for focusing by an autofocus (AF) operation, and specifies the position of the main subject in the captured image.
The PSF calculation unit 14 calculates a PSF (Point Spread Function) from the captured image. The PSF is also called a point spread function, and is used for deconvolution calculation described later.
The PSF calculation unit 14 of the present embodiment includes a multi-region PSF calculation unit 14A that calculates a PSF of a main subject region and a reference region, which will be described later, and an all-pixel PSF calculation unit 14B that calculates a PSF of all pixels. . These multi-region PSF calculation unit 14A and all-pixel PSF calculation unit 14B can detect, for example, image blur components such as rotational blur that remain without being corrected by an optical blur correction mechanism.
The captured image acquisition unit 15 is a part that temporarily stores a captured image captured by the image sensor 12.
The deconvolution execution unit 16 is a part that performs image restoration calculation on the captured image data using the PSF calculated by the PSF calculation unit 14. In this embodiment, the deconvolution execution unit 16 includes the captured image by performing deconvolution. Image blurring.
The recovered image acquisition unit 17 is a part that temporarily stores a recovered image obtained as a result of the deconvolution execution unit 16 executing the deconvolution.
The data storage unit 18 is a part that stores a captured image and a recovered image, and in the present embodiment, the data storage unit 18 records it on a removable memory card.
The input unit 19 is an operation member that can perform various input operations.

Here, the AF operation of the camera of this embodiment will be described.
The camera of this embodiment can select S mode and C mode as AF operation.
The S mode is a focus-priority AF mode in which shooting can be performed only when the release switch 5 is pressed halfway and a focus display (not shown) is lit. In the S mode, when the release switch 5 is pressed halfway and the in-focus display is lit, AF lock can be performed to maintain the in-focus state without performing focus adjustment while the release switch 5 is pressed halfway. . Therefore, if the release switch 5 is half-pressed, even if the composition is changed and the subject is deviated from the AF area, the focus is adjusted (focused) on the subject on which the AF is locked. Can hold.
The C mode is a release-priority AF mode in which shooting can be performed at any time by fully pressing the release switch 5 regardless of the focus display. The focus adjustment is always performed while the release switch 5 is pressed halfway. to continue. Accordingly, when the composition is changed, the lens 11 is continuously driven so that the subject located at the position overlapping the AF area is focused.

Next, the operation of the camera of this embodiment will be described. Note that the operations described below are based on the premise that image recovery settings are set. However, if image recovery is not set, the operations related to image recovery are not performed. It is also possible to perform the same operation as that of a conventional camera.
First, the operation when the S mode is selected as the AF mode will be described. In the following description, it is assumed that a person is a main subject and the person is focused. The main subject is not limited to a person but may be another object. For the sake of simplicity, the description will be made assuming that only one AF area 21 is provided at the center of the photographing range.

FIG. 3 is a flowchart showing an operation flow when the S mode is selected in the camera 10 of the present embodiment.
In step (hereinafter referred to as S) 10, a photographing operation is performed. The operation in S10 will be described in detail with reference to FIG.
FIG. 4 is a flowchart showing in detail the photographing operation in step 10 in the flowchart shown in FIG.
When the photographing operation is started, in S110, it is determined whether or not the release switch 5 is half-pressed. If the release switch 5 is half-pressed, the process proceeds to S120. If the release switch 5 is not half-pressed, the determination in S10 is repeated.
In S120, it is determined whether or not the subject is in focus. If it is in focus, the process proceeds to S130, and if it is not in focus, the process returns to S110.
FIG. 5 is a diagram illustrating the operation of the AF area information acquisition unit 13 when the S mode is selected.
In the S mode, for example, the release switch 5 is half-pressed so that the AF area 21 overlaps with a person to focus (FIG. 5). When focused, a focus display (not shown) is turned on, and the AF locked state is maintained to maintain the focused state. When the AF is locked, the AF area information acquisition unit 13 performs the operation of S130 described below.

Returning to FIG. 4, in S <b> 130, the AF area information acquisition unit 13 temporarily stores the AF locked image obtained by the image sensor 12 when AF locking is performed, and also uses the AF area used at the time of AF locking. 21 positions are stored.
In S140, it is determined whether or not the release switch 5 is half-pressed. When the half-press of the release switch 5 is continued, the process proceeds to S150, and when the release switch 5 is not half-pressed, the process returns to S110.
Here, when it is desired to change the composition, the composition is changed while the release switch 5 is kept half-pressed in a state where the focus display is lit. For example, it is possible to change the composition such as bringing the person to the left and leaving the background tree outside the shooting screen, but since the release switch 5 is being pressed halfway, the focus is on the person. The state is retained.
In S150, it is determined whether or not the release switch 5 is fully pressed. If the release switch 5 is fully pressed, the process proceeds to S160. If the release switch 5 is not fully pressed, the process returns to S140.
In S160, imaging is performed by the imaging device 12, and a captured image is obtained. The process returns to the flow of FIG. 3 and proceeds to S20.

Returning to FIG. 3, in S20, the AF area information acquisition unit 13 compares the AF locked image temporarily captured when the AF is locked with the photographed image, and the AF area 21 overlaps when the AF is locked. It is specified in which position in the captured image the subject is located. Note that this subject comparison can be performed by, for example, extracting a plurality of feature points in the image at the time of AF lock and searching for the feature points from the captured image.
When the position of the subject (area in the screen) where the AF area 21 overlaps when the AF is locked in the captured image is specified, the AF area information acquisition unit 13 transmits the position information to the PSF calculation unit 14.

In S30, PSF calculation is performed on the main subject area specified by the AF area information acquisition unit 13.
In S40, two reference areas are selected in addition to the main subject area. These two reference areas are automatically selected according to the following selection criteria, for example.
Criterion 1: The position is as far as possible from the main subject area in the screen.
Criterion 2: The distance between the reference areas is long.
Standard 3: The contrast of the reference area is high enough to perform PSF calculation.
Note that the reference area can also be determined using the input unit 19 according to the photographer's intention.
In the following description, it is assumed that the background person on the right side and the background tree on the left side in the shooting screen of FIG. 5 are selected. When the reference area is selected, the process proceeds to S50.

In S50, the PSF of the two selected reference areas is calculated. Unlike the PSF of the main subject area calculated earlier, the reference area calculated in this step has a high possibility of so-called blurring (out of focus) due to defocusing. If this blur component is included, the image blur cannot be accurately corrected from the calculated PSF.
FIG. 6 is a diagram schematically showing PSF two-dimensionally. FIG. 6A shows a case where there is no image blur, and FIG. 6B shows a case where an image blur including a rotation component is further added to the PSF of FIG. 6A. The PSF in FIG. 6 is shown corresponding to FIG.
FIG. 7 is a diagram showing the order of PSF calculations performed by the camera of this embodiment.

The PSF in FIG. 6A is a case where the state shown in FIG. 5 is ideally photographed without image blurring. In the state of FIG. 5, the person is in focus and the background tree is blurred. Therefore, the PSF corresponding to this is a point where the PSF at the position corresponding to the person is A1, as shown in FIG. 6A, and corresponds to the PSF (B1) corresponding to the background person and the background tree. The position PSF (C1) is widened due to the blur.
The PSF in FIG. 6B is for a captured image including image blur having a rotation component. The PSF of FIG. 6B is affected by the image blur, and further includes a component corresponding to the locus of the image blur in addition to the PSF of FIG.
The PSF in FIG. 6C is obtained by removing the blur component due to the defocus from the PSFs (B1) and (C1) in FIG. 6B to obtain only the image blur component.

The PSF calculated in S50 is used when correcting the blur component by the image restoration calculation, and therefore needs to be free from the blur component. As a method of calculating the PSF of only the blur component from the PSF including the blur component and the blur component, the PSF of the blur component is calculated from the width of the PSF including the blur component and the blur component, and the PSF of the blur component is calculated. There is a method of using the deconvolution of the PSF including the blur component and the blur component. Further, simply, the PSF of the blur component can be calculated by obtaining the maximum value in the width direction of the PSF including the blur component and the blur component.
As described above, when the PSF of only the blur component is calculated for the two reference regions without including the blur component, the process proceeds to S60.

  In S60, using the PSF of the main subject area obtained in S30 and the PSFs of the two reference areas obtained in S50, the blur of the captured image is separated into a rotation component and a translation component, and the PSF of the rotation component is obtained. Ask.

In S70, the rotation component PSF calculated in S60 is converted into a PSF at each position of the captured image.
If rotation blur does not occur and only translation blur (blur without rotation) occurs, the PSF of the blur component on the screen is equal at any position, and the obtained single PSF If image recovery is performed using, blurring can be corrected over the entire screen.
However, when rotational blur occurs, the blur amount differs between a position close to and far from the rotation center, and the blur direction differs depending on the positional relationship with the rotation center. Therefore, the PSF varies depending on the position in the screen. Therefore, when rotational blur has occurred, it is necessary to obtain the PSF for each position in the screen in order to correct the rotational blur by image restoration.

  As a method for obtaining the PSF of the blur component of the entire screen (each pixel) from the PSF of the blur component of a plurality of regions, the PSF of the blur component of the main subject region and the PSF of the blur component of the two reference regions are compared. In addition to obtaining the rotation center, the rotation angle around the rotation center is obtained, the rotation component is calculated from the positional relationship between the position to be obtained and the rotation center, and the PSF and the rotation component of the shake center at the rotation center are obtained together. Convert to PSF where you want. If the shape of the PSF of the blur component is complicated, the PSF is divided with reference to the feature points on the PSF (for example, bending points, PSF intensity maximum points, etc.), and a rotation component is obtained for each divided portion. It is also possible.

In S80, deconvolution is performed.
Here, image restoration will be described.
As a method of bringing an image deteriorated due to image blurring or the like closer to an ideal image with little deterioration, there is a method called image restoration.
Now, (x, y) is a position coordinate on the screen, an image when there is no blur (hereinafter referred to as an original image) is f (x, y), and an image degraded by blur (hereinafter referred to as a degraded image) is g. Assuming that (x, y), the point spread function (PSF), which is information of a point spread spread by blurring, is h (x, y) and noise is n (x, y), There is a relationship.

Here, if the point spread function h (x, y) and the noise n (x, y) are known, the deconvolution, which is the inverse operation of the convolution, is performed, so that the degraded image g (x, y) is obtained. An original image f (x, y) is obtained.
In S90, the obtained restored image is stored. In addition to the recovered image, the captured image can also be saved.

Since the above description is a case where the S mode is selected as the AF mode, a case where the C mode is selected as the AF mode will be described next.
When the C mode is selected as the AF mode, it operates to focus on the subject on which the AF area 21 overlaps until immediately before the release switch 5 is fully pressed. Therefore, when the C mode is selected, it is appropriate to assume that the main subject in the captured image is in a position where the AF area 21 overlaps at the time of shooting. Therefore, when the C mode is selected, the AF area information acquisition unit 13 notifies the PSF calculation unit 14 of the information on the assumption that the main subject exists at a position where the AF area 21 overlaps at the time of shooting.

In the above description, the AF area 21 is described as being provided at the center of the shooting screen for the sake of simplicity. However, the camera according to the present embodiment is not limited to the center of the shooting screen. AF areas 21 are arranged at a plurality of positions.
FIG. 8 is a diagram illustrating the distribution of the AF area 21 of the camera 10 according to the first embodiment.
In the example shown in FIG. 8, nine AF areas 21 are arranged. In the camera of the present embodiment, the AF operation is performed using one area selected by the photographer or automatically selected from the nine AF areas 21. Note that the AF area 21 used by the photographer is selected by operating the input unit 19. The AF area 21 selected by operating the input unit 19 is displayed differently from other areas in order to be distinguished from other areas. In the present embodiment, the display color is changed as in the AF area 21a shown in FIG.

In the above description, the AF area information acquisition unit 13 automatically specifies the position of the main subject. However, in S20 of FIG. 3, the main subject position specified by the AF area information acquisition unit 13 is displayed. Then, the photographer can correct the position by operating the input unit 19.
Further, by operating the input unit 19 without operating the AF area information acquisition unit 13, the position used for the PSF calculation can also be designated.
When the position used for the PSF calculation is corrected or specified using these input units 19, it is realized by making a separate setting using a selection menu or the like.
Since the position used for the PSF calculation can be specified or corrected using the input unit 19 in this way, the photographer's intention is reflected more accurately, and the probability that a good recovered image can be obtained increases.

According to the present embodiment, the position of the main subject is specified using the information of the AF area 21 used for the AF operation, the PSF is calculated for the position, the PSF of the reference area is obtained, and the captured image Since the rotational blur is corrected, it is possible to correct the rotational blur around the photographing optical axis that cannot be corrected only by the optical blur correction mechanism. Therefore, it is possible to correct image blurring that occurs during pitching, yawing and rolling.
In addition, since the blur component is removed from the PSF on the screen and the recovery is performed, the blur is kept as it is, and an image as intended by the photographer is obtained.
Furthermore, image blur caused by translational movement of the camera that cannot be corrected by an optical blur correction mechanism can be corrected by image restoration.

(Second Embodiment)
FIG. 9 is a block diagram showing an outline of the camera 20 of the second embodiment.
In addition, the same code | symbol is attached | subjected to the part which fulfill | performs the same function as 1st Embodiment mentioned above, and the overlapping description is abbreviate | omitted suitably.
The camera system of the second embodiment has a camera 20 and a program installed in the computer 100.
The camera 20 of the second embodiment does not include the PSF calculation unit 14, the deconvolution execution unit 16, and the recovered image acquisition unit 17 that are included in the camera 10 of the first embodiment.
Further, the AF area information acquisition unit 13 provided in the camera 20 of the second embodiment specifies the position of the main subject in the captured image in the same manner as in the first embodiment. Information regarding the position of the main subject is added to the captured image data of the captured image acquisition unit 15 and stored by the data storage unit 18.

Furthermore, the programs installed in the computer 100 according to the second embodiment include programs that cause the computer 100 to operate as follows by being installed in the computer 100.
The program included in the camera system of the second embodiment allows the captured image data, which is stored by the data storage unit 18 and to which information related to the position of the main subject is added, to be input to the computer 100. In addition, the method of inputting the data stored in the data storage unit 18 to the computer 100 may be via a memory card, the camera 20 and the computer 100 may be communicable by wire or wireless, You may carry out via a network. Further, since the data stored by the data storage unit 18 can be duplicated, it may be input to the computer 100 via a storage medium other than a memory card, such as a recording disk medium.

Furthermore, the program included in the camera system of the second embodiment causes the computer 100 to perform the same operations as the PSF calculation unit 14, the deconvolution execution unit 16, and the recovered image acquisition unit 17 in the first embodiment.
The computer 100 in which the program is installed obtains the main subject area and the PSF of the reference area by referring to the position information of the main subject obtained at the same time with respect to the input captured image data, and the rotation component. , The PSF at each position is calculated, and image restoration is performed by deconvolution to obtain a restored image.
Since the image restoration calculation for recovering the rotational blur caused by the rotation around the optical axis is enormous, the calculation can be performed at higher speed by using the computer 100. In addition, by using the computer 100, it is possible to set the condition for image restoration calculation more precisely.

(Deformation)
The present invention is not limited to the embodiment described above, and various modifications and changes are possible, and these are also within the equivalent scope of the present invention.
(1) In the first embodiment, the AF area information acquisition unit 13 compares the captured image with the AF locked image temporarily stored when the AF is locked, and the AF area 21 overlaps when the AF is locked. Although an example in which the subject is located in the captured image has been shown, the present invention is not limited to this. For example, the AF area information acquisition unit 13 recognizes a subject that overlaps the AF area when the AF is locked. In addition, the position of the subject may be sequentially updated until the time of photographing to specify the position of the subject in the photographed image. This configuration adds a lot of parts if image data used for a so-called through display image that continues to display an image obtained by the image sensor on a display device provided on the back of the camera or the like before shooting operation is used. Can be realized.

(2) In the first embodiment, when the PSF calculation unit 14 and the deconvolution execution unit 16 are set to perform image recovery, an example in which PSF calculation and image recovery are always performed has been described. For example, it may be determined whether or not there is an image blur at the position of the main subject obtained by the AF area information acquisition unit 13 and image recovery may be performed only when there is an image blur. In this case, for example, only when a function for determining the presence / absence of image blur is added to the AF area information acquisition unit 13 and an instruction to restore the image is received from the AF area information acquisition unit 13, the PSF calculation unit 14 and the deconvolution execution part 16 should just perform the process regarding image restoration.

(3) In each embodiment, the position information of the identified main subject is shown as an example used for PSF calculation. However, the present invention is not limited to this. For example, exposure value determination, exposure correction, and gain (sensitivity) You may utilize for other uses, such as a change.

(4) In the first embodiment, the data storage unit 18 has shown an example of storing a captured image and a recovered image. In this case, the position information of the identified main subject is added to the captured image. Alternatively, position information of the main subject, information on the calculated PSF, and the like may be added to the recovered image.

(5) In the second embodiment, an example in which image restoration is performed using the computer 100 in which the program is installed has been described. However, the present invention is not limited to this. For example, the PSF calculation unit 14, the deconvolution execution unit 16, and the restored image acquisition unit An apparatus dedicated to image restoration that can perform the same operation as that of the image 17 may be used. In addition, such a device is installed in a store such as a photo print company, a base office, etc., and a photographer passes photographed image data to which information on a main subject position is added via a recording medium or communication means. A trader may perform image restoration processing, or may record or print a recovered image obtained as a result on a recording medium.

(6) In each embodiment, the example in which the position of the main subject is determined based on the information of the AF area has been described. However, the present invention is not limited to this. For example, the position of the main subject is determined by determining the position of the face. The position of the eye may be determined to specify the position of the main subject, or an arbitrary area in the captured image may be used.

(7) In the second embodiment, an example in which the camera 20 that does not include the PSF calculation unit 14, the deconvolution execution unit 16, and the recovery image acquisition unit 17 included in the camera 10 of the first embodiment has been shown. For example, a camera similar to the camera 10 of the first embodiment may be used, and information regarding the position of the main subject may be added to the captured image data.

It is a figure explaining the optical blurring correction mechanism of the camera 10 by 1st Embodiment. It is a block diagram which shows the outline | summary of the camera 10 of 1st Embodiment. 6 is a flowchart showing an operation flow when an S mode is selected in the camera 10 of the present embodiment. It is a flowchart which shows in detail the imaging | photography operation | movement of step 10 in the flowchart shown in FIG. It is a figure explaining operation | movement of AF area information acquisition part 13 when S mode is selected. It is the figure which represented PSF typically on two dimensions. It is a figure which shows the order of the PSF calculation which the camera of this embodiment performs. It is a figure which shows distribution of AF area 21 of the camera 10 of 1st Embodiment. It is a block diagram which shows the outline | summary of the camera 20 of 2nd Embodiment.

Explanation of symbols

  10, 20: Camera, 13: AF area information acquisition unit, 14: PSF calculation unit, 16: Deconvolution execution unit, 19: Input unit

Claims (12)

A point spread function calculation unit that calculates a point spread function for a first region including a focused subject in the photographed image and a second region different from the first region in the photographed image. When,
Using the point spread function of the first area and the second area, a state of defocus in the photographed image is detected, and a blur component included in the photographed image different from the focus deviation And a component calculation unit that calculates the above.   The detection apparatus according to claim 1, wherein the blur component includes rotational blur around a photographing optical axis. The detection device according to claim 1 or 2,
The component calculation unit calculates the blur component after removing the in-focus shift using the point spread function of the first region and the second region. A detection device according to any one of claims 1 to 3,
And a correction unit that removes the blur component detected by the detection unit. The correction device according to claim 4,
A rotation component calculation unit that calculates the rotation blur component from the point spread function of the first region and the second region;
Conversion for converting the point spread function into a point spread function in a region other than the first region and the second region in the photographed image, using the rotation blur component calculated by the rotation component calculation unit. And
A correction apparatus comprising: an image calculation unit that recovers an image by removing the rotational blur component included in the photographed image using the point spread function converted by the conversion unit. In the correction device according to any one of claims 1 to 5,
The point spread function calculation unit performs calculation using the second region in two places;
A correction device characterized by the above. Including the correction device according to any one of claims 4 to 6,
Camera characterized by. The camera according to claim 7, wherein
The point spread function calculation unit sets a region corresponding to a focus detection region set in a photographing screen as the first region;
Camera characterized by. An input unit for inputting information regarding the captured image and the first region in the captured image;
A point spread function calculation unit for calculating a point spread function for the first area and a second area different from the first area in the captured image;
Using the point spread function of the first area and the second area, a state of defocus in the photographed image is detected, and a blur component included in the photographed image different from the focus deviation A component calculation unit for calculating
And a correction device that removes the component.   The correction apparatus according to claim 9, wherein the blur component includes rotational blur around a photographing optical axis. Computer
Input means for inputting information relating to the photographed image and the first region in the photographed image;
Point spread function calculation means for calculating a point spread function for the first area and a second area different from the first area in the captured image;
Using the point spread function of the first area and the second area, a state of defocus in the photographed image is detected, and a blur component included in the photographed image different from the focus deviation Component computing means for computing
A program for functioning as an image restoration calculating means for removing the component.   12. The program according to claim 11, wherein the blur component includes rotational blur around a photographing optical axis.