JP6705485B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus that picks up an image of a light flux that has passed through a photographing lens.

The image pickup device (CCD or the like) is used by being housed in an image pickup device package. When an image sensor (CCD, etc.) is mounted on a substrate and housed in an image sensor package, the image sensor (also simply called “chip”) is warped due to the difference in linear expansion coefficient between the substrate, image sensor, seal member, etc. I have something to do. For example, the chip may be warped due to a difference in linear expansion coefficient that occurs when the chip is mounted on the substrate by soldering or when the substrate is mounted on the package with resin (such as an adhesive). In an image pickup apparatus such as a digital camera, if there is a warp on the image pickup surface (light receiving surface) of the image pickup element, the focus position is partially displaced, which causes out-of-focus, making it impossible to obtain a good image.

Note that there is a related image processing device (for example, see Patent Document 1). The image processing apparatus of Patent Document 1 is uniform even when the taking lens type identification code of the taking lens is not identified when correcting at least one of distortion, chromatic aberration of magnification, peripheral light amount shortage and out-of-focus caused by the taking lens. Moreover, it is an object of the present invention to provide an image processing device capable of surely performing appropriate correction of the aberration without forgetting.

JP, 2000-184257, A

However, even if the defocusing correction is performed by the taking lens as in the image processing apparatus of Patent Document 1 described above, the defocusing correction is performed by the blurring that changes concentrically due to the characteristics of the taking lens. However, the warp occurring in the image pickup device itself is not limited to the concentric circle shape, and may have a distorted shape or may vary depending on individual differences of the image pickup device. Therefore, it may be impossible to properly reduce the out-of-focus blurring caused by the warpage of the image pickup device itself by using only the information of the taking lens.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image pickup apparatus capable of reducing the out-of-focus image blur caused by the warp of the image pickup element.

The present invention has been made to solve the above problems, and an imaging apparatus according to one aspect is an imaging device having a plurality of regions for imaging a subject, and an image taken in a first region of the plurality of regions. First image processing is performed on the first image signal of the subject using information about the warp of the image sensor in the first region, and in a second region different from the first region in the plurality of regions. for the second image signal of the imaged object, the second image processing unit for performing image processing using information on the warp of the imaging device in the second region, wherein said image processing unit, the first An image is generated from the first image signal subjected to the one image processing and the second image signal subjected to the second image processing .

According to the present invention, it is possible to reduce the out-of-focus blurring of an image caused by the warp of the image sensor.

It is a block diagram which shows the structure of the imaging device concerning embodiment of this invention. It is a figure which shows the example of the curvature of an image sensor. It is a figure which shows the example which calculates|requires a focus blur correction value with respect to a warp value. It is a figure which shows the effect of defocusing correction. It is a figure which shows the example in case an image sensor does not warp. It is a figure which shows the example which controls a focus blur correction|amendment by ISO sensitivity and shutter speed. It is a figure which shows the example which controls out-of-focus correction by F value. It is a figure which shows the example which controls a focus blur correction with ISO sensitivity and F value. It is a figure which shows the example which controls out-of-focus correction in imaging|photography mode. It is a figure which shows the schematic image of an image sensor package. It is a figure which shows the shape of the curvature of an image sensor, and the example of the curvature value. It is a figure which shows the example of the out-of-focus generation|occurrence|production when a chip|tip has a curvature.

Embodiments of the present invention will be described below with reference to the accompanying drawings.

[Explanation of Warp of Image Sensor]
First, a supplementary description will be given of the occurrence of out-of-focus due to the warp of the image sensor. As described above, the image pickup device (CCD or the like) is used by being housed in the image pickup device package. For example, FIG. 10 is a diagram showing a schematic image of an image pickup device package. As shown in this figure, the image pickup device package 30 is a container in which a substrate 31 on which the image pickup device 11 is mounted is fixed to a seal member 32, and the image pickup device 11 and the substrate 31 are formed by a seal member 32 and a protective glass 33. It is configured to be stored inside. As described above, when the image pickup device 11 is mounted on the substrate 31 and housed in the image pickup device package 30, the image pickup device (chip) 11 is warped due to the difference in linear expansion coefficient between the substrate, the image pickup device, the seal member, and the like. Sometimes. For example, FIG. 11A is a diagram showing an image of chip warpage. As shown in FIG. 11, in the image sensor 11, a warp may occur in each direction of the three-dimensional shape (the orthogonal X, Y, and Z axis directions).

Further, FIG. 11B is a diagram showing an image of the warp of the image pickup device, and FIG. 11B shows the image pickup device 11 shown in FIG. 11A in the X direction (the AA′ direction). It shows a cross-sectional view, and schematically shows the warped state of the image sensor 11. As shown in FIG. 11B, the virtual chip reference plane Rp of the image sensor 11 is calculated, and the displacement amount (ΔZ) in the height direction at each position from the reference plane Rp, that is, the Z direction. Is the warp value (warp value). Then, the upper side of the reference plane Rp is defined as plus (+), and the lower side is defined as minus (-). As described above, in an image pickup apparatus such as a digital camera, when the image pickup element (chip) 11 is housed in the image pickup element package, the image pickup surface (light receiving surface) is warped, and the focus position is partially displaced, resulting in defocusing. , It may not be possible to obtain a good image.

For example, FIG. 12 is a diagram showing an example of occurrence of out-of-focus when the chip has a warp. FIG. 12A shows the shape of the warp of the chip, and FIG. 12B shows the image captured by the image sensor 11.

In FIG. 12A, the range of the area 1 is a range in which the image pickup element 11 is less warped and has no influence on the image, and the range of the area 2 is that the image pickup element 11 is largely warped and has an influence on the image. It is a range. In the range where the warp of the image pickup device 11 is large, as shown in the example of the image in FIG. 12B, out-of-focus is generated in the region 2 (the portion surrounded by the broken line S), and it becomes impossible to obtain a good image. The image pickup apparatus according to the present embodiment corrects out-of-focus blur caused by the warp of the image pickup element 11.

[Configuration of imaging device]
FIG. 1 is a block diagram showing the arrangement of an image pickup apparatus according to the embodiment of the present invention. The image pickup apparatus shown in FIG. 1 shows a configuration example of an electronic camera such as a digital camera, and shows only a portion directly related to the present invention. The image pickup apparatus 10 includes an image pickup unit 11A including an image pickup device 11, an optical system taking lens 12, a control unit 13, a recording unit 14, a storage unit 15, a setting unit 17, a signal processing ASIC 20, and Is configured.

The image pickup unit 11A includes an image pickup device 11 and an A/D converter (not shown), and converts the subject image formed on the light receiving surface of the image pickup device 11 into an electric signal according to the amount of received light. The image data after the /D conversion is output to the signal processing ASIC 20.
The control unit 13 detects that an operation switch (not shown) such as a release button included in the imaging device 10 is operated, controls the drive of the taking lens 12, and performs focus detection/adjustment processing in the optical system. .. Further, the control unit 13 controls the operation of the signal processing ASIC 20 by inputting and outputting a synchronization signal and a control signal with the signal processing ASIC 20. Further, the control unit 13 transmits a warp correction control signal Sc to the ASIC 20 for signal processing, thereby performing warp correction processing (more precisely, focus blur correction described later) in the signal processing unit 22 (more accurately, the focus blur correction unit 23). Control execution and non-execution. The recording unit 14 records the image data processed by the signal processing ASIC 20.

The storage unit 15 records the out-of-focus correction value calculated based on the warp value used when performing out-of-focus correction. For example, in the assembly process or the adjustment process of the imaging device 10, the warp of the image sensor 11 is measured, the warp shape of the chip and its warp value are acquired in advance, and the out-of-focus correction value is calculated from this warp value. The corrected out-of-focus correction value 16 is recorded in the storage unit 15. The focus blur correction unit 23 in the signal processing unit 22 reads out the focus blur correction value 16 recorded in the storage unit 15 when the control unit 13 instructs to execute the focus blur correction, and the image captured by the image sensor 11 is read. Out-of-focus correction is performed for.
The setting unit 17 includes an operation unit including a power button, a shutter button, a mode selection dial, a menu button, a cross key, and the like. The user operates the setting unit 17 to set shooting conditions such as shutter speed and ISO sensitivity, and shooting mode. Information on the shooting conditions and shooting modes set by the setting unit 17 is output to the control unit 13.

In the image pickup section 11A of the image pickup apparatus 10, the subject light incident on the taking lens 12 is imaged on the image pickup element 11 which is a CCD. The image signal output from the image sensor 11 is output to the ASIC 20 for image processing. The image signal input to the ASIC 20 for image processing is input to the preprocessing unit 21. The pre-processing unit 21 converts the image signal into a digital signal, performs image pre-processing such as gamma correction, and outputs the pre-processed image signal to the signal processing unit (post-processing unit) 22.

The signal processing unit (post-processing unit) 22 performs a process of generating image data for display based on the image signal input from the pre-processing unit 21. Further, the signal processing unit 22 causes the out-of-focus correction unit 23 to perform out-of-focus correction on the image signal. When performing the focus blur correction by the focus blur correction unit 23, the focus blur correction can be performed by using a generally well-known method such as sharpness and unsharp mask.

[Obtaining the warp value and calculating the out-of-focus correction value]
In the image pickup apparatus 10 of the present embodiment, the image pickup element 11 shown in FIG. 2A is divided into a plurality of areas as shown in FIG. 2B, and the warp value in each area is measured.
2B is a plan view of the image pickup surface (light receiving surface) of the image pickup element (chip) 11 shown in FIG. The drawing shows a warp value in each area by dividing the plane (light receiving surface) of the image sensor 11 into a plurality of rectangular areas of m×n (m rows, n columns). Then, the warp value at the center position p in each region is measured, and this warp value is set as the warp value in the region. For example, when the size (chip size) of the image sensor 11 is 40×30 mm, the warp value in each region is measured by dividing into 35×24 divisions.

When measuring the warp value, for example, a laser interferometer is used to measure the surface shape in a non-contact manner with respect to the image pickup element 11, so that the warp value can be accurately measured (for example, in nanometer units or micrometer units). The warp value can be obtained. As shown in FIG. 11(B) described above, this warp value is a difference in height with respect to the virtual reference plane Rp, that is, a displacement amount in the plus (+) direction and a displacement amount in the minus (−) direction. Can be obtained.

In the imaging device 10 of the present embodiment, the focus blur correction value calculated based on the measured warp value is stored in the storage unit 15. When calculating this out-of-focus correction value, as shown in FIG. 3, the optimum value of out-of-focus correction value with respect to the warp value is obtained in advance at the design stage or the development stage of the imaging device 10 based on the empirical formula c. .. Then, based on this empirical formula c, a focus blur correction value is obtained from the warp value in each region, and stored in the storage unit 15 as the focus blur correction value 16. If the warp value is small, for example, if the warp value in the area 1 shown in FIG. 3 is within the range of the focus shift amount (the range in which the image is not out of focus even if the image is out of focus due to the characteristics of human eyes), then the out-of-focus image is out of focus. The correction value can be zero (0). The defocus amount is a value determined by the type of lens and the F value (size of the diaphragm).

The out-of-focus blur correction unit 23 reads out out-of-focus blur correction values from the storage unit 15 for each area shown in FIG. 2B with respect to the image signal captured by the image sensor 11, and uses the out-of-focus blur correction values to generate an image. Out-of-focus correction is performed for. The out-of-focus correction unit 23 adjusts the emphasis of contours and the reproducibility of details by performing sharpness or unsharp masking on an image, and clearly reproduces a out-of-focus portion.
Note that, when performing the out-of-focus blur correction (for example, sharpness) in the out-of-focus blur correction unit 23, the out-of-focus blur correction value is used as it is to perform the out-of-focus blur correction on the image. Focus blur correction can also be performed (for example, by reducing the focus blur correction value to a predetermined ratio so that “focus blur correction value×0.6”).

FIG. 4 is a diagram showing an example of the effect of the focus blur correction. FIG. 4A is a diagram showing a chip shape when the chip is warped, and is the same as FIG. 12A described above. In FIG. 4A, the range of the region 1 is a range in which the warp due to the chip is small and the image is not affected. That is, in the range of the area 1, the warp value is within the range of the focus shift amount, and the image is not out of focus. On the other hand, the range of the region 2 is a range in which the warp due to the chip is large and the image is affected. In the range of the region 2 where the warp due to the chip is large, as shown in the example of the image in FIG. 4B, out-of-focus (a portion surrounded by a broken line S) occurs, and a good image cannot be obtained. Therefore, the out-of-focus correction unit 23 performs out-of-focus correction (for example, sharpness) on out-of-focus (the portion surrounded by the broken line S), and an image without out-of-focus can be obtained as shown in FIG. 4C.

As described above, by performing the focus blur correction on the warp of the image sensor 11 by the focus blur correction unit 23, the image sensor 11 can be equivalently made to have no warp as shown in FIG. Therefore, the image without the out-of-focus image shown in FIG. 5B can be obtained.

It should be noted that when the focus blur correction is executed, the phenomenon of defocusing is the same regardless of whether the warp is on the plus (+) side or the minus (-) side of the reference plane Rp. However, there is only a difference, and where there is a warp effect, the sharpness or unsharp mask is used to emphasize the contour.
Further, in the image pickup apparatus 10 of the present embodiment, the warp of the image pickup element (chip) 11 is obtained by the height difference (+, −) from the reference plane Rp, but the warp is based on the curvature (bending degree) of the chip surface. May be requested.

[Control of execution and non-execution of out-of-focus correction]
As described above, when sharpness or unsharp masking is performed on an image, the emphasis on contours and the reproducibility of details are adjusted, and out-of-focus areas are clearly reproduced, but on the other hand, image noise is noticeable. It will be easier. For this reason, noise may be emphasized and displayed by performing focus blur correction, and unnaturalness may appear in an image. Therefore, in the image pickup apparatus 10 of the present embodiment, focus blur correction is not always performed at each shooting, but focus blur correction is executed according to shooting conditions such as ISO sensitivity, F value, shutter speed, and shooting mode. And configured to control non-execution.

For example, in high-sensitivity shooting with a lot of noise, it is highly possible that the noise is conspicuously displayed due to the defocusing correction. Therefore, it is possible to perform the correction in low-sensitivity shooting with less noise, and it is also possible to adjust the strength of the out-of-focus correction. Also, when the depth of field is deep (F value is large), there are many scenes in which not only the main subject but also the background are in focus, so if there is chip warpage, out-of-focus blurring may be noticeable. is there. Therefore, it is possible to perform control such that the correction is executed when the F value is large, and the correction is not executed when the depth of field is shallow (the F value is small). Also, depending on the type of shooting mode, the ISO sensitivity may be high and low, and the F value of the shooting lens may be large and small, so focus blur correction may be executed, may not be executed, and the strength of correction may be different. To be able to control.

(Blur correction according to ISO sensitivity and shutter speed)
FIG. 6 is a table showing whether or not the focus blur correction is executed according to the ISO sensitivity and the shutter speed, and is an example of the case where the focus blur correction is performed by the sharpness. The table shown in FIG. 6 shows the ISO sensitivity as a parameter in the horizontal direction and the shutter speed T as a parameter in the vertical direction. Further, in the table, a circle mark indicates that the focus blur correction is executed, and a cross mark indicates that the focus blur correction is not executed (not executed).

As shown in this table, when the shutter speed T is a low speed second, that is, when the shutter speed is longer than 1/8 sec, the focus blur correction is executed when the ISO sensitivity is 400 or less, and the ISO sensitivity exceeds 400. Does not perform defocus correction. Further, when the shutter speed T is a high speed second (1/8 sec or less), focus blur correction is performed when the ISO sensitivity is 800 or less, and focus blur correction is not performed when the ISO sensitivity exceeds 800.

This is because the noise of the image may be emphasized by executing the out-of-focus correction, and the noise is highly likely to be noticeable due to the out-of-focus correction at high sensitivity with a lot of noise. Therefore, it is possible to execute the out-of-focus correction only with low sensitivity with less noise, and it is also possible to adjust the intensity of out-of-focus correction. In this case, since the exposure amount of the image plane changes depending on the shutter speed T, when the exposure amount is large (1/8 sec or more), the focus blur correction is executed at ISO sensitivity of 400 or less, and when the exposure amount is small (1/8 sec). The following) executes out-of-focus correction at an ISO sensitivity of 800 or less. That is, the ISO sensitivity when the exposure amount is large (the ISO sensitivity serving as the determination standard) is set to be lower than the ISO sensitivity when the exposure amount is small.

As described above, by selecting whether or not to execute the out-of-focus correction according to the ISO sensitivity and the shutter speed T, the out-of-focus image due to the warp of the chip is corrected while avoiding the noticeable noise of the image. be able to.
It should be noted that when executing the out-of-focus blur correction, the intensity of the out-of-focus blur correction value may be weakened to perform the out-of-focus blur correction. For example, when the shutter speed T is longer than ⅛ (T>1/8 sec) and the ISO sensitivity 800 does not perform defocus correction, it is adjacent to the area of the ISO sensitivity 400 where defocus correction is performed. In the case of the ISO sensitivity 800, the strength of the out-of-focus correction may be weakened (for example, the out-of-focus correction value may be reduced to a predetermined ratio so as to be “out-of-focus correction value×0.6”).

Note that the table shown in FIG. 6 is an example, and the ISO sensitivity, which is the boundary when determining whether or not to perform the out-of-focus correction, is the range of the warp value of the image sensor, the type of the taking lens, and the F value. It can be changed according to the amount of defocus that is achieved.
For example, when the range of the warp value (the range of the maximum value and the minimum value of the warp) of the entire image pickup surface of the image sensor is small, the out-of-focus correction value becomes small, and the noise due to out-of-focus correction becomes less noticeable. It is possible to increase the ISO sensitivity that is a criterion (boundary) for determining whether or not to execute. On the other hand, when the range of the warp value of the image sensor is large, the amount of defocusing correction becomes large, and noise due to defocusing correction often becomes conspicuous. Therefore, it is necessary to reduce the ISO sensitivity that is the determination standard (boundary).
In this way, the ISO sensitivity, which is the determination reference (boundary) when determining whether to execute the out-of-focus correction, can be set according to the range of the warp value of the entire image pickup surface of the image pickup device 11.

The ISO sensitivity, which is the criterion (boundary), is the range of the amount of defocus of the photographing lens (the range where the image is not out of focus even if the image is out of focus due to the characteristics of the human eye) and the range of the warp value of the image sensor (the warp It can also be changed depending on the range between the maximum value and the minimum value. This is because the out-of-focus correction value can be made smaller as the amount of focus shift becomes larger and the range of the warp value becomes smaller. In this way, the ISO sensitivity, which is the reference (boundary) when determining whether to execute the out-of-focus correction, is determined according to the defocus amount of the photographing lens 12 and the range of the warp value of the entire image sensor 11. It can also be set.

(Focus correction according to F value)
Further, FIG. 7 is a table showing whether or not the focus blur correction is executed according to the F value. The table shown in FIG. 7 shows the F value as a parameter in the horizontal direction, and shows whether or not the focus blur correction is executed according to the F value. Further, in the table, a circle mark indicates that the focus blur correction is executed, and a cross mark indicates that the focus blur correction is not executed (not executed).
As shown in this table, focus blur correction is executed when the F value is 2.8 or less and when the F value exceeds F8, and focus blur correction is executed when the F value exceeds F4 and F8 or less. do not do.

That is, the focus blur correction is performed at the opening of the photographic lens aperture and at both ends of the aperture, and not in the middle. It is empirically known that out-of-focus blurring due to chip warpage is conspicuous on the open side (small F-number side), and out-of-focus correction is executed when the F-number is 2.8 or less. On the other hand, it is expected from the user that a clear image can be obtained by increasing the depth of field on the aperture side (large F value), but when the depth of field is large (F value is large), Since there are many scenes in which the subject as well as the background are in focus, if the chip is warped, out-of-focus blurring due to the warpage may be noticeable. Therefore, when the F value is large (when the F value exceeds F8), the focus blur correction is executed.

As described above, when the depth of field is shallow (F value is small) and when the depth of field is deep (F value is large), the focus blur correction is executed, and the focus blur correction is not executed in the intermediate region. By doing so, it is possible to correct the out-of-focus blur caused by the warp of the chip while avoiding the noise of the image from becoming conspicuous.

Note that the table shown in FIG. 7 shows an example, and the F value, which is the reference (boundary) when determining whether or not to perform focus blur correction, is the range of the warp value of the image sensor and the amount of defocus. It can also be set according to.
For example, when the range of the warp value of the entire imaging surface of the image sensor is small, the out-of-focus correction value is small, and noise and unnaturalness due to out-of-focus correction are less noticeable. Therefore, out-of-focus correction can be performed. The range of values can be expanded. In this way, the F value, which serves as a reference (boundary) when determining whether or not to perform focus blur correction, can be optimally set according to the magnitude of the warp value of the entire image pickup surface of the image pickup device 11. ..

Further, the F value can be changed by the amount of defocus of the photographing lens and the warp value of the entire image pickup element (range of the maximum and minimum warp values). This is because the larger the amount of defocus and the smaller the range of the warp value, the smaller the out-of-focus correction value, and the less noise is emphasized during out-of-focus correction, and the unnaturalness is unnatural. Is less noticeable. Therefore, the larger the focus shift amount and the smaller the range of the warp value range, the wider the F value range in which the focus blur correction can be executed. In this way, the F value, which serves as a reference (boundary) when determining whether or not to perform the out-of-focus correction, may be set according to the focus shift amount and the range of the warp value of the entire image sensor 11. it can.

(Defocusing correction according to ISO sensitivity and F value)
Further, FIG. 8 is a table showing whether or not the focus blur correction is executed according to the ISO sensitivity and the F value. FIG. 8A shows the shutter speed T when the shutter speed T is low speed (when the shutter speed is longer than 1/8 sec), and FIG. 8B shows the shutter speed T when the shutter speed T is high speed (the shutter speed is 1/8 sec). The following cases) are shown. In the tables shown in FIGS. 8A and 8B, the F value is shown as a parameter in the horizontal direction, and the ISO sensitivity is shown as a parameter in the vertical direction, and the focus blurring depends on the F value and the ISO sensitivity. It indicates whether or not the correction is executed. In the table, ○ indicates that focus blur correction is performed, × indicates that focus blur correction is not performed (not executed), and Δ indicates the strength of the focus blur correction value when the focus blur correction is performed. Is weakened (for example, the focus blur correction value is reduced to a predetermined ratio so that “focus blur correction value×0.6”), and focus blur correction is executed.

When the shutter speed is slower than ⅛ sec as shown in FIG. 8A, the F value is F2.8 or less or exceeds F8, and the ISO sensitivity is 400 or less. In some cases, focus blur correction is executed. Further, when the F value exceeds F2.8 and is F8 or less and the ISO sensitivity is 400 or less, the strength of the focus blur correction value is weakened (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. Further, when the F value is F2.8 or less or exceeds F8 and the ISO sensitivity exceeds 400, the focus blur correction value is weakened (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. When the F value exceeds F2.8 and is equal to or less than F8, and the ISO sensitivity exceeds 400, the focus blur correction is not executed.

Further, at the high-speed seconds shown in FIG. 8B, that is, when the shutter speed is 1/8 sec or less, the F value is F2.8 or less or exceeds F8, and the ISO sensitivity is 800 or less. If, then out-of-focus correction is executed. When the F value exceeds F2.8 and is F8 or less and the ISO sensitivity is 800 or less, the strength of the focus blur correction value is weakened (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. Further, when the F value is F2.8 or less or exceeds F8 and the ISO sensitivity exceeds 800, the strength of the focus blur correction value is weakened (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. When the F value exceeds F2.8 and is equal to or less than F8, and the ISO sensitivity exceeds 800, the focus blur correction is not executed.
In this way, whether or not the focus blur correction unit 23 performs the focus blur correction can be determined based on the ISO sensitivity and the F value. Further, the ISO sensitivity used for the determination can be changed according to the shutter speed.

Note that the table shown in FIG. 8 shows an example, and the ISO sensitivity and the F value, which are the criteria (boundary) when determining whether to execute the out-of-focus correction, are the same as those in FIGS. 6 and 7. Similarly, it can be changed according to the range of the warp value of the image sensor and the amount of defocus.

(Focusing blur correction according to shooting mode)
Further, FIG. 9 is a table showing whether or not the focus blur correction is executed according to the shooting mode. In the table shown in FIG. 9, when shooting portraits, landscapes, children's snaps, sports, and night views as shooting modes (shooting targets), focus defocus correction is executed to shoot macros, dishes, and pets. In this case, the focus blur correction is not executed. This is because the ISO sensitivity may be increased or decreased depending on the type of shooting mode, or the F value may be increased or decreased. Therefore, depending on the shooting mode, focus defocus correction may be performed, control of non-execution may be performed, and focus defocus may be performed. Be able to control the strength of the correction. In this way, by deciding whether or not to execute the focus blur correction according to the shooting mode (shooting target), the shooting mode in which it is estimated that noise and unnaturalness are not noticeable is selected, and Correction can be performed.

The selection of the shooting mode (shooting target) is performed by a user who operates the camera by using a menu setting button (for example, a dial button or the like) or a menu screen.
Further, the table shown in FIG. 9 shows an example, and the shooting mode for determining whether or not to execute the focus blur correction according to the configuration of the image pickup apparatus, the type of the image pickup lens, the chip size of the image pickup element, etc. It is something that gets changed.

(Defocus correction according to the amount of defocus and the range of warpage)
The example of controlling whether or not the focus blur correction is executed based on various shooting conditions (ISO sensitivity, F value, shooting mode) has been described above, but here, according to the amount of defocus and the range of the warp value, The out-of-focus correction will be supplementarily described.

In the image pickup apparatus, the F value can be adjusted by adjusting the light amount of the taking lens with a diaphragm. By changing the F value, the range in which an image can be clearly captured before and after the focus changes, and the focus width (focus shift amount) of the image surface (light receiving surface) on the image pickup element (chip) 11 changes. .. For example, increasing the F value (decreasing the aperture hole) increases the amount of defocus. Therefore, if the warp value of the image pickup element (chip) 11 is within the range of the focus shift amount, the out-of-focus blur does not occur.

Therefore, in a certain area on the image sensor 11, if the warp value is within the range of the focus shift amount, it is not necessary to execute the focus blur correction. Then, when the warp value of the chip is out of the range of the defocus amount, it is determined whether or not the focus blur correction is executed based on the above-described shooting conditions such as the ISO sensitivity, the F value, and the shooting mode. Become.
In this way, if the warp value is within the range of the focus shift amount, the out-of-focus correction is not performed, and only when the warp value exceeds the range of the focus shift amount, the out-of-focus correction is performed according to the imaging conditions such as ISO sensitivity. It can be determined whether or not.

As described above, in the image pickup apparatus 10 of the present embodiment, the focus blur correction unit 23 performs the focus blur correction on the warp of the image pickup element 11 to obtain an image without the focus blur. In particular, when a large sensor chip such as a DSLR (Digital Single Lens Reflex camera) is used, the warpage becomes large. Further, as technology advances, pixels are becoming finer, and the effect of out-of-focus blurring is increasing. In such a case, it is possible to effectively reduce the out-of-focus by using the configuration of the present embodiment.

The embodiments of the present invention have been described above. Here, the correspondence between the present invention and the above embodiments will be supplementarily described. In the above embodiment, the image pickup apparatus of the present invention corresponds to the image pickup apparatus 10 shown in FIG. 1, the image pickup element of the present invention corresponds to an image pickup element (chip) 11, and the image pickup section of the present invention corresponds to the image pickup element 11 An image pickup unit 11A that includes an image of a subject and outputs image data of the subject is included in the image pickup unit 11A. The control unit in the present invention corresponds to the control unit 13, and the correction unit in the present invention corresponds to the out-of-focus correction unit 23 in the signal processing ASIC 20. The storage unit in the present invention corresponds to the storage unit 15, and the setting unit in the present invention corresponds to the setting unit 17. Further, the information regarding the warp in the present invention corresponds to the warp value and the out-of-focus correction value calculated based on the warp value.

(1) Then, in the above-described embodiment, the image pickup apparatus 10 includes the image pickup element 11 having a photoelectric conversion function, the image pickup section 11A that generates image data, and the information about the warp in the normal direction of the image pickup surface of the image pickup element 11. And a correction unit (focus blur correction unit 23) that corrects the image data based on the above information.
In the image pickup apparatus 10 having such a configuration, the warp value of the image pickup element (chip) is measured in the image pickup apparatus assembly process, adjustment process, and the like. Then, a correction value for out-of-focus blur is calculated from this warp value, and this out-of-focus correction value is stored in the storage unit 15 of the imaging device 10 in advance. Then, the focus blur correction is performed on the captured image using the focus blur correction value stored in the storage unit 15. A well-known sharpness or unsharp mask method is used when performing defocusing correction.
As a result, it is possible to reduce the out-of-focus blur of the image caused by the warp of the image sensor.

(2) Further, in the above-described embodiment, the storage unit 15 stores the above information for each of a plurality of regions on the imaging surface, and the correction unit (focus blur correction unit 23) based on the above information stored for each region. , The image data corresponding to the area is corrected.
In the image pickup apparatus 10 having such a configuration, the warp value in each region of the image pickup element (chip) is measured in the image pickup apparatus assembling process, the adjusting process, and the like. Then, a correction value for out-of-focus blur is calculated from this warp value, and this out-of-focus correction value is stored in the storage unit 15 of the imaging device 10 in advance. Then, for each area of the captured image, focus blur correction is performed using the focus blur correction value stored in the storage unit 15. A well-known sharpness or unsharp mask method is used when performing defocusing correction.
As a result, it is possible to reduce the out-of-focus blur of the image caused by the warp of the image sensor.

(3) Further, in the above-described embodiment, the imaging device 10 includes the setting unit 17 that sets the shooting conditions of the imaging unit 11A, and the correction unit (focus blur correction unit 23) according to the shooting conditions set by the setting unit 17. And a control unit 13 that controls whether or not the correction of the image data is performed.
In the imaging apparatus 10 having such a configuration, the control unit 13 controls whether or not the focus blur correction unit 23 performs the focus blur correction according to the shooting conditions (for example, ISO sensitivity, F value, shooting mode, etc.). .. Sharpness and unsharp masks are well known as methods for correcting out-of-focus blur. However, when using these methods to correct out-of-focus, noise may be emphasized and unnaturalness may be noticeable. There is. Therefore, the correction is not necessarily performed in each shooting, but the execution or non-execution of the out-of-focus correction or the intensity of the out-of-focus correction is controlled according to the shooting conditions.
This makes it possible to control whether or not to perform the out-of-focus correction according to the shooting conditions. For this reason, it is possible to prevent the noise of the image from being emphasized and the unnaturalness to be conspicuous due to the focus blur correction.

(4) In the above embodiment, the shooting condition is the ISO sensitivity, and the control unit 13 determines whether the correction unit (focus blur correction unit 23) corrects the image data according to the value of the ISO sensitivity. Control.
In the imaging device 10 having such a configuration, for example, when the ISO sensitivity is low, that is, the focus blur correction is performed only with the low sensitivity with less noise.
Thereby, when the ISO sensitivity is high and a lot of image noise is included, it is possible to prevent the image noise from being emphasized by the out-of-focus correction and the unnaturalness from being conspicuous.

(5) In the above embodiment, the shooting condition is the F value of the shooting lens, and the control unit 13 determines whether or not the correction unit (focus blur correction unit 23) corrects the image data according to the F value. Or control.
In the imaging device 10 having such a configuration, whether or not the focus blur correction unit 23 performs the focus blur correction is controlled according to the magnitude of the F value (image brightness and depth of field).
Thereby, for example, when the F value is small, that is, when the depth of field (more accurately, the amount of focus shift) is small and the out-of-focus is likely to occur due to the warp of the image sensor 11, the out-of-focus can be corrected. ..

(6) Further, in the above embodiment, the shooting condition is the ISO sensitivity and the F value, and the control unit 13 corrects the image data by the correction unit (focus blur correction unit 23) according to the ISO sensitivity and the F value. Control whether to execute.
In the image pickup apparatus 10 having such a configuration, whether the focus blur correction unit 23 performs the focus blur correction is controlled according to the ISO sensitivity and the F value.
As a result, whether or not the focus blur correction unit 23 performs the focus blur correction can be finely controlled based on both the ISO sensitivity parameter and the F value parameter.

(7) In the above embodiment, the shooting condition is the shooting mode, and the control unit 13 controls whether or not the correction unit (focus blur correction unit 23) corrects the image data according to the shooting mode. ..
In the image pickup apparatus 10 having such a configuration, whether or not the focus blur correction unit 23 performs the focus blur correction is controlled according to the shooting mode. This is because the ISO sensitivity may be set higher or lower depending on the type of shooting mode, and the F value of the shooting lens may be set higher or lower, so focus blur correction may be performed depending on the shooting mode. , Non-execution, or control the intensity of defocus correction.
As a result, by selecting the shooting mode, it is possible to obtain a clear image without out-of-focus according to the shooting mode.

(8) Further, in the above-described embodiment, the control unit 13 controls the image data by the correction unit (focus blur correction unit 23) according to the amount of warpage and the amount of focus shift determined by the type of the taking lens and the F value. Controls whether or not correction is executed.
In the image pickup apparatus 10 having such a configuration, when the range of the warp value exceeds the focus shift amount determined by the type of the photographing lens 12 and the F value, whether or not the focus blur correction is executed is controlled according to the above-mentioned image pickup condition. In the imaging device 10, the F value can be adjusted by adjusting the light amount of the taking lens with the diaphragm. By changing the F value, the range in which an image can be clearly captured before and after the focus changes, and the focus width (focus shift amount) of the image surface on the image pickup element (chip) 11 changes. Therefore, if the warp value of the image sensor 11 is within the range of the focus shift amount, out-of-focus blur does not occur. Therefore, in a certain area on the image sensor 11, if the warp value is within the range of the focus shift amount, the focus blur correction is not executed, and if the warp value is outside the range of the focus shift amount, the focus blurring is performed according to the above-described image pickup condition. Perform correction.
Accordingly, if the range of the warp value is within the range of the focus shift amount, it is possible to prevent the focus blur correction from being executed.

(9) Further, in the above embodiment, when the shooting condition is the ISO sensitivity, the control unit 13 corrects the image data in the correction unit (focus blur correction unit 23) when the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity. When the predetermined ISO sensitivity is exceeded, the correction unit (focus blur correction unit 23) is not caused to correct the image data.
In the image pickup apparatus 10 having such a configuration, the focus blur correction unit 23 executes the focus blur correction when the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity. Since the out-of-focus correction also has the effect of emphasizing noise, it is highly likely that noise will be noticeable due to the correction in high sensitivity where there is a lot of noise. Therefore, it is possible to perform the correction only with low sensitivity with less noise, and also to adjust the intensity.

(10) In the above embodiment, the predetermined ISO sensitivity is set according to the shutter speed.
In the imaging device 10 having such a configuration, the ISO sensitivity for determining whether or not the focus blur correction is performed is set according to the shutter speed. This is because the exposure amount of the image pickup device 11 is determined according to the ISO sensitivity and the shutter speed. Therefore, when the shutter speed is slow (the exposure amount is large), the ISO sensitivity for determining whether or not the focus blur correction should be performed is lowered. Set to.
This makes it possible to set the ISO sensitivity for determining whether or not focus blur correction is to be performed according to the shutter speed. Therefore, it is possible to prevent noise from being emphasized by the focus blur correction.

(11) In the above embodiment, the predetermined ISO sensitivity is set according to the amount of warpage of the entire image pickup surface of the image pickup device 11.
In the image pickup apparatus 10 having such a configuration, the ISO sensitivity, which is a reference for determining whether to perform the out-of-focus correction, is set to the range of the warp value of the entire image pickup surface of the image pickup device 11 (the range of the maximum value and the minimum value of the warp). Set according to. This is because the smaller the range of the warp value, the smaller the out-of-focus correction value used when executing out-of-focus correction, and it becomes more difficult to emphasize image noise due to out-of-focus correction. Therefore, it is possible to increase the value of the ISO sensitivity that serves as a determination reference (boundary) for performing defocus correction.
This makes it possible to optimally set the ISO sensitivity, which is the determination reference (boundary) for performing the out-of-focus correction, according to the range of the warp value of the image sensor 11.

(12) In the above embodiment, the predetermined ISO sensitivity is set according to the amount of warpage of the entire image pickup surface of the image pickup device 11 and the amount of focus shift determined according to the type of the photographing lens and the F value. It
In the image pickup apparatus 10 having such a configuration, the ISO sensitivity, which is a reference for determining whether to perform the out-of-focus correction, the range of the warp value of the entire image pickup surface of the image pickup element 11, the defocus amount of the photographing lens 12, Set by. This is because the larger the amount of defocus and the smaller the range of the warp value (the range of the maximum and minimum values of the warp), the smaller the out-of-focus correction value used when executing the out-of-focus correction, and the image due to the out-of-focus correction becomes smaller. Noise is less likely to be emphasized. Therefore, it is possible to increase the value of the ISO sensitivity, which is the criterion (boundary) for performing the out-of-focus correction.
This makes it possible to optimally set the ISO sensitivity, which is the determination reference (boundary) for performing the out-of-focus correction, in accordance with the range of the warp value of the image sensor 11 and the focus shift amount of the taking lens 12.

(13) Further, in the above-described embodiment, in the imaging device 10, when the shooting condition is the F value, the control unit 13 controls the case where the F value is equal to or less than the predetermined first F value and the predetermined second F value. When it exceeds the value, the correction unit (focus blur correction unit 23) is made to correct the image data, and when the F value exceeds the first F value and is equal to or less than the predetermined second F value, the correction unit (focus blur correction). The part 23) is not made to execute the correction of the image data.
In the imaging device 10 having such a configuration, when the shooting condition is the F value, the control unit 13 determines that the F value is equal to or less than the predetermined first F value (F2.8) as illustrated in FIG. 7. And when it exceeds the predetermined second F value (F8), the focus blur correction unit 23 is caused to execute the focus blur correction, and the F value exceeds the first F value (F2.8) and the predetermined second F value ( F8) In the following cases, the focus blur correction unit 23 is not caused to perform focus blur correction. It is empirically known that out-of-focus blurring due to chip warpage is conspicuous on the open side (small F-number side), and out-of-focus correction is executed when the F-number is 2.8 or less. On the other hand, it is expected from the user that a clear image can be obtained by increasing the depth of field on the aperture side (large F value), but when the depth of field is large (F value is large), Since there are many scenes in which not only the subject but also the background are in focus, if the image sensor 11 is warped, out-of-focus blurring due to the warpage may be noticeable. Therefore, when the F value is large (when the F value on the diaphragm side exceeds F8), focus blur correction is executed.
As described above, when the depth of field is shallow (F value is small) and when the depth of field is deep (F value is large), the focus blur correction is executed, and the focus blur correction is not executed in the intermediate region. By controlling to 1, it is possible to correct the out-of-focus image caused by the warp of the chip while avoiding that the noise of the image becomes conspicuous.

(14) In the embodiment, the predetermined first F value and second predetermined F value are set according to the amount of warpage of the entire image pickup surface of the image pickup device 11, and the image pickup having such a configuration is performed. In the apparatus 10, the first F value and the second F value are set according to the range of the warp value of the image sensor 11. This is because when the range of the warp value (the range of the maximum value and the minimum value of the warp) of the entire image pickup surface of the image pickup element 11 is small, the warp value is determined by the taking lens and the F value (the amount of focus shift is clear). It often falls within the visible range). Therefore, the focus blur correction value used when executing the focus blur correction is reduced, and the image noise due to the focus blur correction is less likely to be emphasized. Therefore, it is possible to expand the range of the F value in which the focus blur correction can be executed.

(15) In the above embodiment, the predetermined first F value and second predetermined F value are determined according to the magnitude of the warp of the entire image pickup surface of the image pickup device 11, the type of the taking lens 12, and the F value. It is set according to the amount of focus shift determined.
In the image pickup apparatus 10 having such a configuration, the F value serving as a reference for determining whether or not to perform the out-of-focus correction is set by the range of the warp value of the image pickup element 11 and the focus shift amount of the taking lens 12. This is because the larger the amount of defocus and the smaller the range of the warp value, the more often the warp value falls within the range of the amount of defocus determined by the taking lens and the F value. Therefore, the focus blur correction value used when executing the focus blur correction is reduced, and the image noise due to the focus blur correction is less likely to be emphasized. Therefore, it is possible to expand the range of the F value in which the focus blur correction can be executed.

(16) In the above embodiment, when the shooting conditions are the ISO sensitivity and the F value, the control unit 13 determines whether the F value is equal to or less than the predetermined first F value (F2.8) or the predetermined F value. When the second F value (F8) is exceeded and the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity (400), the correction unit (focus blur correction unit 23) corrects the image data and F When the value exceeds the predetermined first F value (F2.8) and is equal to or less than the predetermined second F value (F8) and the ISO sensitivity is equal to or less than the predetermined ISO sensitivity (400). , The correction amount of the image data is reduced to a predetermined ratio, and the correction unit (focus blur correction unit 23) executes the correction so that the F value is equal to or less than a predetermined first F value (F2.8) or a predetermined F value. When the second F value (F8) is exceeded and the ISO sensitivity exceeds the predetermined ISO sensitivity (400), the correction amount of the image data is reduced to a predetermined ratio and the correction unit (focus blur correction unit) is used. 23), the F value exceeds the predetermined first F value (F2.8) and is equal to or smaller than the predetermined second F value (F8), and the ISO sensitivity is the predetermined ISO value. When the sensitivity exceeds 400, the correction unit (focus blur correction unit 23) is not caused to correct the image data.
This makes it possible to control whether or not to execute the focus blur correction according to the ISO sensitivity and the F value.

(17) In the above embodiment, the predetermined ISO sensitivity is set according to the shutter speed.
With this, it is possible to control whether or not to execute the out-of-focus blur correction according to the ISO sensitivity and the F value, and the ISO sensitivity and the F to become the reference (boundary) for determining whether to execute the out-of-focus blur correction. The value and can be set according to the shutter speed.

(18) In the above embodiment, the predetermined first and second F values and the predetermined ISO sensitivity are set according to the magnitude of the warp of the entire image pickup surface of the image pickup device 11.
This makes it possible to set the first and second F-numbers, which are the criteria for determining whether or not to perform the out-of-focus correction, and the ISO sensitivity according to the magnitude of the warp value of the image sensor 11.

(19) Further, in the above-described embodiment, the predetermined first and second F values and the predetermined ISO sensitivity are such that the amount of warpage of the entire image pickup surface of the image pickup device 11, the type of the taking lens, and the F value. It is set according to the focus shift amount determined by
As a result, the first and second F-numbers, which are the criteria for determining whether to perform the out-of-focus correction, and the ISO sensitivity are set according to the range of the warp value of the image sensor 11 and the focus shift amount of the taking lens 12. Can be set.

(20) Further, in the above embodiment, the correction of the image data is performed by the sharpness or unsharp mask.
Thereby, when executing the focus blur correction in the focus blur correction unit 23, the focus blur correction can be performed using a well-known sharpness or unsharp mask method.

Although the embodiments of the present invention have been described above, the image pickup apparatus of the present invention is not limited to the above-described illustrated examples, and various changes may be made without departing from the scope of the present invention. Of course.

Reference numeral 10... Imaging device, 11... Imaging element, 11A... Imaging unit, 12... Shooting lens, 13... Control unit, 14... Recording unit, 15... Storage unit, 17... Setting unit, 20... Signal processing ASIC, 21... Previous Processing unit, 22... Signal processing unit, 23... Defocus correction unit, 30... Imaging device package, 31... Substrate, 32... Seal member, 33... Protective glass

Claims (17)

An image sensor having a plurality of regions for capturing an image of a subject,
First image processing is performed on the first image signal of the subject imaged in the first area of the plurality of areas using information about the warp of the image sensor in the first area, among them, wherein the first region for the second image signal of a subject captured at different second areas, the image processing for performing second image processing using information on the warp of the imaging device in the second region Section, preparation,
The image processing unit, wherein the image processing unit generates an image based on the first image signal subjected to the first image processing and the second image signal subjected to the second image processing . A storage unit that stores information about the warp of the image sensor in the first area and information about the warp of the image sensor in the second area;
Wherein the image processing unit, stored in the storage unit, using said information about the warpage of the imaging device in the first region performs the first stroke picture processing, stored in the storage unit, the second region the imaging apparatus according to claim 1 for the second Ima process using information on the warp of the imaging device in. A substrate on which the image sensor is arranged,
The image pickup apparatus according to claim 1, wherein the substrate has a linear expansion coefficient different from a linear expansion coefficient of the image pickup element. A member for accommodating the image sensor,
The image pickup apparatus according to claim 3, wherein the member has a linear expansion coefficient different from a linear expansion coefficient of the image pickup element. A setting unit for setting conditions for capturing an image of a subject by the image sensor,
A control unit for controlling said second Ima processing said first Ima processed by the set the condition by the setting unit,
The image pickup apparatus according to any one of claims 1 to 4, further comprising: Wherein the control unit, the imaging apparatus according to claim 5, wherein the first stroke picture processing second Ima processing with different image processing to control the image processing unit to be performed. The setting unit sets the ISO sensitivity of the image sensor as the condition,
Wherein the control unit, the ISO sensitivity set by the setting unit, the imaging apparatus according to claim 5 or claim 6 for controlling said second Ima processing said first Ima process. The setting unit sets the shutter speed of the image sensor as the condition,
Wherein the control unit, the shutter speed set by the setting unit, the imaging apparatus according to any one of claims 7 claim 5 for controlling the second stroke picture processing and said first Ima treatment .. 9. The image processing unit according to claim 1, wherein the image processing unit performs sharpness processing or unsharp mask processing on image data of a subject imaged by the image sensor based on information about the warp of the image sensor. The imaging device described. An image sensor having a plurality of regions for capturing an image of a subject,
First image processing is performed on the first image signal of the subject imaged in the first area of the plurality of areas using information about the warp of the image sensor in the first area, among them, wherein the first region for the second image signal of a subject captured at different second region, different from said first image processing using information on the warp of the imaging device in the second region An image processing unit that performs second image processing, and
The image processing unit, wherein the image processing unit generates an image based on the first image signal subjected to the first image processing and the second image signal subjected to the second image processing . A storage unit that stores information about the warp of the image sensor in the first area and information about the warp of the image sensor in the second area;
Wherein the image processing unit, stored in the storage unit, performs the first image processing by using the information on the warp of the imaging device in the first region, information about the warpage of the imaging device in the second region the imaging apparatus according to claim 10 for performing a pre-Symbol second Ima process used. A substrate on which the image sensor is arranged,
The image pickup device according to claim 10, wherein the substrate has a linear expansion coefficient different from a linear expansion coefficient of the image pickup element. A member for accommodating the image sensor,
The imaging device according to claim 12, wherein the member has a linear expansion coefficient different from a linear expansion coefficient of the imaging element. A setting unit for setting conditions for capturing an image of a subject by the image sensor,
A control unit for controlling said second Ima processing said first Ima processed by the set the condition by the setting unit,
The imaging device according to any one of claims 10 to 13, further comprising: The setting unit sets the ISO sensitivity of the image sensor as the condition,
Wherein the control unit is configured by the ISO sensitivity set by the setting unit, the imaging apparatus according to claim 14 for controlling the second stroke picture processing and said first Ima process. The setting unit sets the shutter speed of the image sensor as the condition,
Wherein the control unit, the shutter speed set by the setting unit, the imaging apparatus according to claim 14 or claim 15 for controlling a second stroke picture processing and said first Ima process. 17. The image processing unit according to claim 10, wherein the image processing unit performs sharpness processing or unsharp mask processing on image data of a subject imaged by the image sensor based on information about the warp of the image sensor. The imaging device described.

Images