JP2013172201A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of reducing out-of-focus in an image generated due to warpage of an image sensor.SOLUTION: An imaging device comprises: an imaging unit that is provided with an image sensor having a photoelectric conversion function, and generates image data; a storage unit that stores information on warpage in a normal direction of an imaging surface of the image sensor; and a correction unit that corrects the image data on the basis of the information.

Description

  The present invention relates to an image pickup apparatus that picks up an image of a light beam transmitted through a photographing lens with an image pickup element.

  An image sensor (CCD or the like) is used by being housed in an image sensor package. When an image sensor (CCD, etc.) is mounted on a substrate and housed in an image sensor package, warpage occurs in the image sensor (also simply referred to as “chip”) due to differences in linear expansion coefficients of the substrate, image sensor, seal member, etc. There are things to do. For example, the chip may be warped due to a difference in coefficient of linear expansion that occurs when the chip is mounted on a substrate by soldering or when the substrate is mounted on a package with a resin (adhesive or the like). In an imaging apparatus such as a digital camera, if the imaging surface (light-receiving surface) of the imaging element is warped, the focal position is partially shifted, causing out-of-focus, and a good image cannot be obtained.

  There is a related image processing apparatus (see, for example, Patent Document 1). The image processing apparatus disclosed in Patent Document 1 is uniform even when the photographic lens type identification code of the photographic lens is not identified when correcting at least one of distortion, lateral chromatic aberration, insufficient peripheral light amount, and out-of-focus due to the photographic lens. It is another object of the present invention to provide an image processing apparatus that can reliably perform appropriate correction of the aberration without forgetting.

JP 2000-184257 A

  However, even if the focus blur correction is performed by the photographic lens as in the image processing apparatus of Patent Document 1 described above, the blur that changes concentrically due to the characteristics of the photographic lens is the target of the correction. However, the warpage occurring in the image sensor itself is not necessarily concentric, and may be a distorted shape or may vary depending on individual differences of the image sensor. For this reason, the out-of-focus caused by the warp occurring in the image pickup device itself may not be appropriately reduced with the information only for the taking lens.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus that can reduce the out-of-focus blur of an image caused by warping of the imaging element.

  The present invention has been made to solve the above-described problems, and an imaging apparatus according to the present invention includes an imaging element having a photoelectric conversion function, an imaging unit that generates image data, and an imaging surface of the imaging element. A storage unit that stores information on warpage in the normal direction, and a correction unit that corrects the image data based on the information are provided.

  According to the present invention, it is possible to reduce the out-of-focus blur of an image caused by the warp of the image sensor.

It is a block diagram which shows the structure of the imaging device concerning embodiment of this invention. It is a figure which shows the example of the curvature of an image pick-up element. It is a figure which shows the example which calculates | requires a focus blurring correction value with respect to a curvature value. It is a figure which shows the effect of focus blur correction. It is a figure showing an example in case there is no curvature in an image sensor. It is a figure which shows the example which controls focus blur correction with ISO sensitivity and shutter speed. It is a figure which shows the example which controls focus blur correction by F value. It is a figure which shows the example which controls a focus blurring correction | amendment by ISO sensitivity and F value. It is a figure which shows the example which controls focus blur correction in imaging | photography mode. It is a figure which shows the schematic image of an image pick-up element package. It is a figure which shows the example of the curvature shape and curvature value of an image pick-up element. It is a figure which shows the example of generation | occurrence | production of out-of-focus blur when there exists a curvature in a chip | tip.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[Explanation of image sensor warpage]
First, the occurrence of out-of-focus blur due to the warp of the image sensor will be supplementarily described. As described above, an image sensor (CCD or the like) is used while being housed in an image sensor package. For example, FIG. 10 is a diagram illustrating a schematic image of the image sensor package. As shown in this figure, the image pickup device package 30 is a container in which a substrate 31 on which the image pickup device 11 is mounted is fixed to a seal member 32, and the image pickup device 11 and the substrate 31 are formed with a seal member 32 and a protective glass 33. It is housed and configured. As described above, when the image pickup device 11 is mounted on the substrate 31 and stored in the image pickup device package 30, the image pickup device (chip) 11 is warped due to a difference in linear expansion coefficient among the substrate, the image pickup device, the seal member, and the like. Sometimes. For example, FIG. 11A shows an image of chip warpage. As shown in FIG. 11, in the image sensor 11, warping may occur in each direction of the three-dimensional shape (the orthogonal X, Y, and Z axis directions).

  FIG. 11B is a diagram showing an image of warping of the image sensor, and FIG. 11B is an X direction (AA ′ direction) in the image sensor 11 shown in FIG. FIG. 2 is a cross-sectional view schematically showing a state of warping of the image sensor 11. As shown in FIG. 11B, an imaginary chip reference plane Rp of the image sensor 11 is calculated, and the amount of displacement (ΔZ) in the height direction at various points from the reference plane Rp, that is, in the Z direction. Is the warp value (warp value). The upper side of the reference plane Rp is defined as plus (+) and the lower side is defined as minus (−). As described above, in an image pickup apparatus such as a digital camera, when the image pickup device (chip) 11 is housed in the image pickup device package, the image pickup surface (light receiving surface) is warped, and the focal position is partially shifted, resulting in out of focus. , A good image may not be obtained.

  For example, FIG. 12 is a diagram illustrating an example of occurrence of out-of-focus when the chip is warped. FIG. 12A shows the shape of the warp of the chip, and FIG. 12B shows an image captured by the image sensor 11.

  In FIG. 12A, the range of the region 1 is a range in which there is little warp by the image sensor 11 and does not affect the image, and the range of the region 2 is large in the warp of the image sensor 11 and has an effect on the image. It is a range. In the range where the warp of the image sensor 11 is large, as shown in the example of the image in FIG. 12B, focus blur occurs in the region 2 (portion surrounded by the broken line S), and a good image cannot be obtained. In the image pickup apparatus of the present embodiment, the out-of-focus blur generated by the warp of the image pickup element 11 is corrected.

[Configuration of imaging device]
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. The imaging apparatus shown in FIG. 1 shows a configuration example of an electronic camera such as a digital camera, and shows only a part directly related to the present invention. The imaging apparatus 10 includes an imaging unit 11A including an imaging element 11, an imaging lens 12 of an optical system, a control unit 13, a recording unit 14, a storage unit 15, a setting unit 17, a signal processing ASIC 20, It is comprised.

The imaging unit 11A includes an imaging element 11, an A / D converter (not shown), and the like. The imaging unit 11A converts a subject image formed on the light receiving surface of the imaging element 11 into an electrical signal according to the amount of received light. The image data after / D conversion is output to the signal processing ASIC 20.
The control unit 13 detects that each operation switch (not shown) such as a release button provided in the imaging apparatus 10 is operated, controls the driving of the photographing lens 12, and performs focus detection / adjustment processing in the optical system. . Further, the control unit 13 controls the operation of the signal processing ASIC 20 by inputting / outputting a synchronization signal and a control signal to / from the signal processing ASIC 20. Further, the control unit 13 transmits a warp correction control signal Sc to the signal processing ASIC 20, thereby performing warp correction processing (focus blur correction described later) in the signal processing unit 22 (more precisely, the focus blur correction unit 23). Control execution and non-execution. The recording unit 14 records the image data processed by the signal processing ASIC 20.

The storage unit 15 records the out-of-focus blur correction value calculated based on the warp value used when the out-of-focus blur correction is performed. For example, in the assembly process or adjustment process of the imaging device 10, the warp of the image sensor 11 is measured, the shape of the warp of the chip and the warp value thereof are acquired in advance, and the out-of-focus correction value is calculated from the warp value. The out-of-focus blur correction value 16 is recorded in the storage unit 15. An out-of-focus blur correction unit 23 in the signal processing unit 22 reads out the out-of-focus blur correction value 16 recorded in the storage unit 15 when an instruction to perform out-of-focus blur correction is received from the control unit 13, and an image photographed by the image sensor 11. The focus is corrected.
The setting unit 17 includes an operation unit including a power button, a shutter button, a mode selection dial, a menu button, a cross key, and the like. The user operates the setting unit 17 to set shooting conditions such as a shutter speed and ISO sensitivity, a shooting mode, and the like. Information on the photographing conditions and photographing modes set in the setting unit 17 is output to the control unit 13.

  In the imaging unit 11 </ b> A of the imaging apparatus 10, subject light incident on the photographing lens 12 is imaged on the imaging element 11 that is a CCD. The image signal output from the image sensor 11 is output to the ASIC 20 for image processing. The image signal input to the ASIC 20 for image processing is input to the preprocessing unit 21. The pre-processing unit 21 converts the image signal into a digital signal, performs image pre-processing such as gamma correction, and outputs the pre-processed image signal to the signal processing unit (post-processing unit) 22.

  The signal processing unit (post-processing unit) 22 performs processing for generating image data for display based on the image signal input from the pre-processing unit 21. In addition, the signal processing unit 22 performs focus blur correction on the image signal by the focus blur correction unit 23. Note that when performing out-of-focus correction by the out-of-focus blur correction unit 23, out-of-focus correction can be performed by using generally well-known methods such as sharpness and unsharp mask.

[Acquisition of warp value and calculation of focus blur correction value]
In the imaging apparatus 10 of the present embodiment, the imaging element 11 shown in FIG. 2A is divided into a plurality of areas as shown in FIG. 2B, and the warp value in each area is measured.
2B is a plan view (X and Y axis direction planes) of the image pickup surface (light receiving surface) of the image pickup element (chip) 11 shown in FIG. 2A viewed from the upper direction (+ direction of the Z axis). The warp value in each region is measured by dividing the plane (light receiving surface) of the image sensor 11 into a plurality of m × n (m rows, n columns) rectangular regions. And the curvature value in the center position p in each area | region is measured, and this curvature value is made into the curvature value in the said area | region. For example, when the size (chip size) of the image sensor 11 is 40 × 30 mm, the warp value in each region is measured by dividing into 35 × 24 divisions.

  When measuring the warp value, the surface shape is measured in a non-contact manner with respect to the image sensor 11 using, for example, a laser interferometer, so that the warp value is highly accurate (for example, in nanometer units or micrometer units). The warp value can be acquired. As shown in FIG. 11B described above, this warp value is a difference in height with respect to the virtual reference plane Rp, that is, a displacement amount in the plus (+) direction and a displacement amount in the minus (−) direction. Can be acquired.

  In the imaging apparatus 10 according to the present embodiment, the out-of-focus correction value calculated based on the measured warp value is stored in the storage unit 15. When calculating this out-of-focus blur correction value, as shown in FIG. 3, an optimum value of the out-of-focus blur correction value with respect to the warp value is obtained in advance based on the empirical formula c in the design stage or development stage of the imaging apparatus 10. . Then, based on this empirical formula c, the out-of-focus correction value is obtained from the warp value in each region, and is stored as the out-of-focus correction value 16 in the storage unit 15. If the warp value is small, for example, if the warp value in the region 1 shown in FIG. 3 is within the range of the amount of focus shift (the range in which the image is not out of focus even if the focus is shifted due to the characteristics of the human eye), the out-of-focus blur will occur. The correction value can also be zero (0). The amount of focus shift is a value determined by the type of lens and the F value (aperture size).

The out-of-focus blur correction unit 23 reads out the out-of-focus blur correction value from the storage unit 15 for each area shown in FIG. 2B with respect to the image signal captured by the image sensor 11, and uses this out-of-focus blur correction value to generate an image. The focus is corrected. This out-of-focus correction unit 23 adjusts the contour emphasis and detail reproducibility by performing sharpness and unsharp masking on the image, and clearly reproduces the out-of-focus portion.
In addition, when performing out-of-focus correction (for example, sharpness) in the out-of-focus blur correction unit 23, in addition to performing out-of-focus correction for an image using the out-of-focus correction value as it is, the intensity of the out-of-focus correction value is reduced as described later. The focus blur correction can also be performed (for example, by reducing the focus blur correction value to a predetermined ratio so as to be “focus blur correction value × 0.6”).

  FIG. 4 is a diagram illustrating an example of the effect of defocus correction. FIG. 4A is a diagram showing a chip shape when there is warping of the chip, and is the same view as FIG. 12A described above. In FIG. 4A, the range of region 1 is a range in which there is little warping by the chip and there is no influence on the image. That is, in the range of the region 1, the warp value is within the range of the amount of defocus, and the image is not out of focus. On the other hand, the range of the region 2 is a range in which warping by the chip is large and the image is affected. In the range of the region 2 where the warp due to the chip is large, as shown in the example of the image in FIG. 4B, a focus blur (portion surrounded by a broken line S) occurs, and a good image cannot be obtained. there. The out-of-focus blur correction unit 23 performs out-of-focus correction (for example, sharpness) on the out-of-focus (portion surrounded by the broken line S), and an image with no out-of-focus as shown in FIG. 4C can be obtained.

  As described above, by performing the focus blur correction on the warp of the image sensor 11 by the focus blur correction unit 23, the image sensor 11 can be equivalently free of warp as shown in FIG. And an image without the out-of-focus image shown in FIG. 5B can be obtained.

Note that, when performing defocus correction, the phenomenon of defocusing is the same regardless of whether the warp is on the plus (+) side or minus (−) side with respect to the reference plane Rp. However, the outline is emphasized by using a sharpness or unsharp mask where warping is affected.
Further, in the imaging device 10 of the present embodiment, the warp of the image sensor (chip) 11 is obtained from the height difference (+, −) from the reference plane Rp, but the warp is based on the curvature (bending degree) of the chip surface. May be requested.

[Control of execution and non-execution of focus blur correction]
As described above, when sharpness or unsharp masking is performed on an image, the contour emphasis and detail reproducibility are adjusted, and the out-of-focus area is clearly reproduced. However, on the other hand, the image noise is conspicuous. It becomes easy. For this reason, by performing out-of-focus correction, noise may be emphasized and displayed, and unnaturalness may appear in the image. Therefore, the imaging apparatus 10 of the present embodiment does not always perform out-of-focus correction at every shooting, but performs out-of-focus correction according to shooting conditions such as ISO sensitivity, F value, shutter speed, and shooting mode. And configured to control non-execution.

  For example, in high-sensitivity shooting with a lot of noise, there is a high possibility that noise will be conspicuously displayed due to out-of-focus correction. Therefore, it is possible to perform correction with low-sensitivity shooting with less noise, and it is possible to adjust the intensity of focus blur correction. In addition, when the depth of field is deep (the F value is large), there are many scenes where the background is in focus as well as the main subject. Therefore, if there is warping of the chip, there is a possibility that the out-of-focus blur will be noticeable. is there. Accordingly, it is possible to perform control such that correction is executed when the F value is large, and correction is not executed when the depth of field is shallow (F value is small). Also, depending on the type of shooting mode, ISO sensitivity may be higher and lower, or the F-number of the shooting lens may be larger and smaller. Can be controlled.

(Focus blur correction according to ISO sensitivity and shutter speed)
FIG. 6 is a table showing whether or not to perform out-of-focus correction according to ISO sensitivity and shutter speed, and is an example in which out-of-focus correction is performed by sharpness. The table shown in FIG. 6 shows the ISO sensitivity as a parameter in the horizontal direction and the shutter speed T as a parameter in the vertical direction. In the table, a circle indicates that the out-of-focus correction is performed, and a cross indicates that the out-of-focus correction is not performed (non-execution).

  As shown in this table, when the shutter speed T is a low speed, that is, when the shutter speed is longer than 1/8 sec, focus blur correction is executed when the ISO sensitivity is 400 or less, and the ISO sensitivity exceeds 400. Does not perform out-of-focus correction. When the shutter speed T is a high-speed second (1/8 sec or less), the focus blur correction is performed when the ISO sensitivity is 800 or less, and the focus blur correction is not performed when the ISO sensitivity exceeds 800.

  This is because image noise may be enhanced by performing out-of-focus blur correction, and there is a high possibility that noise is conspicuous due to out-of-focus blur correction at high sensitivity with much noise. Therefore, it is possible to execute the focus blur correction only with low sensitivity with less noise, and it is also possible to adjust the strength of the focus blur correction. In this case, since the exposure amount of the image plane changes depending on the shutter speed T, when the exposure amount is large (1/8 sec or more), the focus blur correction is performed with an ISO sensitivity of 400 or less, and when the exposure amount is small (1/8 sec). In the following, focus blur correction is executed at an ISO sensitivity of 800 or less. In other words, the ISO sensitivity when the exposure amount is large (ISO sensitivity as a criterion) is set lower than the ISO sensitivity when the exposure amount is small.

In this way, by selecting whether or not to perform out-of-focus correction according to the ISO sensitivity and the shutter speed T, it is possible to correct out-of-focus due to warping of the chip while avoiding image noise being noticeable. be able to.
It should be noted that when executing out-of-focus correction, it is possible to perform out-of-focus correction by reducing the intensity of the out-of-focus correction value. For example, when the shutter speed T is longer than 1/8 (T> 1/8 sec), the ISO sensitivity 800 that does not execute the focus blur correction is adjacent to the region of the ISO sensitivity 400 that performs the focus blur correction. In the case of ISO sensitivity 800, the focus blur correction intensity may be weakened (for example, the focus blur correction value is reduced to a predetermined ratio so as to be “focus blur correction value × 0.6”).

Note that the table shown in FIG. 6 shows an example, and the ISO sensitivity, which is a boundary when determining whether or not to perform focus blur correction, is the range of the warp value of the image sensor, the type of the photographing lens, and the F value. It can be changed in accordance with the amount of focus shift and the like.
For example, when the range of the warp value of the entire imaging surface of the image sensor (the range between the maximum value and the minimum value of the warp) is small, the out-of-focus correction value is small, and noise due to out-of-focus correction is less noticeable. ISO sensitivity that is a criterion (boundary) for determining whether or not to execute can be increased. On the other hand, when the range of the warp value of the image sensor is large, the amount of focus blur correction becomes large, and noise due to the focus blur correction often becomes conspicuous. Therefore, it is necessary to reduce the ISO sensitivity that is a determination criterion (boundary).
As described above, the ISO sensitivity that is a determination reference (boundary) when determining whether or not to perform the focus blur correction can be set according to the range of the warp value of the entire imaging surface of the image sensor 11.

  In addition, the ISO sensitivity that is a criterion (boundary) is the range of the focus shift amount of the photographing lens (the range in which the image is not out of focus even if the focus is shifted due to the characteristics of the human eye) and the range of the overall warp value of the image sensor (the warp value). It can be changed according to the range of the maximum value and the minimum value. This is because the out-of-focus correction value can be reduced as the defocus amount increases and the warp value range decreases. As described above, the ISO sensitivity, which is a reference (boundary) when determining whether or not to perform the focus blur correction, is determined according to the defocus amount of the photographing lens 12 and the range of the entire warp value of the image sensor 11. It can also be set.

(Focus blur correction according to F value)
FIG. 7 is a table showing whether or not focus blur correction is performed according to the F value. The table shown in FIG. 7 shows the F value as a parameter in the horizontal direction, and indicates whether or not to perform out-of-focus correction according to the F value. In the table, a circle indicates that the out-of-focus correction is performed, and a cross indicates that the out-of-focus correction is not performed (non-execution).
As shown in this table, when the F value is 2.8 or less and when the F value exceeds F8, focus blur correction is executed. When the F value exceeds F4 and is less than F8, focus blur correction is executed. do not do.

  In other words, focus blur correction is performed at the opening of the photographic lens aperture and at both ends of the aperture, and is not performed in the middle. It is empirically known that occurrence of out-of-focus blur due to chip warpage is conspicuous on the open side (side with a small F value). When the F value is 2.8 or less, out-of-focus correction is executed. On the other hand, on the diaphragm side (F value is large), the user is expected to obtain a clear image by increasing the depth of field. However, when the depth of field is deep (F value is large), Since there are many scenes that focus on the background as well as on the subject, if there is a warp of the chip, the out-of-focus blur may be noticeable. Therefore, when the F value is large (when the F value exceeds F8), the out-of-focus correction is performed.

  As described above, when the depth of field is shallow (F value is small) and when the depth of field is deep (F value is large), focus blur correction is performed, and focus blur correction is not performed in the intermediate range. By doing so, it is possible to correct out-of-focus blur caused by warping of the chip while avoiding that the noise of the image becomes conspicuous.

Note that the table shown in FIG. 7 shows an example, and the F value serving as a reference (boundary) when determining whether or not to perform focus blur correction is the range of the warp value of the image sensor and the amount of focus deviation. It can also be set according to.
For example, when the range of the warp value of the entire imaging surface of the image sensor is small, the out-of-focus correction value is small, and noise and unnaturalness due to out-of-focus blur correction are less noticeable, and thus out-of-focus correction can be performed. The range of values can be expanded. As described above, the F value serving as a reference (boundary) when determining whether or not to perform focus blur correction can be optimally set according to the magnitude of the warp value of the entire imaging surface of the image sensor 11. .

  The F value can also be changed by the amount of defocus of the taking lens and the overall warp value of the image sensor (the range of the maximum and minimum warp values). This is because the greater the defocus amount and the smaller the range of the warp value, the smaller the focus blur correction value, so that noise is less emphasized when focus blur correction is performed, and it is unnatural. This is because there is less conspicuousness. For this reason, the larger the focus shift amount is, and the smaller the range of the warp value range is, the larger the range of the F value at which the focus blur correction can be performed. As described above, the F value serving as a reference (boundary) for determining whether or not to perform the focus blur correction may be set according to the focus shift amount and the range of the overall warp value of the image sensor 11. it can.

(Defocus correction according to ISO sensitivity and F value)
FIG. 8 is a table showing whether or not the focus blur correction is performed according to the ISO sensitivity and the F value. 8A shows the shutter speed T when the shutter speed T is low (when the shutter speed is longer than 1/8 sec), and FIG. 8B shows the shutter speed T when the shutter speed T is high (the shutter speed is 1/8 sec). In the following cases). In the tables shown in FIGS. 8A and 8B, the F value is shown as a parameter in the horizontal direction, and the ISO sensitivity is shown as a parameter in the vertical direction. Depending on the F value and the ISO sensitivity, the focus blur is generated. It indicates whether or not to execute correction. In the table, a circle indicates that focus correction is performed, a cross indicates that focus correction is not performed (non-execution), and a triangle indicates the strength of the focus blur correction value when performing focus blur correction. (For example, reducing the out-of-focus blur correction value to a predetermined ratio so as to be “out-of-focus blur correction value × 0.6”) and executing the out-of-focus blur correction.

  8A, when the shutter speed is longer than 1/8 sec, the F value is F2.8 or less or exceeds F8, and the ISO sensitivity is 400 or less. In some cases, focus correction is performed. Further, when the F value exceeds F2.8 and is F8 or less and the ISO sensitivity is 400 or less, the strength of the focus blur correction value is reduced (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. Further, when the F value is less than or equal to F2.8 or exceeds F8 and the ISO sensitivity exceeds 400, the strength of the focus blur correction value is reduced (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. Further, when the F value exceeds F2.8 and is equal to or less than F8, and the ISO sensitivity exceeds 400, the out-of-focus correction is not executed.

8B, when the shutter speed is 1/8 sec or less, the F value is F2.8 or less or exceeds F8, and the ISO sensitivity is 800 or less. If it is, the out-of-focus correction is executed. Further, when the F value exceeds F2.8 and is F8 or less and the ISO sensitivity is 800 or less, the strength of the focus blur correction value is reduced (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. When the F value is less than or equal to F2.8 or exceeds F8 and the ISO sensitivity exceeds 800, the strength of the focus blur correction value is decreased (the focus blur correction value is reduced to a predetermined ratio). ) Perform out-of-focus correction. Further, when the F value exceeds F2.8 and is equal to or less than F8, and the ISO sensitivity exceeds 800, the out-of-focus correction is not executed.
In this way, it is possible to determine whether or not the focus blur correction unit 23 performs the focus blur correction based on the ISO sensitivity and the F value. Further, the ISO sensitivity used for the determination can be changed according to the shutter speed.

  The table shown in FIG. 8 shows an example, and the ISO sensitivity and the F value that serve as a reference (boundary) when determining whether or not to perform focus blur correction are the same as those in FIGS. 6 and 7. Similarly, it can be changed according to the range of the warp value of the image sensor and the focus shift amount.

(Defocus correction according to shooting mode)
FIG. 9 is a table showing whether or not focus blur correction is performed according to the shooting mode. In the table shown in FIG. 9, when shooting portrait, landscape, children's snap, sports, and night view as shooting modes (shooting targets), focus blur correction is performed and macros, dishes, and pets are shot. If this is the case, focus correction will not be performed. Depending on the type of shooting mode, the ISO sensitivity may be increased and decreased, or the F value may be increased and decreased. Therefore, depending on the shooting mode, execution of focus blur correction, non-execution control, and focus blur may be performed. Allows the intensity of correction to be controlled. In this way, by determining whether or not to perform out-of-focus correction according to the shooting mode (shooting target), a shooting mode in which noise or unnaturalness is estimated to be inconspicuous is selected, and out-of-focus is selected. Correction can be performed.

Note that the selection of the shooting mode (shooting target) is performed by a user who operates the camera by using, for example, a menu setting button (for example, a dial button) or a menu screen.
Further, the table shown in FIG. 9 shows an example, and the shooting mode for determining whether or not to perform the focus blur correction according to the configuration of the imaging device, the type of the shooting lens, the chip size of the imaging device, and the like is as follows. It can be changed.

(Out-of-focus correction according to the amount of defocus and the range of the warp value)
As described above, the example of controlling whether or not to perform the focus blur correction based on various shooting conditions (ISO sensitivity, F value, shooting mode) has been described. Here, depending on the focus shift amount and the range of the warp value. The out-of-focus correction will be supplementarily described.

  In the imaging apparatus, the F value can be adjusted by adjusting the light quantity of the photographing lens with a diaphragm. By changing the F value, the range in which the image can be clearly captured before and after the focus changes, and the focus width (focus shift amount) of the image surface (light receiving surface) on the image sensor (chip) 11 changes. . For example, increasing the F value (decreasing the aperture) increases the amount of focus shift. Therefore, if the warp value of the image sensor (chip) 11 is within the range of the focus shift amount, the out-of-focus is not generated.

For this reason, in a certain area on the image sensor 11, if the warp value is within the range of the amount of defocus, it is not necessary to perform out-of-focus correction. Then, when the warp value of the chip is out of the range of the defocus amount, it is determined whether or not to perform the focus blur correction based on the above-described shooting conditions such as the ISO sensitivity, the F value, and the shooting mode. Become.
As described above, when the warp value is within the range of the focus shift amount, the focus blur correction is not performed, and only when the warp value exceeds the range of the focus shift amount, the focus blur correction is performed according to the imaging conditions such as the ISO sensitivity. It can be determined whether or not.

  As described above, in the imaging apparatus 10 according to the present embodiment, the out-of-focus image can be obtained by performing the out-of-focus correction for the warp of the image sensor 11 by the out-of-focus correction unit 23. In particular, when a large sensor chip such as a digital single lens reflex camera (DSLR) is used, warpage also increases. In addition, with the advance of technology, pixels are miniaturized, and the effect of out-of-focus is increasing. In such a case, it is possible to effectively reduce out-of-focus by using the configuration of the present embodiment.

  Although the embodiment of the present invention has been described above, the correspondence between the present invention and the above embodiment will be supplementarily described. In the above embodiment, the imaging apparatus according to the present invention corresponds to the imaging apparatus 10 illustrated in FIG. 1, the imaging element according to the present invention corresponds to the imaging element (chip) 11, and the imaging unit according to the present invention corresponds to the imaging element 11. In addition, the imaging unit 11A that detects an image of a subject and outputs image data of the subject corresponds. The control unit in the present invention corresponds to the control unit 13, and the correction unit in the present invention corresponds to the out-of-focus correction unit 23 in the signal processing ASIC 20. The storage unit in the present invention corresponds to the storage unit 15, and the setting unit in the present invention corresponds to the setting unit 17. Further, the information regarding the warp in the present invention corresponds to the warp value and the out-of-focus correction value calculated based on the warp value.

(1) In the above-described embodiment, the imaging apparatus 10 includes the imaging element 11 having a photoelectric conversion function, and information relating to the imaging unit 11A that generates image data and the warp in the normal direction of the imaging surface of the imaging element 11. And a correction unit (focus blur correction unit 23) that corrects image data based on the above information.
In the imaging apparatus 10 having such a configuration, the warp value of the imaging element (chip) is measured in an assembly process or an adjustment process of the imaging apparatus. Then, a focus blur correction value is calculated from the warp value, and the focus blur correction value is stored in the storage unit 15 of the imaging device 10 in advance. Then, the out-of-focus correction is performed on the photographed image using the out-of-focus correction value stored in the storage unit 15. When performing out-of-focus correction, a known sharpness or unsharp mask method is used.
As a result, it is possible to reduce the out-of-focus image blur caused by the warp of the image sensor.

(2) In the above embodiment, the storage unit 15 stores the information for each of a plurality of regions on the imaging surface, and the correction unit (focus blur correction unit 23) is based on the information stored for each region. The image data corresponding to the area is corrected.
In the imaging apparatus 10 having such a configuration, a warp value in each region of the imaging element (chip) is measured in an assembly process or an adjustment process of the imaging apparatus. Then, a focus blur correction value is calculated from the warp value, and the focus blur correction value is stored in the storage unit 15 of the imaging device 10 in advance. Then, for each area of the photographed image, the out-of-focus correction is performed using the out-of-focus correction value stored in the storage unit 15. When performing out-of-focus correction, a known sharpness or unsharp mask method is used.
As a result, it is possible to reduce the out-of-focus image blur caused by the warp of the image sensor.

(3) In the above-described embodiment, the imaging apparatus 10 includes the setting unit 17 that sets the shooting conditions of the imaging unit 11A, and the correction unit (the out-of-focus correction unit 23) according to the shooting conditions set by the setting unit 17. And a control unit 13 for controlling whether or not the correction of the image data is executed.
In the imaging apparatus 10 having such a configuration, the control unit 13 controls whether or not the focus blur correction unit 23 performs the focus blur correction according to the shooting conditions (for example, ISO sensitivity, F value, shooting mode, and the like). . Sharpness, unsharp mask, etc. are well known as methods for correcting out-of-focus, but when using these methods to correct out-of-focus, noise may be emphasized and unnaturalness may be noticeable. There is. Therefore, the correction is not necessarily performed every time shooting is performed, but the execution / non-execution of the out-of-focus correction or the intensity of the out-of-focus correction is controlled according to the shooting conditions.
Accordingly, it is possible to control whether or not to perform the focus blur correction according to the shooting condition. For this reason, it can be avoided that the noise of the image is emphasized by the out-of-focus correction and the unnaturalness is conspicuous.

(4) In the above embodiment, the photographing condition is ISO sensitivity, and the control unit 13 determines whether or not to perform image data correction by the correction unit (focus blur correction unit 23) according to the ISO sensitivity value. Control.
In the imaging apparatus 10 having such a configuration, for example, when the ISO sensitivity is low, that is, the focus blur correction is executed only with low sensitivity with low noise.
Thereby, when the ISO sensitivity is high and a lot of image noise is included, it can be avoided that the image noise is emphasized by the out-of-focus correction and the unnaturalness is conspicuous.

(5) In the above embodiment, the photographing condition is the F value of the photographing lens, and the control unit 13 determines whether or not to perform image data correction by the correction unit (focus blur correction unit 23) according to the F value. Control.
In the imaging apparatus 10 having such a configuration, whether or not to perform focus blur correction is controlled in the focus blur correction unit 23 according to the magnitude of the F value (image brightness and depth of field).
Thereby, for example, when the F value is small, that is, when the depth of field (more precisely, the amount of defocus) is small and the blur due to the warp of the image sensor 11 is likely to occur, this blur can be corrected. .

(6) In the above embodiment, the photographing conditions are the ISO sensitivity and the F value, and the control unit 13 corrects the image data by the correction unit (focus blur correction unit 23) according to the ISO sensitivity and the F value. Control whether to execute.
In the imaging apparatus 10 having such a configuration, whether or not the focus blur correction unit 23 performs the focus blur correction is controlled according to the ISO sensitivity level and the F value.
Accordingly, it is possible to finely control whether or not the focus blur correction unit 23 performs the focus blur correction based on both the ISO sensitivity and the F value parameters.

(7) In the above embodiment, the shooting condition is the shooting mode, and the control unit 13 controls whether or not to perform the correction of the image data by the correction unit (focus blur correction unit 23) according to the shooting mode. .
In the imaging apparatus 10 having such a configuration, the focus blur correction unit 23 controls whether or not to perform focus blur correction according to the shooting mode. This is because the ISO sensitivity may be set higher or lower depending on the type of shooting mode, and the F value of the shooting lens may be set larger or smaller. Enable to control the intensity of non-execution or out-of-focus correction.
Thus, by selecting the shooting mode, it is possible to obtain a clear image without out-of-focus according to the shooting mode.

(8) In the above-described embodiment, the control unit 13 determines the amount of image data generated by the correction unit (the out-of-focus correction unit 23) according to the amount of warp and the amount of focus shift determined by the type of the photographing lens and the F value. Controls whether correction is performed.
In the imaging apparatus 10 having such a configuration, when the range of the warp value exceeds the defocus amount determined by the type of the photographing lens 12 and the F value, it is controlled whether or not the focus blur correction is performed according to the imaging condition. In the imaging apparatus 10, the F value can be adjusted by adjusting the light amount of the photographic lens with a diaphragm. By changing the F value, the range in which the image can be clearly captured before and after the focus changes, and the focus width (focus shift amount) of the image plane on the image sensor (chip) 11 changes. Therefore, if the warp value of the image sensor 11 is within the range of the focus shift amount, the out-of-focus is not generated. For this reason, in a certain area on the image sensor 11, if the warp value is within the range of the focus shift amount, the focus blur correction is not performed, and if the warp value is outside the range of the focus shift amount, the focus blur is performed according to the above imaging condition. Perform correction.
Thereby, if the range of the warp value is within the range of the focus shift amount, it is possible to prevent the out-of-focus correction.

(9) In the above embodiment, when the shooting condition is ISO sensitivity, the control unit 13 corrects image data to the correction unit (focus blur correction unit 23) when the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity. When the predetermined ISO sensitivity is exceeded, the correction unit (out-of-focus correction unit 23) is not allowed to execute correction of image data.
In the imaging apparatus 10 having such a configuration, when the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity, the focus blur correction unit 23 is caused to perform focus blur correction. Since the out-of-focus correction also has an effect of enhancing the noise, there is a high possibility that the noise will be noticeable by the correction at high sensitivity with a lot of noise. Therefore, it is possible to perform correction with only low sensitivity with less noise, and to adjust the intensity.

(10) In the above embodiment, the predetermined ISO sensitivity is set according to the shutter speed.
In the imaging apparatus 10 having such a configuration, ISO sensitivity for determining whether or not to perform focus blur correction is set according to the shutter speed. This is because the exposure amount of the image pickup device 11 is determined according to the ISO sensitivity and the shutter speed, and therefore, when the shutter speed is slow (when the exposure amount is large), the ISO sensitivity for determining whether to perform the focus blur correction is lowered. Set to.
Accordingly, it is possible to set the ISO sensitivity for determining whether or not to perform the focus blur correction according to the shutter speed. For this reason, it is possible to avoid noise enhancement due to out-of-focus correction.

(11) In the above embodiment, the predetermined ISO sensitivity is set according to the warpage magnitude of the entire imaging surface of the imaging device 11.
In the imaging apparatus 10 having such a configuration, the ISO sensitivity as a reference for determining whether or not to perform the focus blur correction is set to a range of the warp value (range of the maximum value and the minimum value of the warp) of the entire imaging surface of the image sensor 11. Set according to. This is because, as the warp value range is smaller, the out-of-focus blur correction value used when performing out-of-focus blur correction is reduced, and image noise due to out-of-focus blur correction is less likely to be emphasized. For this reason, it is possible to increase the value of the ISO sensitivity that is the determination criterion (boundary) for performing the blur correction.
As a result, the ISO sensitivity, which is a determination reference (boundary) for performing the focus blur correction, can be optimally set according to the range of the warp value of the image sensor 11.

(12) In the above-described embodiment, the predetermined ISO sensitivity is set according to the amount of warpage of the entire imaging surface of the imaging device 11 and the amount of focus shift determined according to the type of the photographing lens and the F value. The
In the imaging apparatus 10 having such a configuration, the ISO sensitivity, which is a reference for determining whether or not to perform defocus correction, the range of the warp value of the entire imaging surface of the imaging element 11, the focus shift amount of the imaging lens 12, and Set by. This is because as the defocus amount is larger and the warp value range (maximum and minimum warp range) is smaller, the out-of-focus blur correction value used when executing the out-of-focus blur correction is reduced. Noise is less emphasized. For this reason, it is possible to increase the value of the ISO sensitivity that is the determination criterion (boundary) for performing the blur correction.
As a result, the ISO sensitivity, which is a determination criterion (boundary) for performing defocus correction, can be optimally set according to the range of the warp value of the image sensor 11 and the focus shift amount of the photographic lens 12.

(13) In the above embodiment, when the imaging condition is the F value, the imaging unit 10 determines that the control unit 13 has the F value equal to or smaller than the predetermined first F value and the predetermined second F value. When the value exceeds the value, the correction unit (focus blur correction unit 23) corrects the image data. When the F value exceeds the first F value and is equal to or less than the predetermined second F value, the correction unit (focus blur correction). Section 23) does not execute correction of image data.
In the imaging apparatus 10 having such a configuration, when the shooting condition is an F value, the control unit 13 causes the F value to be equal to or less than a predetermined first F value (F2.8) as illustrated in FIG. When the predetermined second F value (F8) is exceeded, the out-of-focus blur correction unit 23 performs the focus blur correction so that the F value exceeds the first F value (F2.8) and the predetermined second F value ( F8) The out-of-focus correction unit 23 is not allowed to execute out-of-focus correction in the following cases. It is empirically known that occurrence of out-of-focus blur due to chip warpage is conspicuous on the open side (side with a small F value). When the F value is 2.8 or less, out-of-focus correction is executed. On the other hand, on the diaphragm side (F value is large), the user is expected to obtain a clear image by increasing the depth of field. However, when the depth of field is deep (F value is large), Since there are many scenes in which the background is in focus as well as the subject, if the image sensor 11 is warped, there is a possibility that the out-of-focus blur will be noticeable. Therefore, when the F value is large (when the F value on the aperture side exceeds F8), focus blur correction is executed.
As described above, when the depth of field is shallow (F value is small) and when the depth of field is deep (F value is large), focus blur correction is performed, and focus blur correction is not performed in the intermediate range. With this control, it is possible to correct out-of-focus blur caused by chip warp while avoiding image noise becoming conspicuous.

(14) In the above embodiment, the predetermined first F value and the second F value are set according to the amount of warpage of the entire imaging surface of the imaging device 11. In the apparatus 10, the first F value and the second F value are set according to the range of the warp value of the image sensor 11. This is because, when the range of the warp value (the range between the maximum value and the minimum value of the warp) of the entire imaging surface of the image sensor 11 is small, the amount of focus shift (the image becomes clear) where the warp value is determined by the photographing lens and the F value. It often falls within the range of (visible range). For this reason, the out-of-focus correction value used when executing out-of-focus correction is reduced, and image noise due to out-of-focus correction is less likely to be emphasized. For this reason, it is possible to expand the range of the F value in which focus blur correction can be performed.

(15) In the above-described embodiment, the predetermined first F value and the second F value depend on the warp size of the entire imaging surface of the image sensor 11, the type of the photographing lens 12, and the F value. It is set according to the determined amount of defocus.
In the imaging apparatus 10 having such a configuration, an F value serving as a reference for determining whether or not to perform focus blur correction is set by the range of the warp value of the imaging element 11 and the amount of focus shift of the photographing lens 12. This is because the greater the defocus amount and the smaller the range of the warp value, the more the warp value falls within the defocus amount range determined by the photographing lens and the F value. For this reason, the out-of-focus correction value used when executing out-of-focus correction is reduced, and image noise due to out-of-focus correction is less likely to be emphasized. For this reason, it is possible to expand the range of the F value in which focus blur correction can be performed.

(16) In the above embodiment, when the shooting conditions are the ISO sensitivity and the F value, the control unit 13 determines whether the F value is equal to or less than a predetermined first F value (F2.8). When the second F value (F8) is exceeded and the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity (400), the correction unit (focus blur correction unit 23) performs image data correction, and F When the value exceeds the predetermined first F value (F2.8) and is equal to or lower than the predetermined second F value (F8) and the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity (400) Then, the correction amount of the image data is reduced to a predetermined ratio and the correction unit (focus blur correction unit 23) performs correction, and the F value is equal to or less than a predetermined first F value (F2.8) or a predetermined value. When the second F value (F8) is exceeded and the ISO sensitivity is When the ISO sensitivity (400) is exceeded, the correction amount of the image data is reduced to a predetermined ratio and the correction unit (focus blur correction unit 23) executes correction, and the F value is set to a predetermined first F value (F2). .8) and below a predetermined second F value (F8), and when the ISO sensitivity exceeds the predetermined ISO sensitivity (400), the image data is sent to the correction unit (focus blur correction unit 23). Do not execute the correction.
Thereby, it is possible to control whether or not to perform out-of-focus correction according to the ISO sensitivity and the F value.

(17) In the above embodiment, the predetermined ISO sensitivity is set according to the shutter speed.
Thus, it is possible to control whether or not to perform out-of-focus correction according to the ISO sensitivity and the F value, and to determine whether or not to execute out-of-focus correction. The value can be set according to the shutter speed.

(18) In the above-described embodiment, the predetermined first and second F values and the predetermined ISO sensitivity are set according to the warpage of the entire imaging surface of the image sensor 11.
As a result, the first and second F values and the ISO sensitivity, which are criteria for determining whether or not to perform focus blur correction, can be set according to the magnitude of the warp value of the image sensor 11.

(19) In the above embodiment, the predetermined first and second F values and the predetermined ISO sensitivity are determined by the amount of warpage of the entire imaging surface of the image sensor 11, the type of the photographing lens, and the F value. It is set according to the amount of focus shift determined by.
As a result, the first and second F values and the ISO sensitivity, which are the criteria for determining whether or not to perform focus blur correction, are set according to the range of the warp value of the image sensor 11 and the amount of focus shift of the photographic lens 12. Can be set.

(20) In the above embodiment, the correction of the image data is performed by a sharpness or unsharp mask.
Thus, when the out-of-focus correction unit 23 performs out-of-focus blur correction, it is possible to perform out-of-focus correction using a well-known sharpness or unsharp mask technique.

  Although the embodiments of the present invention have been described above, the image pickup apparatus of the present invention is not limited to the illustrated examples described above, and various modifications can be made without departing from the scope of the present invention. Of course.

DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Image pick-up element, 11A ... Imaging part, 12 ... Shooting lens, 13 ... Control part, 14 ... Recording part, 15 ... Memory | storage part, 17 ... Setting part, 20 ... ASIC for signal processing, 21 ... Previous Processing unit 22 ... Signal processing unit 23 ... Defocus blur correction unit 30 ... Imaging element package 31 ... Substrate 32 ... Seal member 33 ... Protective glass

Claims (20)

An imaging unit that includes an imaging device having a photoelectric conversion function and generates image data;
A storage unit for storing information on warpage in the normal direction of the imaging surface of the imaging element;
A correction unit that corrects the image data based on the information;
An imaging apparatus comprising: The storage unit stores the information for each of a plurality of regions on the imaging surface,
The imaging apparatus according to claim 1, wherein the correction unit corrects image data corresponding to the region based on the information stored for each region. A setting unit for setting shooting conditions of the imaging unit;
A control unit that controls whether or not the correction of the image data by the correction unit is executed according to the shooting condition set by the setting unit;
The imaging apparatus according to claim 1, further comprising: The shooting condition is ISO sensitivity,
The controller is
The imaging apparatus according to claim 3, wherein whether or not the correction of the image data by the correction unit is executed is controlled according to the ISO sensitivity value. The photographing condition is an F value of a photographing lens,
The controller is
The imaging apparatus according to claim 3, wherein whether to correct the image data by the correction unit is controlled according to the F value. The shooting conditions are ISO sensitivity and F value,
The controller is
The imaging apparatus according to claim 3, wherein whether or not the correction of the image data by the correction unit is executed is controlled according to the ISO sensitivity and an F value. The shooting condition is a shooting mode,
The controller is
The imaging apparatus according to claim 3, wherein whether or not the correction of the image data by the correction unit is executed is controlled according to the shooting mode. The controller is
The control unit controls whether or not to perform the correction of the image data by the correction unit according to the size of the warp and the amount of focus shift determined by the type of the photographing lens and the F value. 8. The imaging device according to any one of items 7. When the shooting condition is ISO sensitivity,
The controller is
When the ISO sensitivity is equal to or lower than a predetermined ISO sensitivity, the correction unit executes correction of the image data,
When the predetermined ISO sensitivity is exceeded, the correction unit is not allowed to execute correction of the image data.
The imaging apparatus according to claim 4. The imaging apparatus according to claim 9, wherein the predetermined ISO sensitivity is set according to a shutter speed. The imaging apparatus according to claim 9, wherein the predetermined ISO sensitivity is set according to a warp magnitude of an entire imaging surface of the imaging element. The predetermined ISO sensitivity is set according to a warp magnitude of the entire image pickup surface of the image pickup device and a focus shift amount determined according to a type of an imaging lens and an F value. The imaging device described. When the shooting condition is an F value,
The controller is
When the F value is equal to or less than a predetermined first F value and exceeds a predetermined second F value, the correction unit executes correction of the image data,
When the F value exceeds the first F value and is equal to or less than the predetermined second F value, the correction unit does not execute correction of the image data.
The imaging apparatus according to claim 5. The imaging apparatus according to claim 13, wherein the predetermined first F value and the second F value are set according to a warp magnitude of the entire imaging surface of the imaging element. The predetermined first F value and the second F value are set according to the amount of warpage of the entire image pickup surface of the image pickup device and the amount of focus shift determined according to the type of the photographing lens and the F value. The imaging device according to claim 13. When the shooting conditions are ISO sensitivity and F value,
The controller is
When the F value is equal to or lower than a predetermined first F value or exceeds a predetermined second F value, and the ISO sensitivity is equal to or lower than a predetermined ISO sensitivity, Execute image data correction,
When the F value exceeds the predetermined first F value and is equal to or lower than the predetermined second F value, and the ISO sensitivity is equal to or lower than the predetermined ISO sensitivity, the correction of the image data is performed. Reduce the amount to a predetermined percentage and let the correction unit perform correction,
When the F value is equal to or less than a predetermined first F value or exceeds a predetermined second F value, and the ISO sensitivity exceeds a predetermined ISO sensitivity, the correction amount of the image data Is reduced to a predetermined percentage and the correction unit executes correction,
When the F value exceeds the predetermined first F value and is equal to or less than the predetermined second F value, and the ISO sensitivity exceeds a predetermined ISO sensitivity, the correction unit receives the image Do not perform data correction,
The imaging apparatus according to claim 6. The imaging apparatus according to claim 16, wherein the predetermined ISO sensitivity is set according to a shutter speed. The predetermined first and second F values and the predetermined ISO sensitivity are set in accordance with the amount of warpage of the entire image pickup surface of the image pickup device. Imaging device. The predetermined first and second F values and the predetermined ISO sensitivity depend on the amount of warpage of the entire imaging surface of the image sensor and the amount of focus shift determined by the type of the photographing lens and the F value. The imaging apparatus according to claim 16, wherein the imaging apparatus is set as follows.   The image pickup apparatus according to claim 1, wherein the correction of the image data is performed by a sharpness or an unsharp mask.

Images