WO2012132685A1

WO2012132685A1 - Focus extending optical system and imaging system

Info

Publication number: WO2012132685A1
Application number: PCT/JP2012/054563
Authority: WO
Inventors: 慶延岸根
Original assignee: 富士フイルム株式会社
Priority date: 2011-03-31
Filing date: 2012-02-24
Publication date: 2012-10-04
Also published as: JP2014115303A

Abstract

Provided are a focus extending optical system and an imaging system that are able to acquire a natural image wherein a nearly uniform resolution is obtained across the entire image and disturbance of the resolution distribution at the portion deviating from the optical axis is cancelled. The focus extending optical system (11) is provided with an optical lens (11) and a focus extender (16) that extends the focal range by adjusting the wavefront in a manner so as to change the image formation position of the optical lens (11) in accordance with distance from the optical axis (L0). The wavefront (ψ) after passing through the optical lens (11) and the focus extender (16) is represented by the formula ψ=ΣK_j·Z_j, which has the Zernicke polynomial Z_j(n,m) as each term. An MTC in common in the sagittal direction and the tangential direction when the coefficient K₁₂ of the 12th term Z₁₂(n=4, m=2) is zero in the polynomial is used as a reference value. At the time of this reference value, the coefficient K₁₂ is prescribed in a range in which the MTF gap of the sagittal direction and the tangential direction is less than twice the reference value.

Description

Focus expansion optical system and imaging system

The present invention provides an optical system that expands the depth of focus of an optical lens (hereinafter referred to as a focus expansion optical system), and an image that is sharpened by performing restoration processing (deconvolution processing) on an image captured through the depth of focus expansion optical system. About the system.

Mobile phones, PDAs, small notebook personal computers, etc. are equipped with digital cameras as standard. Conventionally, in order to manufacture a digital camera mounted on such a cellular phone or the like in a small size and at a low cost, for example, a fixed focus lens is generally used. However, in recent years, such simple digital cameras are also required to improve the image quality of captured images.

A digital camera mounted on a cellular phone or the like is an image that is several tens of centimeters away from a subject such as a person or landscape several meters away, or a subject that is several meters away, for reading characters or two-dimensional codes. It is also used for imaging. To shoot at a wide range of subject distances, it is necessary to focus according to the shooting distance. However, a digital camera mounted on a mobile phone or the like must be small and inexpensive. It is difficult to provide a focus adjustment mechanism.

For these reasons, in recent years, EDoF (Extended 装置 Depth) is a digital camera that can cover an imaging distance range from a macro shooting range of about several tens of centimeters to almost infinity in terms of cost, etc., in terms of cost and the like. of Field) imaging systems are being used (Patent Documents 1 to 3). The EDoF imaging system uses a phase expansion plate or the like to pick up an image using a focus expansion optical system having a different focal length depending on the distance from the optical axis (incident height), and sharpens the resulting defocused image by restoration processing. This is an imaging system that obtains an image equivalent to that captured by a lens having a wide depth.

JP 2010-213274 A JP 2007-206738 A JP 2006-094471 A

The EDoF imaging system has the advantage of being able to capture an image with an extended depth of field by a combination of a focus extension optical system and image processing. On the other hand, in order to use a special lens system called a focus extension optical system, A unique problem arises in EDoF imaging systems.

Specifically, an image at the center of the image near the optical axis can obtain a constant resolution, but an image at a portion outside the optical axis has a problem that the resolution varies depending on the image height. Thus, if the resolution differs for each image height, even after restoration processing, a difference in resolution occurs between the horizontal direction and the vertical direction, resulting in an uncomfortable image. In addition, the resolution is not so constant that the resolution decreases as it approaches the peripheral portion, but the resolution becomes lower at an intermediate position slightly away from the center of the image on the basis of the resolution of the central portion of the image on the optical axis. The part has a complicated resolution distribution such as a slightly improved resolution. Such a resolution distribution causes locally unnatural blur when a distant view such as a landscape or a person is imaged, and causes the entire distant image to be uncomfortable. In addition, when a close-up image such as a character or a two-dimensional code is imaged, there is a problem that the character cannot be read.

An object of the present invention is to provide a focus expanding optical system and an imaging system capable of eliminating a disturbance in the resolution distribution of a portion off the optical axis and acquiring a natural image that can obtain a substantially constant resolution over the entire image. To do.

The focus extending optical system of the present invention has at least one optical lens that forms an image of light from a subject on an image sensor, and a wavefront so as to change an imaging position by the optical lens according to a distance from the optical axis. And a focus expansion element that adjusts and expands the focus of the optical lens. The wavefront ψ after passing through the optical lens and the focus expanding element is expressed by the formula ψ = ΣK _j · Z _j with Zernike polynomials Zj (n, m) as terms. In this polynomial, when the MTF common to the sagittal direction and the tangential direction when the coefficient K ₁₂ of the twelfth term Z ₁₂ (n = 4, m = 2) is 0 is used as a reference value, the sagittal direction and the tangential as the gap in the direction of MTF is within a range smaller than twice the reference value, the coefficient K ₁₂ is defined.

The absolute value | K ₁₂ | of the coefficient K ₁₂ preferably satisfies | K ₁₂ | <0.3.

The coefficient K _{13 of} the twelfth term Z ₁₃ (n = 4, m = −2) is such that the gap between the sagittal and tangential MTFs is smaller than twice the reference value with the MTF at K ₁₃ = 0 as the reference value. as will be within the range, it is preferable that the coefficient K ₁₃ is defined.

The absolute value | K ₁₃ | of the coefficient K ₁₃ preferably satisfies | K ₁₃ | <0.3.

In MTF of sagittal direction and the tangential direction with respect to the coefficients K ₁₂ is within the following point becomes zero, it is preferable that the coefficient K ₁₂ is defined.

The absolute value | K ₁₃ | of the coefficient K ₁₃ preferably satisfies | K ₁₃ | <0.275.

Sagittal MTF tangential direction of the MTF twice to become the value of the first threshold value of the coefficient _{K 12} Th1, the tangential value of the second threshold value of the coefficient _{K 12} of tangential direction of the MTF is twice the sagittal MTF Let Th2. In this case, it is preferable that the coefficient _{K 12} is a value within range satisfying the _Th1 <K 12 <Th2 or _{Th2 <K} 12 _<Th1,.

The absolute value | K ₁₃ | of the coefficient K ₁₃ preferably satisfies | K ₁₃ | <0.08.

In all of the image height, the sagittal direction and tangential within a small range than twice the tangential direction of the reference value gap of MTF, it is preferable that the coefficient K ₁₂ is defined.

It is preferable that the optical lens and the focus expanding element are fixed so as not to move in the optical axis direction.

The optical lens or the focus expanding element is preferably provided so as to be movable in the optical axis direction.

An imaging system according to the present invention includes an imaging device that captures an image of a subject, at least one optical lens that forms light from the subject on the imaging device, and an imaging position of the optical lens according to a distance from the optical axis. And a focal point expansion element that adjusts the wavefront to vary and expands the focal point of the optical lens. Further, the wavefront ψ after passing through the optical lens and the focus expanding element is expressed by the equation ψ = ΣK _j · Z _j with Zernike polynomials Zj (n, m) as terms. In this polynomial, an MTF common to the sagittal direction and the tangential direction when the coefficient K _{12 of} the twelfth term Z ₁₂ (n = 4, m = 2) is 0 is used as a reference value. At this time, the focus expansion optical system, like the gap of the MTF of sagittal direction and the tangential direction is within a range smaller than twice the reference value, the coefficient K ₁₂ is defined.

It is preferable to provide an image processing unit that generates an image equivalent to an image photographed by an optical system having a wide depth of field by performing a restoration process on data output from the image sensor.

According to the present invention, it is possible to provide a focus expanding optical system and an EDoF imaging system that can eliminate a disturbance in the resolution distribution of a portion off the optical axis and obtain a natural image having a substantially constant resolution in the entire image. be able to.

It is a block diagram which shows the structure of an EDoF imaging system. It is explanatory drawing which shows the structure of a focus expansion optical system. It is a figure which shows typically the RAW data showing the image of the arbitrary points of a to-be-photographed object. It is a graph which represents typically the distribution of the pixel value of the point image in RAW data. 4 is a graph schematically showing a frequency MTF of RAW data. It is a figure which shows typically the image data of the point image after a decompression | restoration process. It is a graph which represents typically distribution of a pixel value of a point image after restoration processing. It is a graph which represents typically frequency MTF after restoration processing. It is a graph which shows the axial defocus MTF of a normal lens and a focus expansion optical system. It is a graph which shows the off-axis defocus MTF of the focus expansion optical system in intermediate image height. It is a graph which shows the off-axis defocus MTF of the focus expansion optical system in peripheral image height. It is an image which shows the example of the RAW data of the foreground image obtained by defocusing and imaging on an optical axis. It is an image which shows the example of the RAW data of the distant view image obtained by imaging on the optical axis with the best focus. It is an image which shows the example of the raw data of the foreground image obtained by defocusing and imaging in intermediate image height. It is an image which shows the example of the RAW data of the distant view image obtained by imaging with the best focus in intermediate image height. It is an image which shows the example of the RAW data of the foreground image obtained by defocusing and imaging in peripheral image height. It is an image which shows the example of the RAW data of the distant view image obtained by imaging with the best focus in peripheral image height. It is a graph which shows the change of MTF of S direction and T direction to coefficient K12. It is an explanatory diagram showing a relationship of Z ₁₂ and Section _{Z 13} Section.

As shown in FIG. 1, the EDoF imaging system 10 includes a focus extending optical system 11, an imaging element 12, and an image processing unit 13. The EDoF imaging system 10 is mainly used for two types of applications: imaging of a distant view from a few meters ahead, such as a landscape or a person, and imaging of a close view that captures a distance of several tens of centimeters, such as a character or a two-dimensional code. used.

The focus expansion optical system 11 is for imaging light incident from the subject 14 on the image sensor 12, and includes a plurality of optical lenses and a focus expansion element for extending the depth of focus of the optical lens, as will be described later. Including. The depth-of-focus optical system 11 converges, for example, a light beam near the center including the optical axis toward the front (subject 14), and converges a light beam passing outside toward the back (imaging element 12), depending on the incident height. The focus changes.

The optical performance of the focus expanding optical system 11 is designed to be a performance required by a specific EDoF imaging system 10, but in this embodiment, a fixed focus lens having an F number (Fn) of 2.29 is used. The resolving power δ of the optical lens is inversely proportional to the F number (Fn), and if the reference wavelength is λ (nm), δ = 1 / Fn / λ. Therefore, if the reference wavelength λ is 546 nm, the resolving power δ of the focus extending optical system 11 is about 800 lines / mm, which is almost equal to the sampling frequency fs of the image sensor 12 described later, and about twice the Nyquist frequency Ny. .

As described later, the focal expansion optical system 11, when represented by the formula (so-called Zernike mode) to each term wavefront aberration Zernike the (Zernike) polynomial, absolute coefficient K ₁₂ of paragraph 12 Z ₁₂ The shape of each lens surface is determined so that the value | K ₁₂ | is 0.3 or less. As a result, the EDoF imaging system 10 projects the subject at the same shooting distance with a uniform resolution in the image by focus shift (movement of focus). Further, when the image is blurred, a substantially uniform blur is realized vertically and horizontally at any position in the acquired image 15.

The image pickup device 12 picks up an image of the subject 14 formed by the focus extending optical system 11 by performing photoelectric conversion for each pixel, and outputs the obtained RAW data to the image processing unit 13. The image sensor 12 is arranged such that an imaging surface on which a plurality of pixels are arranged is located within the focal range E of the depth of focus extending optical system 11. In the imaging device 12, the center of the captured image corresponds to the optical axis of the focus extending optical system 11, the horizontal direction of the image corresponds to the S (sagittal) direction, and the vertical direction of the image corresponds to the T (tangential) direction. Are arranged as follows. Since the focus of the focus extending optical system 11 is extended (enlarged) to the focus range E (see FIG. 2), the image of the subject 14 is blurred in the RAW data output from the image sensor 12.

In addition, although the thing of the performance calculated | required by the concrete EDoF imaging system 10 is used as the image pick-up element 12, below, the pixel pitch p is 1.25 micrometers. That is, the sampling frequency fs of the image sensor 12 is assumed to be fs = 1 / p = 800 lines / mm. Therefore, the Nyquist frequency Ny is 400 lines / mm.

The image processing unit 13 is composed of a DSP, a DIP, or the like, and performs various image processing on the image data output from the image pickup device 12 to convert it into a clear image 15 as a whole. Specifically, the image processing unit 13 performs restoration processing, noise reduction processing, color mixture correction processing, shading correction processing, white balance adjustment processing, synchronization processing, color matrix correction processing, YC conversion processing on RAW data, A gamma correction process and an edge enhancement process are performed in this order to generate an image 15 having a predetermined format (for example, jpeg).

As shown in FIG. 2, the focus extending optical system 11 includes, for example, a plurality of

optical lenses

11 a and 11 b and a focus extending element 16. The optical lens 11 a adjusts the light emitted as a spherical wave from the point 14 a of the subject 14 into a plane wave and makes it incident on the focus extending element 16. The optical lens 11b converges the light incident from the focus expanding element 16. The focus extending element 16 extends the focus by the

optical lenses

11a and 11b from one point to the focus range E by adjusting the balance of various aberrations such as spherical aberration and chromatic aberration by wavefront aberration. For example, the focal point expansion element 16 converts a plane wave incident from the optical lens 11a into a light beam (hereinafter referred to as an inner light beam) L1 passing through a central portion including the optical axis L0 and a light beam passing through the periphery of the light beam L1 (hereinafter referred to as an outer light beam). The wavefront is adjusted so that the distance until convergence by the optical lens 11b differs from L2. Specifically, the focus expanding optical system 11 uses the focus expanding element 16 to converge the inner light beam L1 having a small incident height to the short-distance focal point F1 and the outer light beam L2 having a large incident height to the long-distance focal point F2. Therefore, the focus extending optical system 11 expands the focal point by the

optical lenses

11a and 11b to the focal range E between the short-distance focal point F1 and the long-distance focal point F2. The RAW data output from the image sensor 12 is data that is convoluted with an image that converges at the focal point F1, an image that converges at the focal point F2, and the like. In the following, it is assumed that the image sensor 12 is disposed at the focal point F2 so that a distant view such as infinity is in the best focus.

As shown in FIG. 3A, in the RAW data 21 output from the image sensor 12, the image of the point 14a where the subject 14 is located is a blurred image according to the arrangement of the image sensor 12. At this time, as shown in FIG. 3B, the image of the point 14a is broadly distributed in the direction of the line AB of the RAW data 21 as a pixel value. For this reason, as shown in FIG. 3C, with the raw data 21, the MTF for the spatial frequency (hereinafter referred to as the frequency MTF) decreases sharply with the increase of the spatial frequency Fq, and the resolution is low.

However, when the restoration processing is performed by the image processing unit 13, the point 14a where the subject 14 is located becomes a point image without blur as in the image data 22 shown in FIG. 3D. In other words, in the restoration process, the pixel value distribution (FIG. 3B), which is broad, is sharpened so as to have a steep peak as shown in FIG. 3E. As a result, as shown in FIG. 3F, the frequency MTF is recovered to the same level as when imaged with a normal single focus lens, and a predetermined resolution is obtained. Similarly to the image of the point 14a illustrated here, the restoration processing is performed for each of the focal point F1 and the focal point F2 convoluted in the RAW data 21 or an image converged between them according to the focal position. Since sharpening is performed, an image 15 having a deeper depth of field than an image photographed with a fixed focus lens is obtained.

Similar to the focus extending optical system 11, a general simple fixed focus lens (hereinafter referred to as a normal lens) used for a mobile phone or the like and the focus extending optical system 11 are changed in MTF with respect to defocusing of an image on the optical axis. Comparison will be made below (hereinafter referred to as on-axis defocus MTF).

The transmitted wavefront ψ of the normal lens or the focus extending optical system 11 can be expressed by an equation ψ = ΣK _j · Z _j with Zernike polynomials Z _j (j = 1 to) as terms. The Zernike polynomial Z is expressed by the following equation 1 using a distance ρ (ρ <1) from the optical axis, an angle θ from a predetermined direction (for example, the S direction), and integers m, n, and s. Can be expressed by ψ = ΣK _j · Z _j using the coefficient K _j . For example, Z ₄ (n = 2, m = 0) is the defocus of the wavefront, Z ₅ (n = 2, m = 2) is the astigmatism, Z ₇ (n = 3, m = 1) and Z ₈ (n = 3, m = −1) are coma aberrations in the S and T directions, respectively, Z ₉ (n = 4, m = 0), Z ₁₆ (n = 6, m = 0), Z ₂₅ ( n = 8, m = 0) indicates spherical aberration. Among such Zernike polynomials Z _j , the twelfth term Z ₁₂ (n = 4, m = 2) is Z ₁₂ = (4ρ ⁴ −3ρ ² ) cos (2θ), and the direct correspondence with Seidel aberration is This is one mode of wavefront aberration that causes asymmetry in the S and T directions.

Further, the coefficient K ₄ representing the defocus amount and the focus amount d ₁ (mm) on the image plane have the relationship of the following formula 2, and the subject defocus amount (hereinafter referred to as the subject defocus amount) d ₂ is: It is represented by the formula of Formula 3. Here, D is the aperture (mm), f is the focal length (mm), and the wavelength λ (nm).

As described above, since the defocus amounts d ₁ and d ₂ and the coefficient K ₄ are in a fixed relationship, the coefficient K ₄ can be used as a defocus index. In addition, the graph of each MTF shown below is MTF in the spatial frequency of 100 lines / mm. This is a spatial frequency according to the thickness of the character when the character “film” is imaged under the condition of K ₄ = −0.675 described later. In the graph of each MTF, for the sake of simplicity, all coefficients except K ₉ = −0.18, K ₁₆ = −0.057, K ₂₅ = −0.0156, and other K ₄ and K ₁₂ are “0”. MTF. In other words, this is an example of a spherical aberration adjusting type focus expanding optical system in which the wavefront is adjusted by spherical aberration (K ₉ , K ₁₆ , K ₂₅ ).

As shown in FIG. 4, when the on-axis defocus MTF is compared between the normal lens and the focus expansion optical system 11, the normal lens has an MTF value higher than that of the focus expansion optical system 11 at the best focus (K ₄ = 0). Although it is large, the MTF is decreased more than the focus extending optical system 11 due to the defocusing. This indicates that the subject is slightly blurred even when the subject is slightly deviated from the best focus. On the other hand, since the far-field MTF and the near-field MTF are in a trade-off relationship, in the case of the focus extending optical system 11, the best-focus MTF is smaller than that of the normal lens. However, the focus expanding optical system 11 has a moderate decrease in MTF even when deviating from the best focus, and an MTF value above a certain level can be obtained to such an extent that the image can be sharpened by the restoration process.

Note that the best focus MTF represents the resolution when capturing a distant view such as a landscape that can be regarded as infinity, and an image picked up with defocus (for example, about K ₄ = 0 to −1) is 0 from the best focus. ~ -Represents the resolution of a foreground image that is defocused by several hundred mm. For example, K ₄ = −0.657 corresponds to a subject distance of about 520 mm. A subject distance of around 500 mm is a subject distance that is highly likely to be frequently used in capturing a foreground image of a person, a bus timetable, a memo written on a memo paper, or the like.

In the case of a normal lens, the on-axis defocus MTF at K ₄ = −0.657 is about 0.1 (10%), and it can be considered that the image is not resolved. Therefore, the normal lens has a low resolution for foreground imaging, and it is difficult to read characters, two-dimensional codes, and the like from the foreground image. On the other hand, the on-axis defocus MTF at K ₄ = −0.657 of the focus extending optical system 11 is improved to about 0.2 (20%). In the case of the focus extending optical system 11, the resolution is further improved by performing restoration processing on the RAW data 21. Therefore, when the focus extending optical system 11 is used, a subject with a subject distance of about 500 mm has a higher resolution than that of a normal lens, and a foreground image that can read characters, a two-dimensional code, and the like can be obtained.

In addition, the on-axis defocus MTF of the normal lens has a curved shape symmetrical to the best focus regardless of the defocus direction (positive or negative) with respect to the best focus. On the other hand, the focus extending optical system 11 is designed to obtain a resolution higher than a predetermined value when the focus position is shifted in the negative direction in order to improve the MTF of the near view. The upper defocus MTF has a curve shape that is asymmetric with respect to the best focus. Further, the axial defocus MTF shown here has almost no directionality, and the axial defocus MTF is almost the same in the S direction and the T direction in both the case of the focus extending optical system 11 and the case of a normal lens.

Since the focus expansion optical system 11 has the MTF characteristics as described above, the EDoF imaging system 10 can capture an image with a deep depth of field. However, as described below, the focus expansion optical system 11 There are also problems peculiar to. Specifically, in the case of the focus extending optical system 11, the on-axis defocus MTF is substantially the same in the S direction and the T direction as described above, but the MTF by defocus at a position deviating from the optical axis L0. Is measured (hereinafter referred to as off-axis defocus MTF), a difference occurs between the S direction and the T direction of the focus extending optical system 11.

As shown in FIG. 5, the off-axis defocus MTF at a medium image height (hereinafter referred to as an intermediate image height) has a gap between the S direction and the T direction. At the best focus (K ₄ = 0), the MTF in the T direction is larger than the MTF in the S direction, and at K ₄ = −0.675, this relationship is reversed, and the MTF in the S direction is greater than the MTF in the T direction. Also grows. Further, as shown in FIG. 6, the off-axis defocus MTF at the image height that is the periphery of the image 15 (hereinafter referred to as the peripheral image height) makes the gap between the S direction and the T direction more obvious, and in particular, K ₄ = − At 0.675, the MTF in the T direction becomes almost zero.

Such a gap between the off-axis defocus MTFs in the S direction and the T direction causes variations in resolution depending on the image height in the RAW data 21 and the image 15 after the restoration process. For example, FIG. 7A shows RAW data 21 obtained by imaging the character “film” at a subject distance corresponding to K ₄ = −0.675 in a portion substantially on the optical axis. That is, the RAW data 21 in FIG. 7A is a foreground image captured by defocusing near the center of the focus extending optical system 11. For this reason, it is uniformly blurred in the S direction (left-right direction) and the T direction (up-down direction) according to the amount of defocus (K ₄ = −0.675). This is because the on-axis defocus MTF is substantially the same in the S direction and the T direction when K ₄ = −0.675 (see FIG. 4).

FIG. 7B is RAW data obtained by imaging the character “film” at the subject distance corresponding to K ₄ = 0 in the central portion corresponding to the optical axis of the focus extending optical system 11. That is, the RAW data 21 in FIG. 7B is a distant view image captured with the best focus in the vicinity of the optical axis. For this reason, the resolution is constant in both the S direction and the T direction. This is because the on-axis defocus MTF is substantially the same in the S direction and the T direction when K ₄ = 0. As can be seen from FIGS. 7A and 7B, when the image is taken by the focus extending optical system 11, the image is blurred depending on the amount of defocus if it is the central portion of the focus extending optical system 11 on the optical axis. The way the image is blurred is uniform in all directions, and it can be said that the image is naturally blurred.

FIG. 7C shows RAW data 21 obtained by imaging the character “film” at a subject distance corresponding to K ₄ = −0.675 at the intermediate image height. That is, the RAW data 21 in FIG. 7C is a foreground image obtained by defocusing and capturing at the intermediate image height. Accordingly, the image is blurred in the S direction and the T direction according to the amount of defocus (K ₄ = −0.675) as in the case of imaging on the optical axis, but the left and right directions corresponding to the S direction The vertical resolution corresponding to the T direction is slightly lower, and the vertical line portion of the “film” character is slightly noticeable. This is because in the off-axis defocus MTF at the intermediate image height, an MTF with K ₄ = −0.675 has a gap in the S direction and the T direction, and the MTF in the S direction is slightly higher than the MTF in the T direction. It is because there is.

FIG. 7D shows RAW data obtained by imaging the character “film” at the subject distance corresponding to K ₄ = 0 at the intermediate image height. That is, the RAW data 21 in FIG. 7D is a distant view image obtained by capturing the best focus at the intermediate image height. In FIG. 7D, since the image is the best focus, the resolution of the image is high and not so noticeable, but the resolution is different between the S direction and the T direction. Specifically, the resolution in the vertical direction (T direction) is higher than that in the horizontal direction (S direction). This is due to a gap between the S direction and the T direction of the off-axis defocus MTF at the intermediate image height.

FIG. 7E shows the RAW data 21 of the foreground image obtained by imaging the character “film” at the subject distance corresponding to K ₄ = −0.675 at the peripheral image height. FIG. 7F shows RAW data 21 of a foreground image obtained by imaging the character “film” at a subject distance corresponding to K ₄ = 0 at the peripheral image height. In these cases, the non-uniformity in the blur direction between the S direction and the T direction is more pronounced because the gap between the S direction and the T direction of the off-axis defocus MTF at the peripheral image height becomes obvious than in the case of the intermediate image height. stand out.

As described above, in the RAW data 21 obtained by imaging with the focus extending optical system 11, the image blur may be non-uniform depending on the image height. This tendency is the same even in the image 15 after the restoration process. In particular, the resolution of an image taken with a general camera or human vision is lower in the periphery of the image, so the resolution is improved in one direction due to the non-uniformity of blurring between the S direction and the T direction at the peripheral image height. If it looks like this, it will be an image that feels unnatural.

Such inhomogeneity of the blur (resolution) according to the image height can be reduced by adjusting the magnitude of the coefficient K ₁₂ of the Zernike polynomial 12th term Z ₁₂ . Therefore, in the focus extending optical system 11, the surface shapes of the

lenses

11a and 11b and the focus extending element 16 are determined so that the absolute value | K ₁₂ | of the coefficient K ₁₂ satisfies the conditions described below.

FIG. 8 is a graph showing the change in MTF with respect to the coefficient K ₁₂ when K ₄ = −0.675. Comparing this with the on-axis defocus MTF and off-axis defocus MTF shown in FIGS. 4 to 6, the MTF in the case of imaging with defocusing to K ₄ = −0.675 on the optical axis (FIG. 4). K ₄ = −0.675) corresponds to an MTF with Z ₁₂ = 0. The MTF (K ₄ = −0.675 in FIG. 5) when the image is defocused to K ₄ = −0.675 at the intermediate image height is set to an MTF with Z ₁₂ = −0.10, and the peripheral image height in MTF when imaging defocused to _{_{K 4 = -0.675 (K 4 =}} -0.675 in FIG. 6) to the MTF of _Z 12 = -0.30, corresponding. Further, as can be seen from FIG. 8, the larger the absolute value | K ₁₂ | of the coefficient K ₁₂ is, the larger the gap between the MTFs in the S direction and the T direction is. For this reason, if the absolute value | K ₁₂ | is small, the gap between the MTFs in the S direction and the T direction is relaxed, and blurring in the image and nonuniformity in resolution are alleviated.

MTF and variation in relative coefficient K ₁₂ shown in FIG. 8, FIG. 7A, FIG. 7C, the defocused at each image height shown in FIG. 7E considering the correspondence between the image obtained by imaging, in the Z _{12 = 0} Assuming that the MTF of the image is a reference value, if the gap between the MTF in the S direction and the T direction is more than twice the reference value, the image will be unnatural due to the difference in resolution between the S direction and the T direction. . Therefore, the focal expansion optical system 11, to fit in the smaller range than twice the reference value the difference between the MTF in the S direction and the T direction _(MTF of _{Z 12} = 0), the coefficient _{K 12} is defined. Specifically, in FIG. 8, | K ₁₂ | <0.3.

In addition, as shown by arrows P and Q in FIG. 8, within the range of | K ₁₂ | <0.3, there is a point where the MTF in the S direction and the MTF in the T direction are almost “0”. The phase of the MTF is reversed at P and Q. At point P, the phase of the MTF in the S direction is reversed, and at point Q, the phase in the T direction is reversed. When the MTF is “0”, the image is not transmitted, and the phase inversion causes artifacts such as two-line blurring, so that an accurate image cannot be transmitted. Therefore, the focus expansion optical system 11, the size of the coefficient _{K 12} is preferably MTF of the MTF and the T direction of the S-direction with respect to the coefficient _{K 12} is within the scope of the following point becomes zero. Specifically, in FIG. 8, a range larger than the point P (K ₁₂ = −0.275) and smaller than the point Q (K ₁₂ = + 0.275), that is, | K ₁₂ | <0.275. Is preferred.

Furthermore, even if the difference between the MTF in the S direction and the MTF in the T direction is not more than twice the reference value, the ratio of the MTF in the S direction to the MTF in the T direction is more than 1: 2 (or 2: 1). Then, the non-uniformity of the blurring method (resolution) becomes conspicuous in the S direction and the T direction. Therefore, MTF ratio of MTF and T direction S direction 1: 2 or 2: 1 smaller _ranges, it is preferable that the value of _{K 12} is defined.

The relationship between the MTF in the S direction and the MTF in the T direction is reversed at K ₁₂ = 0. Further, the magnitude relationship with respect to the coefficient K ₁₂ of the MTF in the S direction and the T direction varies depending on the actual configuration of the focus extending optical system 11. That is, in FIG. 8, greater MTF of T direction than S direction on the positive side of the coefficients K _12, although the MTF S direction is greater than the T direction at the minus side of the coefficients K _12, this relationship is specific focus It reversed by the configuration of the extended optical system 11, greater MTF of S direction than the T direction at the positive side of the coefficients K _12, which may MTF of T direction than S direction in the minus side of the factor K ₁₂ is increased . Therefore, the above conditions, the coefficient _{K 12} of MTF of the value of the coefficient _{K 12} of MTF of S direction is twice the MTF of the T direction first threshold value Th1, T direction is 2 times the MTF of S direction the value when the second threshold value Th2, the condition that the coefficient _{K 12} is a second threshold value Th2 is smaller than greater than the first threshold _{Th1 (Th1 <K 12 <Th2} ), or, the coefficient _{K 12} is smaller than the first threshold value The condition is greater than the second threshold (Th2 <K ₁₂ <Th1). Specifically, in FIG. 8, | K ₁₂ | <0.08.

Since the surface shape of the

optical lenses

11a and 11b and the focus expanding element 16 is determined so as to satisfy the above-described conditions, the focus expanding optical system 11 is blurred in the image 15 captured by the EDoF imaging system 10. And the uniformity of resolution is improved.

Note that the range of the image height that satisfies the above-mentioned conditions is the range in which the image blur and the resolution are uniform. Therefore, in the focus expansion optical system 11, it is preferable that all image heights are satisfied.

Note that each of the above-described conditions has been described by taking an example where K ₄ = −0.675 is defocused, but other coefficients K ₄ have other values such as the best focus of K ₄ = 0. Is the same. Also, in the above conditions, for simplicity, all coefficients except K ₉ = −0.18, K ₁₆ = −0.057, K ₂₅ = −0.0156, and other K ₄ and K ₁₂ are set to 0. Although the MTF of the spherical aberration adjusting type focus expanding optical system has been described as an example, the present invention is not limited to this. Each coefficient K _j other than the coefficient K ₁₂ may be arbitrarily determined according to the specific performance required for the actual focus extending optical system 11.

In the above embodiment, the paragraph 12 Z ₁₂ of Zernike polynomials Z, has been described a relationship between the off-axis defocusing MTF, in Zernike polynomials Z is paragraph 12 against off-axis defocusing MTF there are terms having substantially the same relationship as Z _12. Specifically, the thirteenth term Z ₁₃ (n = 4, m = −2) = (4ρ ⁴ −3ρ ² ) sin (2θ). Section 13 represents the wavefront mode is rotated 45 degrees wavefront mode represented by Section 12 Z _12, as shown in FIG. 9, the direction of the off-axis defocus MTF inclined 45 degrees from the S direction and S45, if Hakare for the direction T45 inclined 45 degrees from the T direction, resulting in the same effect as the paragraph 12 _{Z 12} to the image 15. Therefore, for the coefficient _{K 13} of Section 13 _{Z 13,} like the coefficient _{K 12} of paragraph 12 _{Z 12} described above, it is preferable above conditions are satisfied. However, the lens 11a, for 11b and focus expansion element 16 is rotationally symmetric with respect to an optical axis L0, a focus extended optical system 11 as the coefficient K ₁₂ when designing the focal expansion optical system 11 satisfies the above condition form it and it is equivalent to form a focal expansion optical system 11 as the coefficient K ₁₃ satisfies the above conditions. However, when the wavefront of the actually manufactured focus expanding optical system 11 is developed by superimposing the Zernike polynomial Z, the coefficient K _{12 is} influenced by the arrangement errors of the

lenses

11a and 11b and the focus expanding element 16, other aberrations, and the like. the coefficient K ₁₃ is not necessarily the same. Therefore, it is preferable to satisfy the same conditions as the coefficient _{K 13} also factor _{K 12.}

Note that the focus extending optical system 11 has an enlarged focus range E, but the focus range E is constant, and thus is a kind of fixed focus lens. Therefore, the focus extending optical system 11 and the EDoF imaging system 10 can be suitably used as an inexpensive fixed focus lens without a focus adjustment function or instead of another imaging system using a fixed focus lens. In this case, there is no resolution distribution (unnatural blur), which is a disadvantage of the focus extending optical system, and an image taken with an extended depth of field can be easily obtained as compared with the case of using a fixed focus lens. it can.

In the above-described embodiment, the example in which the

optical lenses

11a and 11b and the focus extending element 16 of the focus extending optical system 11 are fixed has been described. However, the present invention is not limited to this. By moving the

optical lenses

11a and 11b and the focus expanding element 16 of the focus extending optical system 11, it can be suitably used in an imaging system that performs autofocus. In such an imaging system that performs autofocus, the in-focus state is determined from an image with left and right (or top and bottom) parallax, but a conventional focus expansion optical system having a different blur depending on the position in the image 15 is used. I could not. However, since the defocusing optical system 11 has a substantially constant blur regardless of the position in the image 15, it can be suitably used for such an imaging system.

When the focus expansion optical system 11 is used in an imaging system that performs autofocus, the focus range E is moved along the optical axis by moving the

optical lenses

11a and 11b or the focus expansion element 16 along the optical axis. Further, like the zoom lens, the length of the focal range E may be changed by changing the relative positions of the

optical lenses

11a and 11b. In these cases,

optical lenses

11a, 11b, or after the movement of the focal expansion element 16 (or moving) also, it is devised that the conditions of the above-mentioned factor K ₁₂ (and conditions of the coefficient K ₁₃₎ is satisfied It ’s fine.

The condition of the coefficient K ₁₂ (and coefficient K ₁₃ ) described above is not a condition that uniquely defines the surface shapes of the

optical lenses

11 a and 11 b and the focus expanding element 16 of the focus expanding optical system 11. For this reason, the focus expansion optical system 11 can be formed by various combinations of the surface shapes of the

optical lenses

11a and 11b and the focus expansion element 16 while satisfying the condition of the coefficient K ₁₂ (and coefficient K ₁₃ ). Therefore, the condition of the coefficient K ₁₂ (and coefficient K ₁₃ ) described above can be satisfied by various focus expansion optical systems having different performance as the focus expansion optical system, such as the size of the focus range E.

In the above embodiment, the 12th term Z ₁₂ and the 13th term Z ₁₃ of the Zernike polynomial Z have been described. However, the 21st term Z ₂₁ (n = 6, m = 2: (15ρ ⁶ −20ρ ⁴ + 6ρ). ²⁾ cos ^(2θ)) and paragraph 12 _{Z 12,} Section _{22 Z 22 (n = 6,} m = -2: (15ρ 6 -20ρ 4 + 6ρ 2) sin (2θ)) is Section 13 _{Z 13} And wavefront modes each having similar characteristics. However, as the paragraph 12 _{Z 12} and Section 13 _{Z 13,} effect on the off-axis defocus MTF is not large. For this reason, in order to eliminate the gap between the S direction and the T direction of the on-axis defocus MTF, the coefficient K ₁₂ (and coefficient K ₁₃ ) described above is used as the coefficient of the twelfth term Z ₁₂ (13th term Z ₁₃ ). ) Should be satisfied.

In the above-described embodiment, the image processing unit 13 performs a restoration process to obtain the image 15 with an extended depth of field. By the way, depending on the MTF performance of the focus extending optical system 11, the depth of field may be sufficiently extended only by the focus extending optical system 11. In such a case, it is not always necessary to perform restoration processing. Further, depending on the MTF performance of the focus extending optical system 11, an image that seems to have a deepened depth of field can be easily obtained by performing edge enhancement processing or contrast enhancement processing instead of performing restoration processing. You can also. In these cases, since the restoration process is not performed, the image can be obtained at a high speed, which is effective in capturing a moving image. Of course, even in the EDoF imaging system 10 that performs the restoration processing, the restoration processing may not be performed during moving image imaging.

Note that the image sensor 12 may be a CCD image sensor or a CMOS image sensor. In addition, an image sensor having another structure may be used.

In the above-described embodiment, the focus extending optical system 11 converges the light beam near the center including the optical axis toward the near side (subject 14) and converges the light beam passing outside toward the back side (imaging element 12). However, the light flux near the center including the optical axis may be converged toward the back (imaging element 12), and the light flux passing outside may be converged toward the near side (subject 14).

In addition, although the example provided with the two

optical lenses

11a and 11b in the focus expansion optical system 11 was demonstrated, the number of lenses other than the focus expansion element 16 may be one, and may be three or more. The surface shape of the

optical lenses

11a and 11b is arbitrary, and may include a spherical surface, but the focus extending optical system 11 includes at least one aspherical surface. The focus extending element 16 may be composed of a plurality of optical lenses.

The focus extending optical system 11 includes substantially three lenses, the

optical lenses

11a and 11b and the focus extending element 16, but further includes a lens having substantially no power, a diaphragm, a cover glass, and the like. An optical element other than the lens, a lens flange, a lens barrel, a mechanism portion such as a camera shake correction mechanism, and the like may be included.

DESCRIPTION OF SYMBOLS 10 EDoF imaging stem 11 Focus expansion

optical system

11a, 11b Lens 12 Image sensor 13 Image processing part 14 Subject 14a Point 15 Image 16 Focus expansion element 21 RAW data

Claims

At least one optical lens for imaging light from a subject on an image sensor;
A focus expansion element that adjusts a wavefront so as to change an imaging position by the optical lens according to a distance from an optical axis, and expands a focal point of the optical lens to a focal range having a width;
The wavefront ψ after passing through the optical lens and the focus expanding element is expressed by a formula ψ = ΣK j · Z j where each term is a Zernike polynomial Zj (n, m), and the twelfth term Z 12 (n = 4, m = 2 common MTF in the sagittal direction and the tangential direction when the coefficient K 12 is 0 when the reference value of the) range smaller than twice the gap MTF of sagittal direction and the tangential direction to the reference value within the focus expansion optical system, wherein the coefficient K 12 is defined.
2. The focus extending optical system according to claim 1, wherein an absolute value | K 12 | of the coefficient K 12 satisfies | K 12 | <0.3.
The coefficient K 13 of the twelfth term Z 13 (n = 4, m = −2) is based on the MTF at K 13 = 0, and the gap between the sagittal and tangential MTF is twice the reference value. within a small range, the focus extension optical system ranging first claim of claim, wherein the coefficient K 13 is defined.
4. The focus extending optical system according to claim 3, wherein the absolute value | K 13 | of the coefficient K 13 satisfies | K 13 | <0.3.
Within the scope of the following points MTF of sagittal direction and the tangential direction with respect to the coefficients K 12 is 0, the focus extension optical system ranging first claim of claim, wherein the coefficient K 12 is defined .
6. The focus expanding optical system according to claim 5, wherein the absolute value | K 13 | of the coefficient K 13 satisfies | K 13 | <0.275.
Sagittal MTF tangential direction of the MTF twice to become the value of the first threshold value of the coefficient K 12 Th1, the tangential value of the second threshold value of the coefficient K 12 of tangential direction of the MTF is twice the sagittal MTF when a Th2, the coefficient K 12 is Th1 <K 12 <Th2, or Th2 <focal extension of claims paragraph 1, wherein the a value within range satisfying the K 12 <Th1 Optical system.
The focus expanding optical system according to claim 7, wherein the absolute value | K 13 | of the coefficient K 13 satisfies | K 13 | <0.08.
In all image heights, the gap MTF of sagittal direction and the tangential direction within a small range than twice of the reference value, wherein the range first of claims, wherein the coefficient K 12 is defined Focus expansion optical system.
The focus extending optical system according to claim 1, wherein the optical lens and the focus extending element are fixed so as not to move in the optical axis direction.
2. The focus expanding optical system according to claim 1, wherein the optical lens or the focus expanding element is provided so as to be movable in the optical axis direction.
An image sensor for capturing an image of a subject;
Adjusting at least one optical lens for imaging light from the subject on the image sensor, and adjusting a wavefront so as to change an imaging position by the optical lens according to a distance from an optical axis; A focal extension element that extends the focal point to a focal range having a width, and a wavefront ψ after passing through the optical lens and the focal extension element is a Zernike polynomial Zj (n, m) as a term ψ = ΣK j · Z j , and when the MTF common to the sagittal direction and the tangential direction when the coefficient K 12 of the twelfth term Z 12 (n = 4, m = 2) is 0 is used as a reference value, the gap MTF of sagittal direction and the tangential direction within a small range than twice of the reference value, the focus expansion optical system in which the coefficient K 12 is defined,
An imaging system comprising:
13. The image processing apparatus according to claim 12, further comprising: an image processing unit that generates an image with an extended depth of field corresponding to the focal range by performing a restoration process on data output from the imaging device. The imaging system described.