JP2009147925A - Digital camera and digital camera system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a design of an optical low-pass filter optimal for a predetermined interpolation algorithm. <P>SOLUTION: A digital camera is provided with: a lens unit that forms an image of an object on an image picking-up surface; an image picking-up element that outputs an image signal of the object image, wherein a color filter is set so that a first color component takes at least half the density in a checker flag-like form at respective pixels arranged in a lattice form at pixel intervals (a, b) in two directions (x) and (y) and the other color components take the residual pixels; and an optical low-pass filter that carries out such an optical line separation that light passed through the lens unit is separated into two inclined directions of ((1/2)a, (1/2)b)×(√2/α) and ((1/2)a, -(1/2)b)×(√2/α) with respect to the (x, y) coordinate axes before the light is incident on the image picking-up element and that carries out a frequency modulation of the object image to eliminate a band for coupling spatial frequencies (α/(2a), 0) and (0, α/(2b)) at α times the positions of an x-direction Nyquist frequency 1/(2a) and a y-direction Nyquist frequency 1/(2b). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a digital camera and a digital camera system provided with an optical filter.

  A digital camera converts an analog image formed by a lens into a digital signal by sampling, and therefore follows the Nyquist sampling theorem. A high frequency component equal to or higher than the Nyquist frequency is recognized as a low frequency component in the first Brillouin zone, which is well known in solid state physics, and moire is generated by a so-called aliasing effect. Furthermore, in the case of a single-plate color image pickup device in which color filters are arranged, color moiré occurs due to the density difference and phase difference of the color filters.

  In general, the color moire is called a false color together with a prediction error when performing color interpolation. A false color is generated due to an error in interpolation prediction of a color difference component in color interpolation due to an aliasing phenomenon, and it is generally difficult for an interpolation algorithm to distinguish between a false color and an actual color. Further, an interpolation prediction error associated with aliasing of the luminance component is generally called false resolution or false structure. This false structure is also generally difficult to distinguish between the real structure and the interpolation algorithm.

  Fig. 1 shows a diagram of the frequency resolution area of the sampling signal in the Bayer array, that is, the first Brillouin zone, and a diagram schematically showing the occurrence of false color and false resolution aliasing near the Nyquist frequency. It is shown in 2. FIG. 1 (a) is a drawing of the first Brillouin zone of the R component in the Bayer arrangement, FIG. 1 (b) is a drawing of the first Brillouin zone of the G component in the Bayer arrangement, and FIG. FIG. 5 is a drawing of the first Brillouin zone of the B component in the Bayer arrangement. A drawing example of the first Brillouin zone of another color filter array, for example, a delta array, is shown in Patent Document 1 of the present inventor's application.

  In general, when a color filter arrangement is determined, a center point (hereinafter referred to as a pole) where the false color / false structure is generated is automatically determined in the frequency space (k space). The poles usually appear at the corners of the polygon that forms the first Brillouin zone, and in the case of a square lattice arrangement, they also appear at the midpoint of the polygonal line segment. The circular zone plate (CZP) image corresponds exactly to the figure representing the k-space resolution area, and shows an example in which the false color extreme points that appear when an achromatic CZP image is captured in the Bayer array are classified in FIG. .

  Conventionally, it is possible to suppress the occurrence of false colors by performing color interpolation processing on an imaging signal that has passed through an optical low-pass filter (OLPF) before digital imaging of high-frequency components that cause this aliasing. It has been known. For example, Patent Literature 2, Patent Literature 3, and Patent Literature 4 disclose this effect. In other words, in the case of a square lattice arrangement, if the beam is separated twice for one pixel horizontally and one pixel vertically and the four-point separation type is used, the frequency component at that extreme point is killed until the MTF is completely zero. be able to. It can be called 100% OLPF in the sense that it shifts the amount corresponding to the pitch of one pixel and creates an MTF depression in the dead frequency band at a frequency position of 100% of the Nyquist frequency. In this sense, it can be written as 100% hv for convenience. FIG. 2 also shows a schematic diagram (dotted line indicated by OLPF) in the case of 100% OLPF.

  Patent Document 5 discloses an interchangeable lens type single-lens reflex camera in which two cameras having different pixel pitches, for example, two cameras each having an image sensor of 7 μm / pixel and 5 μm / pixel, have respective pixel pitches. A description of a method for reducing the influence of aberration caused by the difference in the optical path length due to the difference in the thickness of the optical low-pass filter, as to how to realize a four-point separation type optical low-pass filter having a light separation width almost equal to is doing.

  Among these, as common sense of the method of using the optical low-pass filter, it is described that the light beam separation width needs to be set in a range substantially close to the pixel pitch in order to suppress moire [paragraphs [0007] and [0226]. And in Fig.18. In other words, as shown in Fig. 18, the beam separation width is relatively constant with respect to the Nyquist frequency regardless of how the pixel pitch changes from 8.92 μm / pixel to 4.88 μm / pixel. The idea that a certain frequency should be killed has been developed.

JP 2004-7164 A Japanese Utility Model Publication No. 47-18689 U.S. Pat.No. 4,626,897 U.S. Pat.No. 4,663,661 US Patent Application Publication No. 2005/0174467

  As mentioned above, the optical low-pass filter does not make sense if the dead frequency band is not set near the Nyquist frequency. It has been done to set. The scope of design matters of each company followed the idea. However, when the pixel pitch exceeds 5 μm / pixel and becomes smaller, such as 4 μm / pixel and 3 μm / pixel, what changes occur in the environment of single lens reflex lenses, and what image quality There was no systematic study on whether this could cause problems. In other words, how the level of false color and false resolution of the Nakist frequency changes specifically, what has emerged as the most important problem for the entire image, and what can be solved for such a problem? It was completely unknown whether it could be planned, could not be planned, or at what point the end of the increase in the number of pixels could not be planned.

  Therefore, the present invention presents a solution that can be achieved by increasing the number of pixels in the future, particularly in the current single-lens reflex system, by clarifying these and clarifying specific characteristics.

The invention of claim 1 is applied to a digital camera, and is arranged in a lattice pattern with a pixel interval (a, b) in two directions x and y orthogonal to each other, with a lens unit that forms an image of a subject on an imaging surface. In each of the pixels, the color in which the first color component occupies at least half the density in a checkered pattern among the first to nth (n ≧ 2) color components, and the other color components occupy the remaining pixels. An image sensor that outputs an image signal of a subject image with a filter, and two directions oblique to the (x, y) coordinate axis ((1/2) before the light that has passed through the lens unit enters the image sensor. a), (1/2) b) x (√2 / α) and ((1/2) a, − (1/2) b) x (√2 / α) The spatial frequency (α / () at a position α times the Nyquist frequency 1 / (2a) in the x direction and the Nyquist frequency 1 / (2b) in the y direction. an optical low-pass filter unit that performs frequency modulation of a subject image that kills the band connecting a), 0) and (0, α / (2b)), and the pixel interval (a, b) of the image sensor is 2. When in the range of 5 to 5 μm / pixel, the position of the extinction frequency band of the optical low-pass filter is set to a range of 1.5 ≦ α ≦ 3.5 times the Nyquist frequency of the image sensor. Is.
The invention according to claim 5 is applied to a digital camera, and is arranged in a grid pattern with a pixel portion (a, b) in two directions x and y orthogonal to each other, and a lens unit that forms an image of a subject on an imaging surface. In each of the pixels, the color in which the first color component occupies at least half the density in a checkered pattern among the first to nth (n ≧ 2) color components, and the other color components occupy the remaining pixels. An image sensor in which a filter is disposed and outputs an image signal of a subject image, and a light perpendicular to the (x, y) coordinate axis (0, b / α) before the light passing through the lens unit enters the image sensor ) To perform frequency modulation of a subject image that kills the band of the spatial frequency α / (2b) at a position α times the Nyquist frequency 1 / (2b) in the y direction of the image sensor. A low-pass filter section, and the pixel interval (a, b) of the image sensor is 2.5. When in the range of -5 μm / pixel, the position of the extinction frequency band of the optical low-pass filter is set to a range 1.1 ≦ α ≦ 2.0 times the Nyquist frequency of the image sensor It is.
The invention according to claim 9 is applied to a digital camera, and is arranged in a grid pattern with a pixel portion (a, b) in two directions x and y orthogonal to each other, and a lens unit that forms an image of a subject on an imaging surface. In each of the pixels, the color in which the first color component occupies at least half the density in a checkered pattern among the first to nth (n ≧ 2) color components, and the other color components occupy the remaining pixels. An image sensor that has a filter and outputs an image signal of a subject image, and before the light that has passed through the lens unit enters the image sensor, two directions (a / α) with respect to the (x, y) coordinate axis. , 0) and (0, b / α), the position of the imaging element is α times the Nyquist frequency 1 / (2a) in the x direction and the Nyquist frequency 1 / (2b) in the y direction. Each band of spatial frequency (α / (2a), 0) and (0, α / (2b)) An optical low-pass filter unit that performs frequency modulation of a subject image, and when the pixel interval (a, b) of the image sensor is in the range of 2.5 to 4 μm / pixel, the position of the dead frequency band of the optical low-pass filter is determined. In this case, the range is set to 1.45 ≦ α ≦ 2.5 times the Nyquist frequency of the image sensor.
The invention of claim 21 is applied to a digital camera system. The digital camera according to any one of claims 1, 5, and 9 and an image signal output from an image sensor of the digital camera are at least x-axis. Calculate the similarity between different colors with the resolution of the Nyquist frequency domain using color signals between different color components that exist at the minimum pixel interval in the two directions of the y-axis and the similarity based on the similarity between the different colors And an image processing unit that determines a strong direction and generates a color signal of at least one common color component for each pixel based on the similarity determination result.

  We have grasped and clarified the change of the influence on the false resolution and the false color caused by the demosaic of the color filter array with respect to the specific characteristics regarding the pixel pitch dependence of the current lens group. This enables an unprecedented method of using the optical low-pass filter, and maintains a high-quality demosaic result with less false resolution and fewer false colors by showing the optimum setting range of use. In addition, the pixel pitch can be miniaturized while continuing to provide strong and powerful photographs.

  The present invention provides an optimum optical filter design from the viewpoint of taking lens MTF characteristics into consideration. In particular, in the lens group of a single-lens reflex camera having a variable aperture value, an inevitable problem associated with the expansion of the target frequency region of the lens MTF characteristic that occurs when the pixel pitch of the image sensor is reduced to increase the number of pixels. Design an optical low-pass filter from the point of view. It also optimizes the characteristics of the optical low-pass filter based on the compatibility with the interpolation algorithm. In particular, it introduces a predetermined interpolation algorithm to design an optimal optical low-pass filter and provides an improvement in overall image quality. To do.

<Basic policy>
Before the description of the embodiments, the basic idea of the present invention will be described as an introduction. Then, the reason and the basis for the following embodiments will be described by designing the optical low-pass filter for realizing it, comparing the characteristics, and conducting the experiment result by the geometrical optical simulation and considering the wave optics. The specific algorithm for image processing will be described in detail in the embodiments described later. Here, by comparing with the conventional interpolation algorithm with higher performance, what advantages can be utilized, It will be shown how it becomes possible to cope with a problem associated with a new reduction in pixel pitch by utilizing this advantage. The color filter array will be described by taking the most typical Bayer array (square array of pixel pitch a) as an example.

  In general, the conventional interpolation algorithm that determines the direction only by using the same color correlation at the two-pixel pitch interval generates a large false color around the poles centered on the eight points in the first Brillouin zone in the k space (see FIG. 3). ). This is the same as the configuration of the extreme points of the false color generation source automatically determined from the color filter array.

  There are two types of false colors. The first one appears near the vertical line Nyquist frequency (± π / a, 0) and horizontal line Nyquist frequency (0, ± π / a) as seen in the resolution region of the achromatic color. The color moire pattern 1 is a combination of blue and yellow, and the second is the diagonal Nyquist frequency (± π / a, ± π / a), (± π / a, − + π / a) as seen in the achromatic resolution region. ) A color moire pattern 2 of a combination of green and magenta colors appearing in the vicinity.

  Further, the resolving power has a resolution only in the region surrounded by the G component rhombus in FIG. 1, and all of the resolving power is false in the high frequency band outside. In view of this, USP 6,836,572 of the present inventor introduced a technique for determining a direction by using a different color correlation at an interval of one pixel pitch ignoring a color component, and introduced an effective utilization method in combination with the same color correlation. However, although the effect is enormous, it is extremely difficult to handle, and while showing the direction of improvement, the countermeasures against harmful effects on the luminance surface have not reached a sufficient level.

  The color judgment method disclosed in US 2004/0080639 of the present inventor has substantially improved that. In this embodiment, the color gradient judgment method disclosed in US 2006/0092298 of the present inventor is further improved. Adopt combined technology. By using these techniques, even if the MTF and contrast at the Nyquist frequency remain extremely high, it is possible to realize an interpolation algorithm that suppresses false colors and false resolutions to a very low level.

  If the algorithm using this different color correlation is used, the resolution of the Nyquist frequency of the vertical and horizontal lines is provided, so that the color moire pattern 1 in FIG. 3 can be completely eliminated and the color moire pattern 2 can be reduced. Further, the false resolution outside the rhombic region in FIG. 1 can be completely eliminated from the viewpoint of accuracy of direction determination. These effects can be considered separately by using a false color correlation when generating a color difference component and by obtaining a false color suppression effect when using a different color correlation when generating a luminance component.

  Here, the term “complete” means that it is possible to eliminate an error in direction determination, which is the main cause of the occurrence. In addition, even if any countermeasure technique is used in calculating the interpolation value, the slightest prediction error resulting from the absence of a signal in the desired direction is the best countermeasure technique compared to false color / false resolution due to direction determination errors. Is omitted, because the order is small.

  In this way, if different color correlation is used when generating chrominance components and luminance components, false colors and false solutions appearing near the vertical and horizontal Nyquist frequencies (± π / a, 0) and (0, ± π / a). There is no need to worry about the image. Therefore, there is no longer a need to destroy the signal contrast of the band by the optical low-pass filter because there is no need to worry about generation of false color / color moire in the area. That is, it can be said that the environment of the color interpolation algorithm that suppresses the generation of false color and false resolution as much as possible is prepared even if the MTF value in the frequency band existing on the extreme point of the color moire generation source is finite. Therefore, it is possible to give a new degree of freedom to the design of the optical low-pass filter and to allocate all the energy of the coping ability to the new problem that appears with the miniaturization of the pixel pitch. In addition, as a method of selecting one of the best optical low-pass filters, an oblique Nyquist frequency region (± π / a, ± π / a), (± π / a, − The energy of the optical low-pass filter can be concentrated on + π / a).

  Different color correlation is also called similarity between different colors, and same color correlation is also called similarity between same colors, and both expressions are used in the text without distinction. In addition, the expression MTF (Modulation Transfer Function) is used in the same meaning as the contrast that can be transmitted by the optical system. However, strictly speaking, it is allowed to discuss using the MTF. Limited to meet. In other words, the intensity distribution image of the light calculated by further squaring the electromagnetic field that satisfies the linearity superposition principle is limited to the case where superposition by Fourier expansion is allowed again. Therefore, here we will discuss using a contrast that can be defined in a broader sense.

<Optical low-pass filter candidates and contrast characteristics>
As is apparent from the schematic representation of FIG. 2, when an optical low-pass filter is inserted, the MTF lowering action works to the middle frequency region where the original high MTF characteristic is desired to be maintained. Sharpness and resolution are reduced. Therefore, sharpness and suppression of false color are in a trade-off relationship in a single-plate image sensor, and follow the uncertainty principle of quantum mechanics with the spatial quantization operation of measurement by sampling. That is, both cannot be accurately determined simultaneously from the relationship between the contrast and the color uncertainty, and if one of them is regarded as important, the performance of the other is lowered, and there is a limit to the accuracy with which both can be achieved simultaneously.

  In the square lattice arrangement, 100% hv-type OLPF corresponds to the case where false color suppression is most important in the above trade-off relationship, and the state without OLPF corresponds to the case where contrast and sharpness are most important. The former corresponds to the characteristic of the solid line shown as “synthetic MTF” in FIG. 2, and the latter corresponds to the characteristic of the solid line shown as “optical system MTF” in FIG. Therefore, the former has a problem that the false color is hardly generated, but has a problem that it is too blurred, and the latter has a problem that the false color and the false structure frequently occur although it is very clear.

  Therefore, in practice, the dead frequency band of the optical low-pass filter must be set in such a way that it is as close as possible to the frequency band of the pole of the false color generation source (see FIG. 3) automatically determined from the color filter array. Can not be separated from 100% while maintaining the vertical / horizontal configuration. In conventional digital cameras, the most important thing is to avoid the occurrence of false colors and color moire in still image quality. For the Bayer array, the hv type has to take a value close to 100%, which is a little away from it. Even so, the limit was to use 133% hv.

1. Three types of optical low-pass filters used in the simulation As described above, it was common sense that the optical low-pass filters were used near the Nyquist frequency. However, in order to find out if there is a possible way to deal with the essential and most important problems that appear with the reduction in pixel pitch of single lens reflex lenses described at the end of the introduction, we will take the conventional idea away. The continuous intensity from the Nyquist frequency to the infinite frequency is set boldly, and whether or not they effectively act on the Nyquist frequency region with respect to the change to the pixel pitch dependence of the lens MTF is examined.

(First form: hv2 direction type)
An optical low-pass filter often uses a birefringent plate to separate one light beam into two light beams each having 50% intensity (FIG. 4). Therefore, the conventional 100% vertical / horizontal separation method uses a single birefringent plate to separate the light beam by one pixel once in the vertical direction, and further separates it into a horizontal direction using another birefringent plate. In general, light is separated by one pixel and a total of four points are separated.

  For convenience, an optical low-pass filter that uses two birefringent plates and separates four points vertically and horizontally 100% will be expressed as 100% hv. An optical low-pass filter that is separated by 4 pixels by 3/4 pixels in the vertical and horizontal directions is expressed as 133% hv from the reciprocal of the separation width using a percentage display for the Nyquist frequency in the dead frequency band.

  In the first mode, the conventional vertical / horizontal 2-direction 4-point separation type is used, and in order to investigate the pixel pitch dependency and the image processing performance dependency, the mode is gradually decreased from 100% to ∞%. A typical example of the relationship between the amount of beam separation shift and the convenience display is shown in FIG. FIG. 5 shows a list of typical examples of the convenience display of the first embodiment of the optical low-pass filter and the shift amount of light beam separation (light beam shift amount). Let us examine whether there is a solution that can achieve both suppression of color moire and maintenance of sharpness with respect to changes in pixel pitch using these optical low-pass filters. FIG. 6 is a diagram showing a corresponding k space diagram and a real space displacement formula in the case of 133% hv.

  If the pixels of the image sensor are arranged in a grid with pixel intervals (pitch) (a, b) in two directions x and y orthogonal to each other, the Nyquist frequency 1 / (2a) in the x direction of the image sensor and The respective bands of the spatial frequencies (α / (2a), 0) and (0, α / (2b)) at positions α times the Nyquist frequency 1 / (2b) in the y direction are indicated by dotted lines in FIG. It is shown. FIG. 6 shows an example in which α = 133%. However, a = b.

  In addition, separating the light beam of the optical low-pass filter as described above corresponds to an operation of killing a carrier (signal transmission capability) that transmits a signal in a frequency band corresponding to the separation width to zero. Equivalent to making a depression in the dead zone.

(Second form: dd2 direction type)
One of the candidates corresponding to the optical low-pass filter described at the end of <Basic Policy> in the previous section rotates the two birefringent plates configured in the first form as they are by 45 degrees, and gradually increases their strength from 141%. This is a method of changing to ∞%. In the case of 141%, it is possible to write 141% dd for convenience because light rays are separated in two diagonal directions. When 141% dd, the optical low pass filter's dead frequency band crosses diagonally on the diagonal Nyquist frequencies (± π / a, ± π / a), (± π / a,-+ π / a) The dead frequency band crosses the frequency axis at (± 2π / a, 0), (0, ± 2π / a), which is exactly twice the vertical and horizontal Nyquist frequencies.

  Therefore, the contrast intensity around the vertical and horizontal Nyquist is kept high. Countermeasures against false colors and false resolutions near the vertical and horizontal Nyquist frequencies are left to high-performance image processing that can sufficiently handle even if the contrast in the area remains strong.

  FIG. 7 shows a typical example of the relationship between the amount of beam separation shift and the convenience display when the intensity of the optical low-pass filter is weakened. Here, it is pointed out that the percentage display is just the reciprocal of the Euclidean distance of the light beam shift amount in one direction. FIG. 7 shows a list of typical examples of the convenience display of the optical low-pass filter according to the second embodiment and the amount of ray separation shift (light ray shift amount). Let us examine whether there is a solution that can achieve both suppression of color moire and maintenance of sharpness with respect to changes in pixel pitch using these optical low-pass filters. FIG. 8 is a diagram illustrating a corresponding k-space diagram and a real space displacement formula in the case of 141% dd.

  If the pixels of the image sensor are arranged in a grid with pixel intervals (pitch) (a, b) in two directions x and y orthogonal to each other, the Nyquist frequency 1 / (2a) in the x direction of the image sensor and The band connecting the spatial frequencies (α / (2a), 0) and (0, α / (2b)) at a position α times the Nyquist frequency 1 / (2b) in the y direction is indicated by a dotted line in FIG. Has been. FIG. 8 shows an example in which α = 141%. However, a = b.

(Third form: v1 direction type)
Another candidate corresponding to the optical low-pass filter described at the end of the <Basic Policy> in the previous section is to remove one of the two birefringent plates configured in the first form and make it only in one direction. This is a method of increasing the strength up to 100% and increasing it, and then gradually decreasing it to ∞%. At this time, there is a problem of whether to exit the vertical shift or the horizontal shift, but generally the probability that the image structure appears as a vertical line rather than a horizontal line due to the stability in the gravitational field. Since there are a little more, it is better to increase the vertical resolution and to remove the horizontal shift.

  When 100% 2 points are separated in the vertical direction, 100% v can be written for convenience. At 100% v, exactly four poles (± π / a, ± π / a) and (± π / a, − + π / a) on the diagonal Nyquist frequency band, and the vertical Nyquist frequency (0, ± π The extinction frequency band crosses at the two extreme points of / a). Therefore, the contrast intensity around the transverse Nyquist frequency (± π / a, 0) of the remaining two extreme points is kept high. Countermeasures against false color and false resolution near the horizontal Nyquist frequency are left to high-performance image processing that can sufficiently handle even if the contrast in the area remains strong.

  FIG. 9 shows a typical example of the relationship between the amount of light separation shift and the convenience display when the intensity of the optical low-pass filter is weakened. FIG. 9 shows a list of typical examples of the convenience display of the optical low-pass filter according to the third embodiment and the amount of ray separation shift (light ray shift amount). Let us examine whether there is a solution that can achieve both suppression of color moire and maintenance of sharpness with respect to changes in pixel pitch using these optical low-pass filters. FIG. 10 is a diagram illustrating a corresponding k-space diagram and a real space displacement formula in the case of 100% v.

  Assuming that the pixels of the image sensor are arranged in a grid with pixel intervals (pitch) (a, b) in two directions x and y orthogonal to each other, the Nyquist frequency 1 / (2b) in the y direction of the image sensor. On the other hand, the band of the spatial frequency α / (2b) at the position of α times is shown by a dotted line in FIG. FIG. 10 shows an example in which α = 100%. However, a = b.

  Note that the vertical Nyquist frequency is used to mean the Nyquist frequency located on the vertical axis (y-axis) in the frequency space, and can be called a horizontal Nyquist frequency in the sense that the horizontal stripes have a Nyquist structure. Similarly, the horizontal Nyquist frequency is used in the same meaning as the vertical Nyquist frequency.

2. Definition and comparison of sharpness Here, we will examine the contrast characteristics of each optical low-pass filter. The energy spectrum of the electromagnetic field is calculated by a model based on the superposition principle of the field that the plane waves of normal and extraordinary refracted waves coming out of the birefringent plate satisfy, and the sharpness is obtained by comparing the sum of the density of states. Define. Such an idea can be said to be a deduction similar to the wave optical approximation when considering a system in which the deviation from geometric optics becomes significant. The MTF characteristic function of the optical low-pass filter often shown in the conventional literature is based on a geometric optical model. The derivation method in that case is shown, for example, in Japanese Patent No. 51-14033. This is generally based on the concept of incoherence of two light beams of normal light and extraordinary light in birefringence due to polarization characteristics. However, in systems such as those currently considered, especially when the incident angle deviates from normal incidence increases, wave properties appear largely reflecting the angle dependence of wave reflection and refraction laws in birefringence, and geometric optics. Deviations from the model are significant.

光の進行波が作る電磁場は、次式によって記述する。

ただし、x^,y^は実空間内のx軸とy軸のそれぞれの方向の単位ベクトルを表す。 (For vertical and horizontal two-way type)
As shown in FIG. 5, the amount of light separation when shifted by α × 100% is expressed by the following equation. α represents the reciprocal of the separation width when the separation width corresponding to the pixel pitch is 1.

The electromagnetic field created by the traveling wave of light is described by the following equation.

Here, x ^ and y ^ represent unit vectors in the respective directions of the x axis and the y axis in real space.

  The unsteady term of the first term represents a traveling wave, and the stationary term of the second term and below represents a frequency characteristic that can be transmitted. The intensity of light actually observed by the sensor is determined by the square of the electromagnetic field, that is, the square of the absolute value of the complex conjugate product. The physical quantity corresponds to the energy spectrum (power spectrum) as a function of the frequency space (k space), and the signal intensity ratio observed in the real space corresponds to the contrast. Since the first term becomes 1 by taking the complex conjugate product, only the square of the absolute value of the second term or less remains.

An index of subjective sharpness that expresses the strength of the optical low-pass filter per unit pixel, and this power spectrum is the fundamental resolution reproduction area of the achromatic color of the Bayer array, that is, the basic lattice vector (a, 0) in real space. The first Brillouin zone | kx | ≦ π / a, | in the reciprocal lattice space spanned by reciprocal lattice vectors (2π / a, 0) and (0,2π / a) for the lattice points created by (0, a) Integrates in the region of ky | ≦ π / a and evaluates based on an objective physical definition. That is,

The sharpness index

  A graph showing this sharpness index as a function of the intensity α of the optical low-pass filter is shown in FIG. 11 together with other optical low-pass filter configurations calculated below. FIG. 11A is a graph in which the horizontal axis represents the intensity α of the optical low-pass filter and the vertical axis represents the sharpness index. FIG. 11B is a graph in which the horizontal axis represents the reciprocal of the intensity α of the optical low-pass filter and the vertical axis represents the sharpness index. The intensity α of the optical low-pass filter indicates the reciprocal of the separation width when the separation width corresponding to the pixel pitch is 1. Therefore, the reciprocal of α corresponds to a value obtained by normalizing the pixel pitch width to 1. Although the above formula shows an example in the case of the Bayer array, in general, in the case of another color filter array, the integration range is set in the first Brillouin zone of the reciprocal lattice space corresponding to the basic lattice vector. The definition of the first Brillouin zone and its derivation method are described in detail in Chapter 2 “Reciprocal lattice” of Kittel “Introduction to Solid State Physics” (6th edition).

光の進行波が作る電磁場は、

鮮鋭感の指標steepness indexは、

(In the case of diagonal two-way type)
The amount of light separation when shifted by α × 100% is expressed by the following equation as shown in FIG.

The electromagnetic field created by the traveling wave of light is

The sharpness index

光の進行波が作る電磁場は、

鮮鋭感の指標steepness indexは、

(For vertical one-way type)
The amount of light separation when shifted by α × 100% is expressed by the following equation as shown in FIG.

The electromagnetic field created by the traveling wave of light is

The sharpness index

  It has been experimentally confirmed that the sharpness index calculated in this way is in good agreement with the impression that is actually applied to the image by simulation and subjectively evaluated. However, since the sharpness index is defined in a direction closer to the real system, it may not be completely the same as in the simulation using only geometric optics, but it is expected that the magnitude relationship will be preserved. Is done. In fact, compared to the standard image for simulation, in the comparison experiment between 1/4 resolution and OLPF without the following, 1/4 resolution is higher in sharpness so that it is close to the expected range or vice versa. It is confirmed that.

  From the above experimental results, it is possible to distinguish the difference in sharpness as the image quality even if the index value actually differs by 0.01, that is, 1%. When 0.05 or 5% is different, it can be clearly recognized that sharpness, transparency, and three-dimensionality are greatly different. The higher this value is, the better. The value is 1 per unit pixel (a × a) when there is no optical low-pass filter, that is, α = ∞.

  The sharpness index of 0.25 at the conventional 100% hv satisfies the energy conservation law because the light ray entering one pixel is blurred to 4 pixels, and it is understandable that it becomes 1/4. In other words, the conventional 100% hv type optical low-pass filter has a sharpness of “10 million pixels + OLPF 100% hv” when compared to the sharpness of the pixel unit, regardless of the resolution of the number of pixels. It suggests that it is close to the sharpness of “pixel + no OLPF”.

  Since 100% v of the configuration of one birefringent plate is separated into only two pixels, the decrease starts from 0.5. Therefore, aside from the possibility of false color and false resolution, the difference in sharpness between 0.25 and 0.5 is tremendous.

  Another important conclusion is that there is almost no difference in sharpness for the same α value of α × 100% dd and α × 100% hv between two birefringent plate configurations. Therefore, the hv type used to be only in the range of 1.00 to 1.33, whereas the dd type is used at 1.41 or higher, so that the dd type can be sharper from the start of use. I can expect. FIG. 12 shows a table summarizing representative values of the sharpness index.

<Geometric optics simulation>
A simulation experiment of pixel pitch dependence was conducted. The optical system was assumed assuming an interchangeable lens group in a single-lens reflex camera. As we will touch on again in the last part of this specification, we will proceed with the discussion while identifying the problems peculiar to single-lens reflex lenses regarding the pixel pitch dependency. In order to investigate whether or not the intensity of the optical low-pass filter can be made variable, and to determine the optimum intensity at that time, it is necessary to completely eliminate the color moire and false resolution together with the demosaicing image processing. It suffices if a boundary point that can be detected is searched. This point provides a sufficient condition for the optimum optical low-pass filter strength.

  At that time, if simulation is performed with a lens that seems to have the highest MTF performance in the single-lens reflex interchangeable lens group, there is no concern that color moiré and false resolution will occur in other lenses. Therefore, this time, the point spread function (PSF) of a lens with sufficiently high resolution among the interchangeable lens groups designed for 35 × 24 mm film size or 23.4 × 16.7 mm APS-C size is used. With optical ray tracing, aberration simulations are performed for each use condition of the lens and obtained in matrix form.

  In general, interchangeable lenses for single-lens reflex cameras are often designed so that the aperture value is about F8, the geometrical optical aberration is the smallest, and the MTF is high. Therefore, the prepared lens use conditions are F8 where the geometric optical aberration performance is high with respect to the aperture value and the open end F2.8 where the geometric optical aberration performance is low.

  Also, regarding the image height, the geometric optical MTF performance is the highest at the center and worse as it goes to the periphery, so the image sensor used for SLR is 18x13.5mm in addition to the above film size and APS-C size Assuming that there is also a 4/3 type, PSFs with image heights y = 0mm, 5mm, 10mm and 15mm were prepared. Furthermore, the focus accuracy is not always perfect, and a focus shift on the optical axis also occurs depending on the subject distance. Therefore, in addition to the focus position z = 0 μm, a PSF with z = ± 100 μm is prepared. When z = ± 100 μm, when shooting a subject several meters away with a focal length of about medium telephoto, it corresponds to a defocus of about 10 cm in the front-rear direction. That is, it corresponds to the degree of blurring of the subject at the position of the shoulder or ear when the face is photographed and the nasal head is in focus.

  In the actual simulation, the optimization conditions are determined after blurring with the PSF of the pixel pitch spread width assumed for the original image (circular zone plate, natural image that can assume no aberration, or Siemens Star chart). The image is further blurred by various optical low-pass filters necessary for sampling, sampled by a color filter array, and subjected to demosaicing processing.

  When examining the image height dependency and defocus dependency, only one condition was changed for the aperture value F8, the image height 0 mm, and the focus position z = 0 μm, which are considered to have the highest MTF performance geometrically. By seeing if there is a point that can suppress the occurrence of false color and false resolution even if the optical low-pass filter is weakened by PSF, and how much that point is, the rate of change information for that variable can be obtained . This rate-of-change information is seen under the most severe conditions, so in the more loose conditions that changed at the same time as other variables, it is considered that the rate-of-change information is more blurred than the combined rate-of-change information. It can be said that it is guaranteed.

  For each of the three types of optical low-pass filters prepared in the previous section, whether or not the optical low-pass filter can be weakened as the pixel pitch becomes finer using the high-performance demosaicing algorithm shown in the following embodiment. And to what extent it can be weakened experimentally. We also examined how much the tolerance varies depending on the demosaicing algorithm.

  The trade-off between suppressing false colors and maintaining sharpness is the relationship in which the product of both variables is represented by an inequality sign, so where to set one of the variables and evaluate the other in order to reduce the error due to subjective evaluation It is important to decide. This time, regarding the false color suppression, it is examined which intensity α corresponds to the weakest optical low-pass filter that achieves almost no color moire and false resolution free state at any frequency of the achromatic circular zone plate. Thus, a final judgment is made by subjectively and objectively comparing the sharpness indicators that can be realized there.

  The objective index of sharpness is consistent with subjective judgment, and it is possible to replace the sharpness index with the magnitude relationship of the strength α of the optical low-pass filter in the same two-sheet hv type and dd type. Shown in the discussion. Therefore, in the graph of the experimental results shown below, there is no problem even if the vertical axis is replaced with a sharpness index. In other words, even if scaling changes, the magnitude relationship does not collapse.

  FIG. 13 is a demosaicing algorithm using the different color correlation described in the following embodiment. The conditions for the color moire / false resolution free for the two aperture values F8 and F2.8 are dependent on the pixel pitch. FIG. In the algorithm using different color correlation, it is possible to entrust image processing with false color and false resolution measures near the poles of the vertical and horizontal Nyquist frequencies, so we compared the vertical and horizontal two-way type and diagonal two-way type optical low-pass filters. However, the conditions are shown.

  FIG. 15 shows the case of a single vertical optical low-pass filter using the same algorithm. In the case of a single-sheet type, the contrast near the extreme points of the horizontal Nyquist frequency, which is not affected by 6 μm / pixel or higher, is still too high, so color moiré and false resolution free cannot be realized. It has become.

  FIG. 14 is a graph in which it is verified whether a change due to the F value corresponding to FIG. 13 can be seen by the demosaicing algorithm in the case where the conventional same color correlation is used. FIG. 16 is a graph in which the image height dependency is examined by the algorithm of the embodiment using the different color correlation. Typically, only image heights y = 0mm and y = 10mm are shown. It is easy to imagine that y = 5mm is plotted in the middle, and y = 15mm is plotted above y = 10mm. FIG. 17 is a graph in which the defocus dependency is examined by the same algorithm.

  From these figures, the first thing that can be read is that the geometric optical MTF performance of the lens is worse on the open side as the design trend with respect to the aperture value, and in F8, the pixel pitch is small at the best focus position at the center of almost 0mm image height. It is predicted that it will not fall. However, in this regard, the conclusion must be drawn with the caution that the premise will be greatly broken if the wave optical MTF performance of the lens described in the next section is taken into consideration.

The important conclusions obtained from the geometric optical simulation are as follows.
1) In the algorithm with only the same color correlation, a false color created artificially by image processing is generated by mistake due to the low resolution of direction determination. Once such a false color is generated, this signal component is F8. The blur effect of the geometric optical lens MTF reduction between F2.8 and F2.8 cannot suppress false colors, and the strength of the optical low-pass filter cannot be reduced. That is, the artificially created false color cannot be erased unless the frequency component is killed from the beginning. Therefore, if the image processing capability is low, it is not possible to benefit from the possibility of substituting the optical low-pass filter due to the decrease in the lens MTF when the pixel pitch is reduced. Therefore, there is a high possibility that the lens performance will continue to be directly affected by the reduction in pixel size.

  2) In the algorithm using different color correlation, the diagonal two-direction optical low-pass filter is more sharp and pixel pitch dependent on the condition that the same color moiré and false resolution are achieved than the vertical and horizontal two-way type. It can always be realized regardless.

  3) MTF reduction due to geometrical optical aberrations should begin to have a dramatic impact below 5-6μm / pixel as a factor that affects the ability to separate false and real colors in demosaicing algorithms using different color correlations. , Rising for each optical condition.

  4) Under actual conditions of use of the lens, the blur due to the geometrical aberration F value, the peripheral blur due to the image height, and the defocus due to the defocus are averaged, and those below 5-6μm / pixel The effects of are likely to appear.

  Looking at 4) in more detail, the effect of each blur is affected by the blur at 5-6 μm / pixel, 4-6 μm / pixel, and 6-8 μm / pixel in the order of F value, image height, and defocus. Although it becomes easy, the point where the blur countermeasure becomes possible differs depending on the configuration of the optical LPF. In other words, exceeding the value of α = 100 to 133% in the vertical and horizontal directions, which is the conventional common sense range, and taking into account the influence of wave optics shown below, the measures in the inexperienced area are at least in the diagonal two-way type. This is possible while suppressing color moire at 5 μm / pixel or less, at least 5 μm / pixel or less for the vertical one-way type, and 4 μm / pixel or less for the vertical / horizontal two-way type. This is shown in the embodiment in combination with these drawings and FIGS. 24 and 25 described later.

<Consideration of wave optics>
The simulation in the previous section only considered MTF reduction due to the geometrical optical aberration of the lens, but when the pixel pitch approaches the same order as the wavelength of the incident light, the influence of wave optical diffraction is ignored. I can't. These physical explanations are explained in Section 58 “Limitations of Geometrical Optics” in Chapter 7 “Light Propagation” in Volume 2 “Classical Field Theory” in the Landau-Lifschitz Theory of Physics Theory. It has been shown that the sharpness limit of the image depends on the wavelength and the opening angle of the incident light beam, ie the F value.

  This is an ideal imaging optical system with a wave-optically non-aberration circular aperture, and how much MTF when a fringe pattern from 0 / mm to 300 / mm is imaged for each F value It is shown in the document "Optical Technology Contact Magazine Vol.41, No.9 (2003), pp.3-12." The graph is cited in FIG. FIG. 18 is a diagram showing a wave optical MTF in an aberration-free lens.

  250 lines / mm means that 250 lines of 1 mm white and black lines are resolved on the imaging surface, so the pixel pitch of the image sensor where the Nyquist frequency exists is 1 mm / 500 pixel = 2 μm / pixel It becomes. FIG. 19 is a table showing the correspondence between the pixel pitch and typical values of the Nyquist frequency.

  What is actually observed through the image sensor is the number of electrons excited to the excitation level by photoelectric conversion of the light energy (number of photons). MTF is defined in a dimension that matches the intensity of this light, that is, the contrast of the observed signal gradation value. FIG. 20A illustrates the MTF of FIG. 18 as a function of contrast relating to the pixel pitch dependency with respect to the aperture values F8 and F2.8. In addition, a graph displayed by the reciprocal of the contrast is also shown to facilitate comparison with the diagrams shown in FIGS. 13 to 17 (FIG. 20B).

  When the contrast is reduced to 1/100, the image structure of the stripe pattern that oscillates between 0 and 100 is imaged only to the stripe pattern that oscillates between 50 and 51. It's okay.

  It can be read from this graph that the contrast of F8, whose geometric optical aberration characteristics are extremely well designed, decreases sharply as the pixel pitch decreases, and the contrast decreases due to geometric optical aberration, F8 and F2. The relationship is reversed compared to the relationship of 8 (see FIG. 20A and FIG. 22A described later). The contrast between F8 and F2.8 falls in the same way as it continues asymptotically in the section of 8-5 μm / pixel, but the contrast of F8 drops rapidly and does not resolve below 5 μm / pixel. . Even if only the contrast value is seen, it can be said that F8 is less decomposed at 2μm / pixel or less. On the other hand, with F2.8, sufficient contrast remains from the viewpoint of wave optics until it reaches 1 μm / pixel.

  For comparison, let us examine how much the Nyquist frequency contrast is reduced by the point spread function used in the geometric optics simulation in the previous section. Experimentally, the signal contact width of the circular zone plate chart before entering the optical low-pass filter created without going through the lens optical system is the image after filtering with the point spread function of the lens optical system. It can be easily estimated by measuring how much the signal amplitude has decreased. The state of observing the signal fluctuation width is shown as an example in the case of 6 μm / pixel in FIG. 21A and in the case of 2 μm / pixel in FIG. FIG. 22 shows a plot of these measurement results with respect to pixel pitch dependency. Error bars correspond to cases where the reading error is large.

  It can be read from these graphs that the contrast decrease related to the wave-optical pixel dependency of F8 is more rapidly reduced than the contrast decrease related to the geometrical pixel dependency of F2.8. Therefore, the results for F8 in FIGS. 13 to 17 of the geometric optical simulation show that the criteria of F8, y = 0 mm, z = 0 um are greatly deviated, and loose approximation is less than F2.8, y = 0 mm, z = 0 um. It can be considered that the range of change due to the image height effect and defocus effect shown in the graph is added on top of it.

<Estimation of synthetic system>
If a wave optical simulation including the influence of geometric aberration can be performed, the true system can be known. However, even if there is no such data at present, the dimensions of the geometrical aberration and the wave-optic MTF contrast of the aberration-free system are aligned. You can think of it as the product of the contrast.

  Unless an interference term due to phase appears when considering geometric optical aberrations in terms of wave optics, the first approximation of this simple product is correct and can withstand discussions that grasp the overall trend. FIG. 20 described above is a diagram showing the contrast of a wave-optic MTF of an aberration-free system. FIG. 22 is a diagram showing the contrast due to geometric optical aberration. FIG. 23 is a diagram illustrating a state in which the contrast of these synthesis systems is plotted. FIG. 22B and FIG. 23B also show graphs displayed by the reciprocal of contrast as in FIG. 20B. Further, the contrast values in FIGS. 20, 22, and 23 are summarized in tables as shown in FIGS. FIG. 24 is a contrast table for F8, and FIG. 25 is a contrast table for F2.8.

  The important conclusions obtained from these facts are that, as is apparent from the graph of FIG. 23, the combined contrast of both real geometric aberration and wave optical diffraction is less than 8 μm / pixel for F2.8 than for F8. That is, it exceeds the pixel pitch. Also, 12 μm / pixel may be considered to be approximately the same.

  Therefore, in the geometric optical simulation that actually evaluated the false color and false resolution using the image, the F8 result is almost unreliable below 6 μm / pixel, and the decrease in contrast handled in F2.8 is evaluated. If so, you are dealing with high contrast under the toughest conditions that will never be exceeded. Therefore, the strength of the optical low-pass filter that is free of color moire and false resolution determined under the conditions of F2.8 image height y = 0mm and best focus position z = 0μm is sufficient for all other lens usage conditions. It can be said that.

  In this way, when the pixel pitch is 5 to 6 μm / pixel or less in the single lens reflex lens group, the MTF drops due to geometric optical aberrations on the open side of F2.8, and in F8 the MTF due to wave optical diffraction phenomenon It is predicted that the image height effect and the effect of the focus position accuracy will overlap with each other, and it will be impossible to focus anywhere.

  This is because today's 6μm / pixel cameras have become mainstream, but we continue to have a new recognition that it is difficult to focus even though the problems of false color, color moire and false resolution continue to exist. It is consistent with new issues. In other words, at 5-6um / pixel or less, the effect of optical system MTF decline starts to appear suddenly. There is no solution to the problem that the focus that has just started to occur cannot be solved only by reviewing the strength. To that end, as an environment where this can be realized, it is necessary to provide strong support on the image processing side so that false colors and false resolutions do not occur, and to give a great degree of freedom to the design of optical low-pass filters. is there.

-First embodiment-
FIG. 26 is a diagram illustrating a configuration of the camera system 100 according to the first embodiment. The camera system 100 includes a digital camera (electronic camera) 1 and a personal computer 10. The digital camera 1 includes a photographic lens 2, an optical low-pass filter 3, an image sensor 4, an image processing unit 5, a control unit 6, and the like.

  The digital camera 1 is a single-lens reflex digital camera, and the photographing lens 2 is a replaceable lens. The photographing lens 2 is attached to the digital camera 1 via a lens mount (not shown). The interchangeable lens group to which the digital camera 1 can be attached is an interchangeable lens group that can be used in common between the conventional film camera and the present digital camera. The photographic lens 2 has an aperture 7, and in this embodiment, includes an aperture mechanism capable of controlling the aperture from the maximum open aperture value (at least F2.8 or less) to F8 or more. The taking lens 2 takes at least a value of a focal length f in a range of 12 mm to 50 mm to 200 mm so that a field angle from wide angle to standard to telephoto can be taken in accordance with the size of the imaging device 4 described below. .

  The image pickup device 4 is a single-plate color image pickup device in which RGB color filters in a Bayer array are arranged, and is composed of a CCD or the like. FIG. 32 is a diagram showing a Bayer array of color filters. In the Bayer array, as shown in the figure, the G component occupies half the density in a checkered pattern, and the other R component and B component are arranged at a density that evenly distributes the remaining pixels. The size of the image sensor 4 is any one of the above-described 35 mm × 24 mm, 23.4 mm × 16.7 mm, and 18 mm × 13.5 mm, or any size therebetween.

  The image processing unit 5 outputs various types of image processing such as white balance adjustment and compression processing on the RGB color system image data output from the image sensor 4, subjected to various analog processes, and subjected to A / D conversion. I do. The G component of the RGB color system is also a color component responsible for luminance. The image processing unit 5 is composed of an ASIC. The control unit 6 controls the entire digital camera 1, outputs the image data after the image processing by the image processing unit 5 to an external device, and records it on a recording medium such as a memory card 20.

  The personal computer 10 inputs image data captured by the digital camera 1 and subjected to certain image processing via the cable 30 or the memory card 20. Note that wireless communication may be used instead of the cable 30. The personal computer 10 performs interpolation processing described later on the input image data to generate color image data. Accordingly, the image data output from the digital camera 1 is data in which each pixel has only RGB color component data in the RGB color system. The image data output from the digital camera 1 is still image data. However, even in the case of a moving image, processing similar to that for a still image may be performed for each frame of the moving image.

<Optical low-pass filter configuration>
In the first embodiment, among the three types of optical low-pass filters first listed as candidates, a two-way configuration type having symmetry is adopted as an oblique two-direction type. As already examined in the section on geometric optical simulation, the color moiré pattern is weaker in either the vertical / horizontal two-way type or the diagonal two-way type optical low-pass filter under the lens use conditions shown in FIGS. 13, 16, and 17.・ As for the result of seeing whether it is possible to make false resolution free, the oblique type was more advantageous in all conditions. Therefore, in the first embodiment, an example of the intensity to be taken in the oblique two-way type with respect to the pixel pitch dependency is shown.

  If an interpolation algorithm using a different color correlation described below is used, sufficient conditions for the optical low-pass filter intensity that requires the simulation of F2.8 in FIG. 13 are given, and the optimum configuration is as shown in FIG.

  If the optical low-pass filter of the first embodiment is arranged in a grid pattern with pixel intervals (pitch) (a, b) in two directions x and y orthogonal to each other, , y) Two directions ((1/2) a, (1/2) b) × (√2 / α) and ((1/2) a,-(1/2) b) × The spatial frequency characteristics of the incident light are modulated by separating light into (√2 / α).

FIG. 27A shows that α is set to α = √2 = 1.41 when the pixel pitch is about 6 μm / pixel or more, and α ≧ 1.5 when the pixel pitch is about 5 μm / pixel. It is shown that α ≧ 1.90 is set when is about 4 μm / pixel, and α ≧ 2√2 = 2.83 is set when the pixel pitch is about 3 μm / pixel. This means that α ≧ √2 is set when the pixel pitch in the two directions x and y is 6 μm / pixel or less. As the pixel pitch becomes smaller than 6 μm / pixel, the value of α is monotonically adjusted. Increasing. In other words, α is a function inversely proportional to the pixel pitch, and gradually approaches α = ∞ at a predetermined pixel pitch (in this case 3 μm / pixel) or less, and α = α at a predetermined pixel pitch (in this case 6 μm / pixel) or higher. It can be said that it is a function having a characteristic asymptotic to ₀ (const).

  In FIG. 27A, the lower limit value is defined by the sufficient condition of the OLPF configuration in which color moire and false resolution are free. However, the optimum setting range for α is to set the lower limit given by 1 μm / pixel smaller than the corresponding pixel pitch as the upper limit. A feeling of recovery can be realized. The optimum relationship between a and α in consideration of these matters is shown in the graph of FIG. The hatched range in the graph of FIG. 27B is the optimum range of a and α when 2.5 μm ≦ a ≦ 5.0 μm.

  The graph (black dots) on the left side of FIG. 27B is a plot of the black dots having the values of α of 3 μm, 4 μm, 5 μm, and 6 μm in FIG. The black spot of 2.5 μm is not necessary for the low-pass filter in the range smaller than 2.5 μm, and is in a level sufficiently close to the state without the low-pass filter in FIG. 11 in the range exceeding 350%. %. The straight line above 2 μm is an asymptotic line that becomes ∞ at 2 μm. The graph on the right side (x point) in FIG. 27B is a graph obtained by shifting the left graph to the right by 1 μm.

  As described above, when both the pixel intervals (a, b) of the image sensor 4 are in the range of 2.5 to 5 μm, the position of the kill frequency band of the optical low-pass filter is set to 1.5 with respect to the Nyquist frequency of the image sensor 4. It is shown that it is optimal to set it in a range of ≦ α ≦ 3.5 times.

  In addition, when the aperture control from the maximum open aperture value that can be taken by the taking lens 2 to F8 or more is possible, the pixel interval value of the image sensor 4 corresponds to about 2 to 4 μm / pixel and 6 μm / pixel or more. In this case, it is preferable to use an optical low-pass filter having the above characteristics. Further, it is preferable not to use the optical low-pass filter when the pixel interval value is 2 μm / pixel or less. That is, the optical low-pass filter is selectively provided in the digital camera 1 and is not provided when the pixel interval value is 2 μm / pixel or less.

  The knowledge about the image quality obtained from the geometric optical simulation is that aberrations such as axial chromatic aberration are extremely conspicuous even at an image height y = 0 mm and best focus z = 0 μm at 1 μm / pixel. The stronger the optical low-pass filter is inserted, the more the aberration component spreads over a plurality of pixels, causing only a harmful effect of severely amplifying purple fringes, purple ghosts, or ghosts of the lens itself. Therefore, it is absolutely necessary to eliminate the optical low-pass filter at a fine pixel pitch from the viewpoint of geometric optical aberration.

  In addition, as a precondition for providing a degree of freedom in setting the optical low-pass filter, the color filter array of the image sensor is at least spatially responsible for the luminance component (G component) as in the Bayer array. It should be pointed out that the case of symmetrical arrangement is kept in mind.

<Interpolation algorithm>
As described above, since the image sensor 4 is a single-plate image sensor having a Bayer array color filter, the output of each pixel output from the image sensor 4 has only one color component color information. Accordingly, in order to have each color component of R, G, B corresponding to each pixel, the following interpolation processing is performed. This interpolation process is performed by the personal computer 10.

[1] CrCb surface generation Vertical / horizontal direction judgment 1
The following calculation is performed for the R and B positions.
1) Calculation of similarity
a) Similarity between different colors
GR (GB) similarity component
Cv0 [i, j] = (| G [i, j-1] -Z [i, j] | + | G [i, j + 1] -Z [i, j] |) / 2
Ch0 [i, j] = (| G [i-1, j] -Z [i, j] | + | G [i + 1, j] -Z [i, j] |) / 2
b) Peripheral Addition of Similarity This process is for improving the accuracy of similarity by considering continuity with surrounding pixels, and may be omitted when priority is given to simplicity.
Cv [i, j] = (4 * Cv0 [i, j]
+ 2 * (Cv0 [i-1, j-1] + Cv0 [i + 1, j-1] + Cv0 [i-1, j + 1] + Cv0 [i + 1, j + 1])
+ Cv0 [i, j-2] + Cv0 [i, j + 2] + Cv0 [i-2, j] + Cv0 [i + 2, j]) / 16
Ch [i, j] = (4 * Ch0 [i, j]
+ 2 * (Ch0 [i-1, j-1] + Ch0 [i + 1, j-1] + Ch0 [i-1, j + 1] + Ch0 [i + 1, j + 1])
+ Ch0 [i, j-2] + Ch0 [i, j + 2] + Ch0 [i-2, j] + Ch0 [i + 2, j]) / 16

2) Similarity judgment
if | Cv [i, j] -Ch [i, j] | = <Th0 HVd [i, j] = 0 Vertical / horizontal similarity unknown
else if Cv [i, j] <Ch [i, j] HVd [i, j] = 1 Vertical similarity
else HVd [i, j] = -1 Horizontal similarity The threshold Th0 takes a value of around 10 for 256 gradations, and is set higher when the image is noisy.

2. Color difference generation 1) R position Cr surface generation
if [i, j] is a R site in a Bayer plane {
if HVd [i, j] = 1 Cr [i, j] = R [i, j]
-{(G [i, j-1] + G [i, j + 1]) / 2
+ (2 * R [i, j] -R [i, j-2] -R [i, j + 2]) / 4}
else if HVd [i, j] =-1 Cr [i, j] = R [i, j]
-{(G [i-1, j] + G [i + 1, j]) / 2
+ (2 * R [i, j] -R [i-2, j] -R [i + 2, j]) / 4}
else Cr [i, j] = R [i, j]
-{(G [i, j-1] + G [i, j + 1] + G [i-1, j] + G [i + 1, j]) / 4
+ (4 * R [i, j] -R [i, j-2] -R [i, j + 2] -R [i-2, j] -R [i + 2, j]) / 8}
}

Or it calculates with the following formula | equation.
if [i, j] is a R site in a Bayer plane {
if HVd [i, j] = 1 Cr [i, j] = R [i, j]
-{(G [i, j-1] + G [i, j + 1]) / 2
+ (2 * G [i-1, j] -G [i-1, j-2] -G [i-1, j + 2]
+ 2 * G [i + 1, j] -G [i + 1, j-2] -G [i + 1, j + 2]) / 8}
else if HVd [i, j] =-1 Cr [i, j] = R [i, j]
-{(G [i-1, j] + G [i + 1, j]) / 2
+ (2 * G [i, j-1] -G [i-2, j-1] -G [i + 2, j-1]
+ 2 * G [i, j + 1] -G [i-2, j + 1] -G [i + 2, j + 1]) / 8}
else Cr [i, j] = R [i, j]
-{(G [i, j-1] + G [i, j + 1] + G [i-1, j] + G [i + 1, j]) / 4
+ (2 * G [i-1, j] -G [i-1, j-2] -G [i-1, j + 2]
+ 2 * G [i + 1, j] -G [i + 1, j-2] -G [i + 1, j + 2]
+ 2 * G [i, j-1] -G [i-2, j-1] -G [i + 2, j-1]
+ 2 * G [i, j + 1] -G [i-2, j + 1] -G [i + 2, j + 1]) / 16}
}
Here, the reason why the direction index HVd [i, j], in which the similarity is determined only by the similarity between different colors, is used because the generation of the false color near the Nyquist frequency can be significantly suppressed.

2) Cr surface interpolation
B sites
Cr [i, j] = (Cr [i-1, j-1] + Cr [i-1, j + 1] + Cr [i + 1, j-1] + Cr [i + 1, j + 1 ])/Four
G sites (same lines with R rows)
Cr [i, j] = (Cr [i-1, j] + Cr [i + 1, j]) / 2
G sites (same lines with B rows)
Cr [i, j] = (Cr [i, j-1] + Cr [i, j + 1]) / 2

  The same applies to the Cb surface.

3. Temporary color difference correction 1
2. There are plenty of false colors on the color difference surface obtained in step 1, which causes color moire and color spot noise at high ISO sensitivity. These should be adaptively removed in distinction from the color structure, but provisional removal is performed in advance so that the false color boundary is not misidentified as a color boundary when the color gradient described in the next section is distinguished as an index. . Two methods are introduced below, but not limited to this method.
Method 1 (low-pass processing)
An example of a 5 × 5 separable filter is shown.
Horizontal lowpass filtering
tmp_Cr [i, j] = {70 * Cr [i, j] + 56 * (Cr [i-1, j] + Cr [i + 1, j])
+ 28 * (Cr [i-2, j] + Cr [i + 2, j]) + 8 * (Cr [i-3, j] + Cr [i + 3, j])
+ (Cr [i-4, j] + Cr [i + 4, j])} / 256
Vertical lowpass filtering
TCr1 [i, j] = {70 * tmp_Cr [i, j] + 56 * (tmp_Cr [i, j-1] + tmp_Cr [i, j + 1])
+ 28 * (tmp_Cr [i, j-2] + tmp_Cr [i, j + 2])
+ 8 * (tmp_Cr [i, j-3] + tmp_Cr [i, j + 3])
+ tmp_Cr [i, j-4] + tmp_Cr [i, j + 4]} / 256

       The same applies to TCb1 [i, j].

Method 2 (median processing)
TCr1 [i, j] = Median {Cr [i + m, j + n]}
m = 0, ± 1, ± 2, ± 3, ± 4 n = m = 0, ± 1, ± 2, ± 3, ± 4

4). Color Gradient Analysis Next, the color gradient is examined to discriminate between false colors and real colors so as not to destroy the color structure. The actual color is 3. Even if the color difference correction processing is added, the contrast is more likely to remain than the false color, so that it can be distinguished with very high accuracy due to its statistical properties. At that time, in order to protect the color structure as accurately as possible, a color index surface is created that raises the color contrast between the existing colors and lowers the color contrast between the false colors. In general, since false colors tend to appear between opposite colors, it is preferable to generate a color index between primary colors.

That is, the temporary color difference signals TCr1 [i, j] and TCb1 [i, j] obtained as described above are converted into a color index Cdiff [i, j] for evaluating the color in pixel units.
Cdiff [i, j] = (| TCr1 [i, j] | + | TCb1 [i, j] | + | TCr1 [i, j] -TCb1 [i, j] |) / 3

The color index uses all the color difference information that can be combined among the three RGB primary colors to improve the protection performance of the color structure. Expanding the color difference definition is as follows.
Cdiff = (| RG | + | GB | + | BR |) / 3

演算を高速化するために、周辺全画素との微分を調べなくても、もう少し間引いた画素間微分に減らしてもよい。 Next, a color gradient representing the degree of color change is examined in a plane in which the color contrast information is pressed into a single color index plane. At this time, the size of the differential filter for detecting the color gradient is made the same as the size of the temporary color difference correction filter so that all possible ranges of damage can be examined, thereby improving the non-destructive color structure.

In order to speed up the calculation, the differential with respect to all the surrounding pixels may not be examined, and the differential between pixels may be reduced to a little more.

  Although a 5 × 5 size filter is used together with “3. Temporary color difference correction 1” here, a 9 × 9 size filter may be used for both. As a result, more powerful false color and real color separation performance can be obtained.

5. Adaptive color difference correction 1
Whether or not to perform color difference correction processing is switched according to the magnitude of the color gradient. The original Cr [i, j] is used as it is for the color boundary portion with a large color gradient.
if Cgrad [i, j] ≦ ThG {Cr [i, j] = TCr1 [i, j], Cb [i, j] = TCb1 [i, j]}
As the threshold ThG is set to a very small value, the color spot noise and color moire generation region corresponds to the color difference correction target region regardless of the chromatic color portion and the achromatic color portion, and the color structure portion is well excluded. When the gradation is 255, the value is about 5 or less.

Thereafter, a uniform low-pass process of about 3x3 may be applied to the entire surface. For example,
Cr [i, j] = {4 * Cr [i, j]
+ 2 * (Cr [i-1, j] + Cr [i + 1, j] + Cr [i, j-1] + Cr [i, j + 1])
+ 1 * (Cr [i-1, j-1] + Cr [i + 1, j-1] + Cr [i-1, j + 1] + Cr [i + 1, j + 1])} / 16
The same applies to Cb [i, j].

  The above-mentioned “3. provisional color difference correction 1” “4. color gradient analysis” “5. adaptive color difference correction 1” will be described later in “6. provisional color difference correction 2” “7. derivation of color index” “8. It may be replaced by the processing of “determination” “9. adaptive color difference correction 2”. In addition, the processing of “6. Temporary color difference correction 2”, “7. Derivation of color index”, “8. Color determination”, and “9. Adaptive color difference correction 2” described later will be described in “3. It may be replaced by the processing of “4. Color gradient analysis” and “5. Adaptive color difference correction 1”.

6). Temporary color difference correction 2
In order to accurately perform the color evaluation, the color moire generated on the color difference surface is further removed. Although a color difference median filter is also conceivable, the object can be achieved at high speed only by using a low-pass filter as described below.
TCr2 [i, j] = {4 * Cr [i, j]
+ 2 * (Cr [i-2, j] + Cr [i + 2, j] + Cr [i, j-2] + Cr [i, j + 2])
+ 1 * (Cr [i-2, j-2] + Cr [i + 2, j-2] + Cr [i-2, j + 2] + Cr [i + 2, j + 2])} / 16
TCb2 [i, j] is calculated in the same way.

7. Derivation of Color Index The color difference component calculated in this way is a scalar quantity that can completely eliminate the structural factors of the image and can measure the actual hue of the image in units of pixels. Although the color difference between RG indicated by the Cr component and the color difference between BG indicated by the Cb component have already been obtained as color indicators, since it is a scalar amount in pixel units, the color difference between RBs can be further evaluated, which is extremely human. Color evaluation close to visual judgment can be made. Here is the derivation formula for the color index Cdiff.
Cdiff [i, j] = (| TCr2 [i, j] | + | TCb2 [i, j] | + | TCr2 [i, j] -TCb2 [i, j] |) / 3
Therefore, it is possible to obtain a very accurate color index by separating the disturbing factors associated with the actual color and structural factors that were problematic in the prior art color level and evaluating the color difference between all color components. be able to. This color index can be said to represent the degree of color.

  Here, in order to further improve the accuracy of the color index, the same process as that performed by the peripheral addition of the similarity may be performed for the color index.

8). Color Determination Next, the continuous color index is subjected to threshold determination and converted to a discrete color index BW [i, j] as a color determination result.
if Cdiff [i, j] = <Thbc BW [i, j] = 'b'
else BW [i, j] = 'c' Colored portion The threshold takes a value of about Thbc to 15 at 256 gradations.

9. Adaptive color difference correction 2
Since the Nyquist false color that occurred before correction is reduced in saturation on the color difference surface that has been temporarily corrected, the false color of the achromatic part is suppressed as much as possible based on the color judgment result, and the color structure of the chromatic part is leave.
if BW [i, j]! = 'c' {Cr [i, j] = TCr2 [i, j], Cb [i, j] = TCb2 [i, j]}

10. Derivation of the color index In this way, the color index Cdiff is derived once again using the color difference surface that is cleanly and adaptively subjected to false color removal. Thereby, the discrimination accuracy between the false color and the actual color for switching between the same color correlation and the different color correlation for generating the luminance plane is increased to the limit.
Cdiff [i, j] = (| Cr [i, j] | + | Cb [i, j] | + | Cr [i, j] -Cb [i, j] |) / 3
Here, in order to further improve the accuracy of the color index, the same process as that performed by the peripheral addition of the similarity may be performed for the color index.

11. Color Determination Next, the continuous color index is subjected to threshold determination and converted to a discrete color index BW [i, j] as a color determination result.
if Cdiff [i, j] = <Thab BW [i, j] = 'a' Achromatic part The threshold value is about Thab ~ 5 when 256 gradations.

[2] G-plane generation Vertical / horizontal direction judgment 2
The following calculation is performed for the R and B positions.
1) Calculation of similarity
a) Similarity between same colors
Cv0 [i, j] = (Cv1 + Cv2) / 2
Ch0 [i, j] = (Ch1 + Ch2) / 2
However,
GG similarity component
Cv1 = | G [i, j-1] -G [i, j + 1] |
Ch1 = | G [i-1, j] -G [i + 1, j] |
BB (RR) similarity component
Cv2 = (| Z [i-1, j-1] -Z [i-1, j + 1] | + | Z [i + 1, j-1] -Z [i + 1, j + 1] | ) / 2
Ch2 = (| Z [i-1, j-1] -Z [i + 1, j-1] | + | Z [i-1, j + 1] -Z [i + 1, j + 1] | ) / 2
b) Peripheral Addition of Similarity Similar to the similarity between different colors, this processing may be omitted when priority is given to simplicity.
Cv [i, j] = (4 * Cv0 [i, j]
+ 2 * (Cv0 [i-1, j-1] + Cv0 [i + 1, j-1] + Cv0 [i-1, j + 1] + Cv0 [i + 1, j + 1])
+ Cv0 [i, j-2] + Cv0 [i, j + 2] + Cv0 [i-2, j] + Cv0 [i + 2, j]) / 16
Ch [i, j] = (4 * Ch0 [i, j]
+ 2 * (Ch0 [i-1, j-1] + Ch0 [i + 1, j-1] + Ch0 [i-1, j + 1] + Ch0 [i + 1, j + 1])
+ Ch0 [i, j-2] + Ch0 [i, j + 2] + Ch0 [i-2, j] + Ch0 [i + 2, j]) / 16

2) Similarity judgment
if | Cv [i, j] -Ch [i, j] | = <Th1 HVs [i, j] = 0
else if Cv [i, j] <Ch [i, j] HVs [i, j] = 1 Vertical similarity
else HVs [i, j] = −1 Transverse similarity The threshold value Th1 takes the same value as Th0.

2. Direction Index Selection Based on the color determination result, the direction determination result based on the similarity between different colors and the direction determination result based on the similarity between the same colors are properly used.
if BW [i, j] = 'a' HV [i, j] = HVd [i, j]
else HV [i, j] = HVs [i, j]

3. Oblique direction judgment
The following calculation is performed for the R and B positions.
1) Calculation of similarity
a) Similarity
C45_0 [i, j] = (a1 * C45_1 + a2 * C45_2 + a3 * C45_3) / (a1 + a2 + a3)
C135_0 [i, j] = (a1 * C135_1 + a2 * C135_2 + a3 * C135_3) / (a1 + a2 + a3)
However,
BR (RB) similarity component
C45_1 = (| Z [i + 1, j-1] -Z [i, j] | + | Z [i-1, j + 1] -Z [i, j] |) / 2
C135_1 = (| Z [i-1, j-1] -Z [i, j] | + | Z [i + 1, j + 1] -Z [i, j] |) / 2
GG similarity component
C45_2 = (| G [i, j-1] -G [i-1, j] | + | G [i + 1, j] -G [i, j + 1] |) / 2
C135_2 = (| G [i, j-1] -G [i + 1, j] | + | G [i-1, j] -G [i, j + 1] |) / 2
BB (RR) similarity component
C45_3 = | Z [i + 1, j-1] -Z [i-1, j + 1] |
C135_3 = (| Z [i-1, j-1] -Z [i + 1, j + 1] |
The values of the constants a1, a2, and a3 are preferably set to a1 = a2 = a3 = 1, a1 = a2 = 2, a3 = 1, and the like.

b) Peripheral addition of similarity This process may be omitted when priority is given to simplicity, as with the similarity in the vertical and horizontal directions.
C45 [i, j] = (4 * C45_0 [i, j]
+ 2 * (C45_0 [i-1, j-1] + C45_0 [i + 1, j-1]
+ C45_0 [i-1, j + 1] + C45_0 [i + 1, j + 1])
+ C45_0 [i, j-2] + C45_0 [i, j + 2] + C45_0 [i-2, j] + C45_0 [i + 2, j]) / 16
C135 [i, j] = (4 * C135_0 [i, j]
+ 2 * (C135_0 [i-1, j-1] + C135_0 [i + 1, j-1]
+ C135_0 [i-1, j + 1] + C135_0 [i + 1, j + 1])
+ C135_0 [i, j-2] + C135_0 [i, j + 2] + C135_0 [i-2, j] + C135_0 [i + 2, j]) / 16

2) Similarity judgment
if | C45 [i, j] -C135 [i, j] | = <Th2 DN [i, j] = 0 Diagonal similarity unknown
else if C45 [i, j] <C135 [i, j] DN [i, j] = 1 Similar to 45 °
else DN [i, j] = -1 Similar to 135 ° diagonally Th2 has the same value as Th0 and Th1.

4). G plane generation
if [i, j] is not a G site in a Bayer plane {
if HV [i, j] = 0 {
if DN [i, j] = 0 G [i, j] = (Gv + Gh) / 2
else if DN [i, j] = 1 G [i, j] = (Gv45 + Gh45) / 2
else G [i, j] = (Gv135 + Gh135) / 2
}
else if HV [i, j] = 1 {
if DN [i, j] = 0 G [i, j] = Gv
else if DN [i, j] = 1 G [i, j] = Gv45
else G [i, j] = Gv135
}
else {
if DN [i, j] = 0 G [i, j] = Gh
else if DN [i, j] = 1 G [i, j] = Gh45
else G [i, j] = Gh135
}
}
here,
Gv = (G [i, j-1] + G [i, j + 1]) / 2
+ (2 * Z [i, j] -Z [i, j-2] -Z [i, j + 2]) / 8
+ (2 * G [i-1, j] -G [i-1, j-2] -G [i-1, j + 2]
+ 2 * G [i + 1, j] -G [i + 1, j-2] -G [i + 1, j + 2]) / 16
Gv45 = (G [i, j-1] + G [i, j + 1]) / 2
+ (2 * Z [i, j] -Z [i, j-2] -Z [i, j + 2]) / 8
+ (2 * Z [i-1, j + 1] -Z [i-1, j-1] -Z [i-1, j + 3]
+ 2 * Z [i + 1, j-1] -Z [i + 1, j-3] -Z [i + 1, j + 1]) / 16
Gv135 = (G [i, j-1] + G [i, j + 1]) / 2
+ (2 * Z [i, j] -Z [i, j-2] -Z [i, j + 2]) / 8
+ (2 * Z [i-1, j-1] -Z [i-1, j-3] -Z [i-1, j + 1]
+ 2 * Z [i + 1, j + 1] -Z [i + 1, j-1] -Z [i + 1, j + 3]) / 16
Gh = (G [i-1, j] + G [i + 1, j]) / 2
+ (2 * Z [i, j] -Z [i-2, j] -Z [i + 2, j]) / 8
+ (2 * G [i, j-1] -G [i-2, j-1] -G [i + 2, j-1]
+ 2 * G [i, j + 1] -G [i-2, j + 1] -G [i + 2, j + 1]) / 16
Gh45 = (G [i-1, j] + G [i + 1, j]) / 2
+ (2 * Z [i, j] -Z [i-2, j] -Z [i + 2, j]) / 8
+ (2 * Z [i + 1, j-1] -Z [i-1, j-1] -Z [i + 3, j-1]
+ 2 * Z [i-1, j + 1] -Z [i-3, j + 1] -Z [i + 1, j + 1]) / 16
Gh135 = (G [i-1, j] + G [i + 1, j]) / 2
+ (2 * Z [i, j] -Z [i-2, j] -Z [i + 2, j]) / 8
+ (2 * Z [i-1, j-1] -Z [i-3, j-1] -Z [i + 1, j-1]
+ 2 * Z [i + 1, j + 1] -Z [i-1, j + 1] -Z [i + 3, j + 1]) / 16

[3] Color image output Since Cr [i, j], Cb [i, j] and G [i, j] are obtained for each pixel, a color image is completed.
When using the RGB color system, the following color conversion is performed.
R [i, j] = Cr [i, j] + G [i, j]
B [i, j] = Cb [i, j] + G [i, j]

  The above interpolation processing algorithm is expressed in words as follows. The image signal (image data) output from the image pickup device 4 and converted into a digital signal between different colors using color signals between different color components that exist at a minimum pixel interval in at least two directions of the x axis and the y axis. The similarity is calculated, the direction of strong similarity is determined based on the similarity between different colors, and the color signal of the color component common to each pixel is generated based on the similarity determination result.

  In addition, based on the similarity determination result, each pixel receives a color difference component signal (Cr, Cb) between the color component signal (G component) responsible for luminance and the other color component signals (R component, B component). Based on the generated color difference signal, at least one index of a color index indicating the degree of tint and a color gradient index indicating the degree of color change is calculated, and generated based on at least one index The obtained color difference component signal is adaptively corrected and then output as image data.

  Further, a color index or a color gradient index is calculated based on the color difference signal obtained by virtually correcting the generated color difference signal.

  Also, a color index representing the degree of color is calculated based on the adaptively corrected color difference signal, and based on this color index, the similarity between different colors composed of color signals between different color components and the same color are calculated. Judge which similarity of the same-color similarity composed of color signals between components is more reliable, determine the direction of strong similarity based on the similarity configured based on the reliability, and this similarity Based on the sex determination result, a color component signal (G component) responsible for luminance is generated and output for each pixel.

-Second Embodiment-
In the digital camera of the second embodiment, the optical low-pass filter is different from that of the first embodiment. Other configurations are the same as those of the first embodiment, and thus the description thereof is omitted. Hereinafter, the optical low-pass filter of the second embodiment will be described.

<Optical low-pass filter configuration>
In the second embodiment, among the three types of optical low-pass filters that were first listed as candidates, a one-piece vertical one-way type that has the greatest advantage of sharpness even when asymmetric is adopted. To do. In addition, the simple configuration contributes to cost reduction. From FIG. 15 in the section of geometrical optical simulation, if the intensity that is free of color moire and false resolution in F2.8 is determined, sufficient conditions are given for the pixel pitch dependency using the interpolation algorithm of the first embodiment.

  The optimum configuration of the optical low-pass filter of the second embodiment is shown in FIG. The single-type optical low-pass filter of the second embodiment can be used only with a camera of 6 μm / pixel or less.

  In the optical low-pass filter of the second embodiment, assuming that the pixels of the image sensor are arranged in a grid pattern with pixel intervals (a, b) in two directions x and y orthogonal to each other, (x, y) The spatial frequency characteristic of the incident light is modulated by performing light beam separation in only one direction (0, b) × (1 / α) with respect to the coordinate axis.

  FIG. 28A shows that α is set to α = 1 when the pixel pitch is about 6 μm / pixel, α ≧ 1.1 when the pixel pitch is about 5 μm / pixel, and the pixel pitch is about 4 μm / pixel. In the case of pixel, α ≧ 1.3 (= 4/3) is set, and when the pixel pitch is about 3 μm / pixel, α ≧ 1.67 (= 5/3) is set. This means that α ≧ 1 is set when the pixel pitch in the two directions x and y is 6 μm / pixel or less. Further, it is preferable not to use the optical low-pass filter when the pixel interval value is 2 μm / pixel or less. That is, the optical low-pass filter is selectively provided in the digital camera 1 and is not provided when the pixel interval value is 2 μm / pixel or less.

  In FIG. 28A, the lower limit value is defined according to sufficient conditions of the OLPF configuration in which color moire and false resolution are free. However, the optimum setting range for α is to set the lower limit given by 1 μm / pixel smaller than the corresponding pixel pitch as the upper limit. A feeling of recovery can be realized. The optimum relationship between a and α in consideration of these matters is shown in the graph of FIG. The hatched range in the graph of FIG. 28B is the optimum range of a and α when 2.5 μm ≦ a ≦ 5.0 μm.

  The graph (black dot) on the left side of FIG. 28 (b) shows the black dots having the values of α of 3 μm, 4 μm, 5 μm, and 6 μm in FIG. 28 (a), as in FIG. 27 (b) of the first embodiment. It is plotted with a curve. The black spot of 2.5 μm is not necessary for the low-pass filter in the range smaller than 2.5 μm, and is in a level sufficiently close to the state without the low-pass filter from FIG. 11 in the range exceeding 200%. 200%. The straight line above 2 μm is an asymptotic line that becomes ∞ at 2 μm. The graph on the right side (x point) in FIG. 28B is a graph obtained by shifting the left graph to the right by 1 μm.

  As described above, when both the pixel intervals (a, b) of the image sensor 4 are in the range of 2.5 to 5 μm, the position of the extinction frequency band of the optical low-pass filter is 1.1 with respect to the Nyquist frequency of the image sensor 4. It is shown that it is optimal to set it in a range of ≦ α ≦ 2.0 times.

  In addition, when the aperture control from the maximum open aperture value that can be taken by the photographing lens 2 to F8 or more is possible, the pixel spacing value of the image sensor 4 corresponds to about 4 to 6 μm / pixel. It is preferable to use an optical low-pass filter having the following characteristics.

  Note that the assumed color filter array of the image sensor is in accordance with the first embodiment.

-Third embodiment-
In the digital camera of the third embodiment, the optical low-pass filter is different from that of the first embodiment. Other configurations are the same as those of the first embodiment, and thus the description thereof is omitted. Hereinafter, the optical low-pass filter of the third embodiment will be described.

<Optical low-pass filter configuration>
In the third embodiment, among the three types of optical low-pass filters first listed as candidates, the two-way configuration type having symmetry is used in the vertical and horizontal two-way type. As examined in the section on geometrical optical simulation, under the lens usage conditions in FIGS. 13, 16, and 17, either the vertical / horizontal two-way type or the diagonal two-way type optical low-pass filter has a weaker intensity and color moiré. As a result of seeing whether it is possible to make false resolution free, the oblique type was more advantageous under all conditions. However, even in the case of the vertical and horizontal two-way type, there is a region that can produce a sufficient result that can be made free of color moire and false resolution with a weaker intensity than before in terms of pixel pitch dependency. In this region, the lens MTF can play the role of the optical low-pass filter.

  Therefore, in the third embodiment, a vertical and horizontal two-direction optical low-pass filter is employed. FIG. 29 is a diagram showing the optimum configuration. If the color moire-free condition is not aimed completely, the range of 100 to 133% hv is continuously used during a large pixel pitch, and below a certain pixel pitch (for example, 4 μm / pixel). Set a weak optical low-pass filter.

  If the optical low-pass filter of the third embodiment is arranged in a lattice pattern with pixel intervals (pitch) (a, b) in two directions x and y orthogonal to each other, , y) Modulates the spatial frequency characteristics of incident light by separating light beams in two vertical and horizontal directions (a / α, 0) and (0, b / α) with respect to the coordinate axis.

  In FIG. 29A, the lower limit value is defined according to sufficient conditions of the OLPF configuration in which color moire and false resolution are free. However, as the optimum setting range of α, if the lower limit value given on the side 1 μm / pixel smaller than the corresponding pixel pitch is set as the upper limit value, sharpness is maintained while maintaining a level close to color moiré and false resolution free. Recovery can be realized. The optimum relationship between a and α in consideration of these matters is shown in the graph of FIG. The hatched range of the graph of FIG. 29B is the optimum range of a and α when 2.5 μm ≦ a ≦ 4.0 μm.

  The graph (black dots) on the left side of FIG. 29B shows the black dots having the values of α of 3 μm, 4 μm, 5 μm, and 6 μm in FIG. 29A, as in FIG. 27B of the first embodiment. It is plotted with a curve. The black spot of 2.5 μm has the same upper limit of α as 250% because a low-pass filter is unnecessary in a range smaller than 2.5 μm, and in the range exceeding 250%, it is the same as a state without a low-pass filter. The straight line above 2 μm is an asymptotic line that becomes ∞ at 2 μm. The graph on the right side (x point) in FIG. 29B is a graph obtained by shifting the left graph to the right by 1 μm.

  As described above, when the pixel interval (a, b) of the image sensor 4 is in the range of 2.5 to 4 μm, the position of the extinction frequency band of the optical low-pass filter is 1.45 with respect to the Nyquist frequency of the image sensor 4. It has been shown that it is optimal to set it in a range of ≦ α ≦ 2.5 times.

  Note that the assumed color filter array of the image sensor is in accordance with the first embodiment.

-Fourth embodiment-
Since the first embodiment, the second embodiment, and the third embodiment both satisfy the condition of color moire and false resolution free, the pixel pitch with the higher sharpness is selected. It is advantageous to switch and configure the dependency. FIG. 30 shows a comparison of the sharpness index given by each optical low-pass filter under the sufficient condition determined by each pixel pitch. For reference, the vertical / horizontal two-way low-pass filter satisfies the conditions for satisfying the condition of color moire and false resolution free from the point of F2.8 in FIG. 13, and the graph that calculates the corresponding sharpness index is also included. Put it on.

  From FIG. 30, it is considered that the configuration of the optical low-pass filter shown in FIG. 31 is preferable.

  Here, the reason why the optical low-pass filter may be omitted at 3 μm / pixel is added for the following reason. That is, from the geometrical optical simulation at 2 μm / pixel, F2.8 in FIG. 13, when the contrast is reduced to the Nyquist frequency to this level, the optical low-pass filter may be unnecessary if the interpolation algorithm of this embodiment is used. Thus, when the value of the contrast at that time is read from the table of FIG. 25, 1 / contrast = 9 ± 5. This corresponds to an MTF of about 0.1 or 10% with a large error.

  On the other hand, 1 / contrast = 5 ± 1 in the severe condition of F2.8 having the highest contrast of the composite system at 3 μm / pixel in FIG. Therefore, if a simulation including actual wave optics is performed, there is a high possibility that color moire and false resolution are free within the range of errors. Therefore, when an interpolation algorithm using a different color correlation is used in a single-lens reflex lens group, it can be estimated that an optical low-pass filter is unnecessary from 2.5 ± 0.5 μm / pixel or less. In this case, when the film size is 35 × 24 mm, the camera is 135 M pixels or 93 to 210 M pixels when the error range is estimated.

  As the pixel pitch decreases in this way, the MTF of a single-lens reflex lens is blurred due to the decrease. By weakening the optical low-pass filter and raising the sharpness index, these lens groups can continue to be used. Can be rescued. Further, an optical low-pass filter is selectively provided in the digital camera 1 and is not provided when the pixel interval value is 2.5 ± 0.5 μm / pixel or less. Thereby, it is possible to prevent unnecessary provision of the optical low-pass filter.

The camera system 100 according to the embodiment described above has the following operational effects.
(1) By grasping and using the fundamental capabilities of image processing, the lens MTF can replace the role of the optical low-pass filter for pixel pitch miniaturization, and the overall MTF is maintained. Is possible. Therefore, it is possible to cope with the blurring problem associated with the increase in the number of pixels of the current lens system.

(2) Using color interpolation processing, that is, the characteristics of improving the directional resolution and false color suppression effect in the demosaicing technology, we are good at high-performance image processing even when the number of pixels is not increased. The frequency band false color and false resolution countermeasures are left to the image processing unit, and the MTF of that part is kept high, and the energy of the optical low-pass filter is concentrated in a frequency band that cannot be completely handled by image processing. As a result, it is possible to obtain a high-quality interpolated image in which color moire is further suppressed while increasing the MTF intensity within the resolution limit frequency band. Therefore, the optical low-pass using the high-performance image processing algorithm makes the best use of the relationship between the inequality sign and the fact that the uncertainty principle of quantum mechanics works "cannot maintain sharpness and false color suppression beyond a certain level". By designing the filter, it is possible to achieve a level as close as possible to the establishment of an equal sign.

(3) In addition to substituting part of the role played by the MTF characteristics of the conventional optical low-pass filter with image processing technology, it began to appear prominently only as the pixel pitch of the image sensor became smaller. By replacing the wave optics lens MTF of the imaging optics with the MTF reduction effect due to geometrical aberrations that occur on the image plane on average depending on the lens usage conditions, color can be obtained under many general usage conditions. While suppressing the occurrence of moiré and false resolution, it is possible to provide images that are always sharp and focused on average.

(4) In addition, it is possible to provide an image with sharpness sufficient for appreciation by reviewing the configuration of the optical low-pass filter while following the conventional lens system, even if the number of pixels further increases, The life of existing lens systems can be extended and saved. That is, the conventional lens group can be used without preparing a new dedicated lens group in accordance with a single-lens reflex camera having an extremely high number of pixels.

(5) Further, by substituting the lens MTF and image processing for the part that relied on the conventional optical low-pass filter, the birefringent plate of the expensive optical low-pass filter can be made thinner than the conventional one. Cost reduction can be expected by reducing the number of sheets to zero.

The following modifications and supplementary explanations are added to the description of the above embodiment.
(1) In addition to a single-lens reflex large image sensor, the compact camera system is used as a set of a small image sensor and a dedicated lens. Small image sensors already use a pixel pitch of around 2.0 μm / pixel. In these systems, the wave optical MTF drop described above is too large, so the function of narrowing down is eliminated and a bright dedicated lens such as F2 is attached. Furthermore, it seems that the geometrical aberration is often designed so that the MTF is high on the bright F value side. In this case, since the MTF is kept high even when the pixel pitch is reduced, an optical low-pass filter is required.

  This is a solution in the direction indicated by USP 6,111,608. In other words, the lens performance is improved by changing the lens design while maintaining the optical low-pass filter as the pixel pitch is miniaturized. On the other hand, in the present invention, the lens system is used as it is, and the design of the optical low-pass filter is attempted with the help of the image processing performance to seek a countermeasure. Of the configurations presented this time, the optimal for the pixel pitch of a compact camera system is an optical low-pass filter composed of diagonally bi-directional 141% dd in combination with image processing using different color correlation. Conceivable.

  Here, the problem that the single-lens reflex lens optical system encounters as the pixel pitch of the image sensor is reduced will be summarized as compared with the compact camera system.

  In general, geometrical aberrations are difficult to suppress in proportion to the focal length of the lens. The lens of a single lens reflex lens generally has a focal length f of, for example, a wide angle of 17 mm, a standard of 50 mm, and a telephoto of 200 mm or more for an image sensor of 35 mm × 24 mm. On the other hand, the compact lens has an image sensor as small as 1/3 type, 4.8mm x 3.6mm, so even if the same angle of view is taken, a similar focal length is required. In general, the value is about 10 mm.

  Therefore, in general, the single-lens reflex lens having a long focal length is generally worse in geometric optical aberration, and in order to obtain a bright F value, it must be a large-diameter lens. Aberration characteristics are bad. For this reason, in a single-lens reflex system, the geometrical aberration is usually designed to be best at about F5.6 to F8. Therefore, in a single lens reflex interchangeable lens group, it is gradually difficult to maintain contrast in the vicinity of the Nyquist frequency on the open side where the MTF characteristics are poor as the image sensor is reduced. .

On the other hand, a single-lens reflex lens is usually provided with a diaphragm mechanism up to about F2.8 to F22. In this case, as shown in FIG. 18, the influence of MTF reduction due to wave-optical diffraction is reduced, and the influence is reduced even when the aperture value is about F2.8 to F11 when the pixel pitch a is 5 μm / pixel or less. It is becoming enormous. According to USP 6,111,608, the MTF decreases linearly to the limit resolution frequency 1 / (Fλ) with respect to the frequency f (lines / mm), and can be approximated by an expression that does not resolve any more.
Ie
MTF (f) = 1−f · Fλ
Here, λ = 0.54 μm. However, it is connected with the frequency expressed in k space by the relationship of k = 2πf.

FIG. 33 schematically shows the relationship between this and the limit resolution frequency f _N = 1 / (2a) of the image sensor, and the kill frequency band α / (2a) of the optical low-pass filter. Thus, in accordance with approaching the Nyquist frequency f _N is 1 / (F [lambda), resolution of geometrical optical aberration is most Any Good F8 wave optically Nyquist frequency around can not be maintained.

  As described above, from the viewpoint of MTF drop at the open side F value due to geometric optical aberration from the viewpoint of the focal length and from the side face of MTF drop at the wave optically large F value from the viewpoint of the variable aperture mechanism. This is influenced by the pixel pitch dependence of the lens optical system.

  FIG. 24 and FIG. 25 represent the two facts numerically. Comparing F8, which has the best aberration in terms of geometric optics, with the open side F2.8, which has the best performance in terms of wave optics, the contrast of the combined system of geometric aberration and wave optics MTF is almost the same for both F2.8 and F8. At the same time, the contrast is reduced as the pixel pitch is reduced, but F8 is slightly faster.

  The only way to deal with this fact is to reduce the OLPF intensity to reduce the pixel pitch. That is, it is impossible to improve the optical performance of existing lenses.

  In the first to fourth embodiments described above, the condition that the color moiré is free on the higher MTF side of these two is presented. Another method for determining the optical low-pass filter intensity is another one. There is also a way to decide. That is, in order to make the variable diaphragms, which are a major feature of single-lens reflex lenses, have substantial significance, they maintain a finite contrast up to at least the Nyquist frequency between F2.8 and F8 in the normal range. Thus, it is necessary to weaken the OLPF.

  Since the decrease in contrast is faster on the F8 side than the reduction in pixel pitch, the OLPF intensity for each pixel pitch is determined so as to always maintain a finite MTF at the Nyquist frequency with respect to the reference aperture value F8. This is the idea that even if some color moiré is allowed on the F2.8 side, the minimum lens resolution is guaranteed until the regular F8. In this way, it provides an image quality that can withstand viewing with the minimum sharpness and focus at the normal aperture value.

  When the method shown in this embodiment is used, the contrast that can make color moiré free by image processing decreases the MTF to about 0.1 as described at the end of the fourth embodiment. Is the case. However, this value depends on the image processing method, and a value of 0.1 is guaranteed when adaptive color difference correction is performed by the different color correlation direction determination and the color determination method and the color gradient determination method.

  Taking the example in which F8 is the reference aperture value, 1) the OLPF is always constant regardless of the pixel pitch from the positional relationship of the Nyquist frequency 1 / (2a) with respect to the lens limit resolution frequency 1 / (Fλ). 2) Pixel pitch region (FIG. 34 (a)) to be applied, 2) Pixel pitch region (FIG. 34 (b)) that weakens the intensity of the OLPF depending on the pixel pitch, 3) Pixel pitch region (FIG. 34) that does not require the OLPF at all 34 (c)) is divided into three types, and FIG. 34 schematically shows this.

Since 1 / (Fλ) = 1 / (8 × 0.54 μm) = 230 (lines / mm) at the time of F8, the first interval is when the Nyquist frequency 1 / (2a) is about 1/3 or less. It corresponds to. That is, from the relationship of 1 / (2a) to <1 / (3 (Fλ)) = 77 (lines / mm), a> to 6.5 μm / pixel. The third section corresponds to a region where the Nyquist frequency exceeds the resolution frequency of the lens that maintains the minimum resolution MTF value Ma at F8. That is, (1-Ma) / (Fλ) ≦ 1 / 2a
From the relationship of the left side = (1-0.1) / (8 × 0.54 μm) = 208 (lines / mm), a ≦ 2.4 μm. Here, the left side is obtained from the condition of the frequency f satisfying MTF (f) = 1−f · (Fλ) = Ma.
However, an error of about a ≦ 2.4 ± 0.5 μm is added. The second section is a pixel pitch region sandwiched between the first section and the third section.

  Furthermore, F8 at 6 μm / pixel and MTF at the Nyquist frequency at 133% hv type OLPF were actually examined using a wave optical simulator. As a result, it has been experimentally confirmed that the good lens that has been subjected to the above-described geometric optical simulation still has about 10% of MTF remaining, and the problem that the focus is not felt has already started in a serious situation. . Furthermore, it was confirmed that the condition of almost no color moire was reached at this point.

  This is because the sufficient condition of the OLPF intensity of the fourth embodiment obtained from only the geometrical aberration of F2.8 in FIG. 25 is 120% hv, but is weaker in the synthesis system MTF with wave optics. Therefore, 6 μm / pixel means that the lower limit of the intensity may already be set to 133% hv. That is, in the PSF data of the synthesis system MTF using wave optics, the lower limit value of the inequality sign of the sufficient condition obtained in the first to fourth embodiments is further increased to approximately 133 / 120≈1.1 times as a whole. It can also be said that it suggests the possibility of shifting and setting. For reference, FIG. 35 shows a signal sectional view of CZP obtained by applying PSF in the wave optical simulation at the time of 6 μm / pixel, F8, 133% hv, image height 0 mm, and best focus. Since CZP gradually becomes a high-frequency signal, it can be considered that the pixel position on the horizontal axis directly represents the frequency. It is also reproduced that the MTF has dropped to 0% at just 133%.

(2) In the description of the optical low-pass filter, what is often referred to as a two-element configuration means that two birefringent plates are used to form a four-point separation type. Further, it is normal that a (λ / 4) plate is inserted between them to restore a circularly polarized state in which linearly polarized light is degenerated.

(3) What has been called color moiré-free in terms of image quality so far means that it can be achieved with circular zone plate charts of achromatic images, and the true meaning of including chromatic parts in a Bayer array Full color moire free at is impossible unless you add 50% hv OLPF. From the standpoint of leaving luminance resolution, matching the low-density R and B components to the low-density R and B components for the G-density Bayer array is a self-denial against adopting the Bayer array. As long as the actual color filter arrangement is used, it is impossible to make the color moire free including the chromatic color portion. However, the proposed OLPF uses the term color moire free in the sense that it will be achieved for the most part.

(4) When the pixel array is not a square lattice but a rectangular lattice, the 141% dd type in two diagonal directions is ((1 /) with respect to the basic lattice vector (a, 0), (0, b) in real space. 2) It is only necessary to change the way of separating the rays as in a, ± (1/2) b).

(5) If the color filter array is not a Bayer array, for example, pixels in a square lattice Bayer array are divided into two subpixel structures, the right and left, without changing the position of the color filter, the limit resolution in the horizontal direction Since the region extends to the second Brillouin zone instead of the first Brillouin zone, when calculating the sharpness index, it is only necessary to integrate in the range of | kx | ≦ 2π / a, | ky | ≦ π / a .

(6) The above discussion can be applied to other filter arrays, for example, what is called a honeycomb array obtained by rotating a Bayer array by 45 degrees.

(7) In the above embodiment, the example in which the interpolation process is performed by the personal computer 10 has been described. However, this interpolation process may be performed in the digital camera 1. In that case, the image processing unit 5 is preferable.

(8) In the above embodiment, an example has been described in which the digital camera 1 is a single-lens reflex digital camera and the photographing lens 2 is an interchangeable interchangeable lens. However, the digital camera 1 may be a lens-integrated compact digital camera, and the photographing lens 2 may be fixed to the digital camera 1.

(9) In the above embodiment, the color filter array has been described taking the most typical Bayer array (square array of pixel pitch a) as an example. However, as described above, color filters in other arrangements may be used. For example, in addition to the Bayer array, the present invention can be applied to a structure in which R, G, and B are replaced with complementary colors Cy, Mg, and Ye, and a honeycomb array in which the Bayer is only rotated 45 °. Also, R: G: B = 6: 8: 2 (US Pat. No. 5,541,653) described in US Pat. No. 5,541,653, in which the color distribution ratio is slightly shifted from the ratio of R: G: B = 1: 2: 1 in the Bayer array. Fig. 5), R: G: B = 7: 8: 1 (Fig. 4 of US Pat. No. 5,541,653) basically has the same OLPF configuration because the shape of the Brillouin zone is not largely changed. The same α value is used for the pixel pitch dependence.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

It is a figure which shows construction of the frequency resolution area which the sampling signal in a Bayer arrangement has, ie, the 1st Brillouin zone. It is a figure which shows typically a mode that false color and false resolution aliasing generate | occur | produce in the Nyquist frequency vicinity. It is a figure which shows the example which classified the false-colored extreme point which appears when an achromatic CZP image was image | photographed by the Bayer arrangement | sequence. It is a figure which shows the method of isolate | separating one light beam into the light with an intensity | strength of 50% of each two using a birefringent plate. It is a figure which shows the list | wrist of the typical example of the convenience display of the optical low-pass filter of a 1st form, and the shift amount (light ray shift amount) of a light beam separation. It is a figure which shows a corresponding k space figure in the case of 133% hv, and a real space displacement type | formula. It is a figure which shows the list | wrist of the typical example of the convenience display of the optical low-pass filter of a 2nd form, and the shift amount (light beam shift amount) of a light beam separation. It is a figure which shows the corresponding k space figure and real space displacement type | formula in the case of 141% dd. It is a figure which shows the list | wrist of the typical example of the convenience display of the optical low-pass filter of a 3rd form, and the shift amount (light beam shift amount) of a light beam separation. It is a figure which shows a corresponding | compatible k space figure and real space displacement type | formula in the case of 100% v. It is a figure which shows the graph which represented the parameter | index of a sharp feeling as a function of the intensity | strength (alpha) of an optical low-pass filter. It is a figure which shows the table | surface which put together the typical value of the sharpness index. FIG. 10 is a diagram showing pixel pitch dependency in terms of pixel moire / false resolution free conditions for two types of aperture values F8 and F2.8 in a demosaicing algorithm using different color correlation. It is a figure which shows the graph which verified whether the change by the F value corresponding to FIG. 13 was seen with the demosaicing algorithm at the time of using the conventional same color correlation. Depends on pixel pitch in the demosaicing algorithm when using different color correlation, the conditions for becoming free of color moire and false resolution for two aperture values F8 and F2.8 in the case of a single vertical optical low-pass filter It is the figure shown regarding sex. It is a figure which shows the graph which investigated image height dependence with the algorithm using different color correlation. It is a figure which shows the graph which investigated the defocus dependence with the algorithm using different color correlation. It is a figure which shows the wave optical MTF in an aberration lens. It is a figure which shows the table | surface of the correspondence of the typical value of a pixel pitch and a Nyquist frequency. FIG. 6 is a diagram illustrating a wave optical MTF in an aberration lens as a function of contrast with respect to pixel pitch dependency with respect to aperture values F8 and F2.8. Observe the signal fluctuation width in the image after filtering with the point spread function of the lens optical system against the signal touch width of the circular zone plate chart before entering the optical low-pass filter created without going through the lens optical system It is a figure which shows a mode that it is doing. It is a figure which shows the contrast by geometric optical aberration. It is a figure which shows a mode that the contrast of the synthesizing | combining system of the contrast by geometric optical aberration and the contrast of the wave optical MTF of a non-aberration system was plotted. FIG. 24 is a table in which the contrast values of FIGS. 20, 22, and 23 are summarized in a table (in the case of F8). FIG. 24 is a table in which the contrast values in FIGS. 20, 22, and 23 are summarized in a table (in the case of F2.8). It is a figure which shows the structure of the camera system 100 of 1st Embodiment. It is a figure which shows the optimal structure of the optical low-pass filter of 1st Embodiment. It is a figure which shows the optimal structure of the optical low-pass filter of 2nd Embodiment. It is a figure which shows the optimal structure of the optical low-pass filter of 3rd Embodiment. It is the figure which compared the sharpness index | index which it gives when taking the structure of each optical low-pass filter of 1st Embodiment-3rd Embodiment. It is a figure which shows the optimal structure of the optical low-pass filter of 4th Embodiment. It is a figure which shows the Bayer arrangement | sequence of a color filter. Relationship between the resolution limit frequency f _{N =} 1 / (2a) of the image pickup element, and a diagram schematically showing the relationship between the dead band α / (2a) of the optical low-pass filter. FIG. 6 is a diagram schematically showing three pixel pitch regions from the positional relationship of Nyquist frequency 1 / (2a) with respect to lens limit resolution frequency 1 / (Fλ) when F8 is a reference aperture value. It is a figure which shows the signal sectional drawing of CZP which applied PSF by the wave optical simulation at the time of 6 micrometers / pixel, F8, 133% hv, image height 0mm, and a best focus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Shooting lens 3 Optical low-pass filter 4 Image pick-up element 5 Image processing part 6 Control part 7 Aperture 10 Personal computer 20 Memory card 30 Cable 100 Camera system

Claims (23)

A lens unit that forms an image of a subject on the imaging surface;
Of the first to nth (n ≧ 2) color components, the first color component is included in each of the pixels arranged in a grid pattern with pixel intervals (a, b) in two directions x and y orthogonal to each other. An image sensor that occupies at least half the density in a checkered pattern and other color components occupy the remaining pixels is arranged, and outputs an image signal of the subject image;
Before the light that has passed through the lens unit enters the image sensor, it is inclined in two directions ((1/2) a, (1/2) b) x (√2 / α) with respect to the (x, y) coordinate axis. ) And ((1/2) a,-(1/2) b) x (√2 / α), the Nyquist frequency in the x direction of the image sensor 1 / (2a) and y direction Frequency modulation of the subject image to kill the band connecting the spatial frequencies (α / (2a), 0) and (0, α / (2b)) that are α times the Nyquist frequency 1 / (2b) of An optical low-pass filter unit to perform,
When the pixel interval (a, b) of the image sensor is in the range of 2.5 to 5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is 1.5 with respect to the Nyquist frequency of the image sensor. A digital camera characterized in that the range is set to ≦ α ≦ 3.5 times. The digital camera according to claim 1, wherein
When the pixel interval (a, b) of the image sensor is 5 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 1.5 ≦ α ≦ 1.9 times. A featured digital camera. The digital camera according to claim 1 or 2,
When the pixel interval (a, b) of the image sensor is 4 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 1.9 ≦ α ≦ 2.83 times. A featured digital camera. The digital camera according to any one of claims 1 to 3,
When the pixel interval (a, b) of the image sensor is 3 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 2.83 ≦ α ≦ 3.5 times. A featured digital camera. A lens unit that forms an image of a subject on the imaging surface;
Of the first to nth (n ≧ 2) color components, the first color component is included in each of the pixels arranged in a grid pattern with pixel intervals (a, b) in two directions x and y orthogonal to each other. An image sensor that occupies at least half the density in a checkered pattern and other color components occupy the remaining pixels is arranged, and outputs an image signal of the subject image;
Before the light that has passed through the lens unit enters the image sensor, light separation in one direction (0, b / α) perpendicular to the (x, y) coordinate axis is performed, and the y direction of the image sensor An optical low-pass filter unit that performs frequency modulation of the subject image to kill the band of the spatial frequency α / (2b) at a position α times the Nyquist frequency 1 / (2b) of
When the pixel interval (a, b) of the image sensor is in the range of 2.5 to 5 μm / pixel, the position of the extinction frequency band of the optical low-pass filter is 1.1 with respect to the Nyquist frequency of the image sensor. A digital camera that is set in a range of ≦ α ≦ 2.0 times. The digital camera according to claim 5, wherein
When the pixel interval (a, b) of the image sensor is 5 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 1.1 ≦ α ≦ 1.33 times. A featured digital camera. The digital camera according to claim 5 or 6,
When the pixel interval (a, b) of the image sensor is 4 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 1.33 ≦ α ≦ 1.67 times. A featured digital camera. The digital camera according to any one of claims 5 to 7,
When the pixel interval (a, b) of the image sensor is 3 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 1.67 ≦ α ≦ 2.0 times. A featured digital camera. A lens unit that forms an image of a subject on the imaging surface;
Of the first to nth (n ≧ 2) color components, the first color component is included in each of the pixels arranged in a grid pattern with pixel intervals (a, b) in two directions x and y orthogonal to each other. An image sensor that occupies at least half the density in a checkered pattern and other color components occupy the remaining pixels is arranged, and outputs an image signal of the subject image;
Before the light that has passed through the lens unit enters the image sensor, the light beam is separated into two directions (a / α, 0) and (0, b / α) with respect to the (x, y) coordinate axis. And the spatial frequency (α / (2a), 0) at a position α times the Nyquist frequency 1 / (2a) in the x direction and the Nyquist frequency 1 / (2b) in the y direction An optical low-pass filter unit that performs frequency modulation of the subject image to kill each band of 0, α / (2b)),
When the pixel interval (a, b) of the image sensor is in the range of 2.5 to 4 μm / pixel, the position of the dead frequency band of the optical low-pass filter is 1.45 with respect to the Nyquist frequency of the image sensor. A digital camera characterized in that the range is set to ≦ α ≦ 2.5 times. The digital camera according to claim 9, wherein
When the pixel interval (a, b) of the image sensor is 4 ± 0.5 μm / pixel, the position of the dead frequency band of the optical low-pass filter is set to a range of 1.45 ≦ α ≦ 2.0 times. A featured digital camera. The digital camera according to claim 9 or 10,
When the pixel interval (a, b) of the image sensor is 3 ± 0.5 μm / pixel, the position of the extinction frequency band of the optical low-pass filter is set to a range of 2.0 ≦ α ≦ 2.5 times. A featured digital camera. The digital camera according to any one of claims 1, 5, and 9,
The digital camera according to claim 1, wherein the lens unit further includes an aperture mechanism having a brightness with an open aperture value of at least F2.8 or more and capable of variably controlling the aperture from the open aperture value to at least F8 or more. The digital camera according to claim 12, wherein
The digital camera according to claim 1, wherein the lens unit has at least one focal length between 50 mm and 200 mm. The digital camera according to claim 12, wherein
The digital camera according to claim 1, wherein the lens unit is a lens unit exchangeable with a plurality of types of lenses. The digital camera according to claim 14, wherein
2. The digital camera according to claim 1, wherein the lens unit is a lens unit to which an interchangeable lens group that can be used in common for a film camera and a digital camera can be mounted. The digital camera according to any one of claims 1, 5, and 9,
The digital imaging device includes an imaging device having a size of 35 mm × 24 mm, 23.4 mm × 16.7 mm, 18 mm × 13.5 mm, or any size therebetween. The digital camera according to any one of claims 1, 5, and 9,
A digital camera using an image sensor in which a color filter in which a green component occupies a half density in a checkered pattern and a red and blue component is arranged uniformly in the remaining pixels is used as the image sensor. The digital camera according to any one of claims 1, 5, and 9,
Different colors having resolution in the Nyquist frequency region by using color signals between different color components existing at minimum pixel intervals in at least two directions of the x-axis and y-axis with respect to the image signal output from the image sensor The similarity between the different colors is calculated, the direction of strong similarity is determined based on the similarity between the different colors, and the color signal of at least one common color component is generated for each pixel based on the similarity determination result A digital camera further comprising an image processing unit. The digital camera according to claim 18, wherein
The image processing unit generates a chrominance component based on the similarity determination result, performs adaptive smoothing processing for each pixel on the generated chrominance component, and applies at least one to each pixel. A digital camera that generates two common color components. The digital camera according to claim 18 or 19,
The digital camera according to claim 1, wherein the image processing unit performs processing on a still image output from the imaging device. A digital camera system,
The digital camera according to any one of claims 1, 5, and 9,
The resolution of the Nyquist frequency region is obtained by using color signals between different color components that exist at a minimum pixel interval in at least two directions of the x-axis and the y-axis with respect to the image signal output from the image sensor of the digital camera. The similarity between the different colors provided is calculated, the direction of strong similarity is determined based on the similarity between the different colors, and the color signal of at least one common color component for each pixel based on the similarity determination result An image processing unit for generating The digital camera system according to claim 21, wherein
The image processing unit generates a chrominance component based on the similarity determination result, performs adaptive smoothing processing for each pixel on the generated chrominance component, and applies at least one to each pixel. A digital camera system that generates two common color components. 24. The digital camera system according to claim 22 or 23.
The digital camera system, wherein the image processing unit performs processing on a still image output from the imaging device.

Images