JP7137290B2 - Imaging device - Google Patents

Description

TECHNICAL FIELD The present invention relates to an imaging device and an image generation method, and more particularly to a technique effective for improving the performance of an imaging device.

A digital camera mounted on a smartphone or the like needs to be made thinner. As a thinning technique for this type of digital camera, there is a technique for obtaining an object image without using a lens (for example, refer to Patent Document 1).

This technology attaches a special diffraction grating substrate to the image sensor, and obtains the incident angle of the incident light from the projection pattern generated on the image sensor by the inverse problem calculation of the light that passes through the diffraction grating substrate. It obtains an image of an object.

U.S. Patent Application Publication No. 2014/0253781

In Patent Document 1 described above, the pattern of the diffraction grating formed on the upper surface of the substrate attached to the image sensor is a special grating pattern such as a spiral pattern. An image of the object is obtained by solving an inverse problem for reproducing the image from the projection pattern received by the image sensor. There is

If the calculation is complicated, naturally the processing time becomes long, and there is a problem that the time until the photograph is displayed becomes long. In order to perform high-speed arithmetic processing, a high-performance CPU or the like is used, but in that case, there is a risk that the cost of the digital camera will increase and power consumption will increase.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technique capable of reducing the cost and power consumption of an imaging device by facilitating the detection of the incident angle of light rays passing through a grating substrate.

The above and other objects and novel features of the present invention will become apparent from the description of the specification and the accompanying drawings.

A brief outline of typical inventions disclosed in the present application is as follows.

That is, a typical imaging device has an image sensor, a modulator, and an image processor. The image sensor converts an optical image captured by a plurality of pixels arranged in an array on an imaging surface into an image signal and outputs the image signal. The modulator is provided on the light receiving surface of the image sensor and modulates the intensity of light. The image processing section performs image processing on an image signal output from the image sensor.

The modulator also has a grating substrate, a first grating pattern, and a second grating pattern. A first grid pattern is formed on a first surface of the grid substrate proximate to the light receiving surface of the image sensor. A second grid pattern is formed on a second surface opposite the first surface.

The first grid pattern and the second grid pattern are each composed of a plurality of concentric circles. The modulator intensity-modulates the light transmitted through the second grating pattern with the first grating pattern and outputs the modulated light to the image sensor.

In particular, the plurality of concentric circles in the first lattice pattern and the second lattice pattern are each formed of a plurality of concentric circles whose pitch becomes finer in inverse proportion to the reference coordinates at the center of the concentric circles. A plurality of concentric circles become finer in inverse proportion to the reference coordinates that are the centers of the concentric circles.

Among the inventions disclosed in the present application, the effects obtained by representative ones are briefly described below.

(1) It is possible to shorten the processing time up to the acquisition of the object image.

(2) The hardware cost of the imaging device can be reduced.

2 is an explanatory diagram showing an example of the configuration of the imaging device according to Embodiment 1; FIG. FIG. 2 is an explanatory diagram showing an example of photographing by the imaging device of FIG. 1; 2 is a flowchart showing an outline of image processing by an image processing circuit included in the imaging device of FIG. 1; FIG. 10 is an explanatory diagram showing an example of an in-plane deviation of a projected image from the front surface to the rear surface of the double-sided grating substrate by obliquely incident parallel light. FIG. 4 is a schematic diagram for explaining the generation of moire fringes and the frequency spectrum when the axes of the lattice patterns of the double-sided lattice substrate are aligned; FIG. 4 is an explanatory view showing an example of a double-sided lattice substrate formed by shifting the axes of the lattice pattern on the front side and the lattice pattern on the back side. FIG. 10 is a schematic diagram for explaining the generation of moire fringes and the frequency spectrum when grid patterns are arranged in a staggered manner; It is explanatory drawing which shows the calculation result of a spatial frequency spectrum image when it irradiates with a total of 10 light of a normal incidence plane wave and another 9 plane waves with a different incident angle. FIG. 11 is a bird's-eye view showing calculation results of spatial frequency spectrum images when irradiated with a total of 10 light beams, ie, a normal incident plane wave and nine other plane waves with different incident angles. FIG. 4 is an explanatory diagram for explaining angles formed by light from respective points forming an object with respect to an image sensor; FIG. 10 is an explanatory diagram showing an example of a spatial frequency spectrum when the grid pattern is shifted in the lateral direction; FIG. 10 is an explanatory diagram showing an example of a spatial frequency spectrum when the lattice pattern is shifted in the vertical direction; FIG. 11 is an explanatory diagram showing an example of the configuration of an imaging device according to Embodiment 3; 14 is a flowchart showing an outline of image processing by an image processing circuit included in the imaging device of FIG. 13; FIG. 10 is an explanatory diagram showing that the projection of the grid pattern on the front side onto the back side is magnified from the grid pattern when the object to be imaged is at a finite distance; FIG. 13 is an explanatory diagram showing an example of the configuration of an imaging device according to Embodiment 4; FIG. 11 is an explanatory diagram showing an example of the configuration of a double-sided lattice substrate according to Embodiment 5; FIG. 20 is an external view showing an example of a portable information terminal according to Embodiment 6;

For the sake of convenience, the following embodiments are divided into a plurality of sections or embodiments when necessary, but unless otherwise specified, they are not independent of each other, and one There is a relationship of part or all of the modification, details, supplementary explanation, etc.

In addition, in the following embodiments, when referring to the number of elements (including the number, numerical value, amount, range, etc.), when it is particularly specified, when it is clearly limited to a specific number in principle, etc. Except, it is not limited to the specific number, and may be more or less than the specific number.

Furthermore, in the following embodiments, the constituent elements (including element steps, etc.) are not necessarily essential, unless otherwise specified or clearly considered essential in principle. Needless to say.

Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of components, etc., unless otherwise explicitly stated or in principle, it is considered that the shape is substantially the same. It shall include things that are similar or similar to, etc. This also applies to the above numerical values and ranges.

In addition, in all the drawings for explaining the embodiments, the same members are basically given the same reference numerals, and repeated description thereof will be omitted.

Embodiments will be described in detail below.

(Embodiment 1)
<Configuration example of imaging device>
FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging device 101 according to the first embodiment.

The imaging device 101 acquires an image of an object in the outside world without using a lens for forming an image, and as shown in FIG. there is

The modulator 102 is fixed in close contact with the light-receiving surface of the image sensor 103, and has a structure in which lattice patterns 104 and 105 are formed on a lattice substrate 102a. The grating substrate 102a is made of a transparent material such as glass or plastic.

In the modulator 102, a lattice pattern 104 serving as a second lattice pattern is formed on the surface of the lattice substrate 102a. Also, the surface of the grating substrate 102a is the second surface. The lattice pattern 104 is a concentric lattice pattern in which the interval between the lattice patterns, that is, the pitch, becomes narrower in inverse proportion to the radius from the center toward the outside.

A grid pattern 105 serving as a first grid pattern is formed on the rear surface of the grid substrate 102 a , that is, the surface on the side in contact with the light receiving surface of the image sensor 103 . The rear surface of the lattice substrate 102a is the first surface.

Similar to the grid pattern 104, the grid pattern 105 is also a concentric grid pattern in which the pitch of the grid pattern becomes narrower toward the outside in inverse proportion to the radius from the center.

The lattice pattern 104 and the lattice pattern 105 are formed by evaporating aluminum or the like by, for example, a sputtering method used in semiconductor processes. The pattern is shaded by patterns with and without aluminum vapor deposition.

The formation of the grid patterns 104 and 105 is not limited to this, and they may be formed with shading by printing with an ink jet printer or the like.

The intensity of light transmitted through the grating patterns 104 and 105 is modulated by the grating patterns. The transmitted light is received by the image sensor 103 . The image sensor 103 is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

On the surface of the image sensor 103, pixels 103a, which are light receiving elements, are regularly arranged in a grid. The image sensor 103 converts an optical image received by the pixel 103a into an image signal, which is an electrical signal. An image signal output from the image sensor 103 is subjected to image processing by an image processing circuit 106 as an image processing unit and output to a monitor display 107 or the like.

<Example of shooting with an imaging device>
FIG. 2 is an explanatory diagram showing an example of photographing by the imaging device 101 of FIG. FIG. 2 shows an example in which an object 301 is photographed by the imaging device 101 and displayed on the monitor display 107 .

As shown in the figure, when photographing an object 301, one surface of the modulator 102, specifically the surface of the lattice substrate 102a on which the lattice pattern 104 is formed, faces the object 301. shooting is performed.

<Example of image processing by image processing circuit>
Next, an outline of image processing by the image processing circuit 106 will be described.

FIG. 3 is a flow chart showing an outline of image processing by the image processing circuit 106 included in the imaging apparatus 101 of FIG.

First, a moire fringe image output from the image sensor 103 is subjected to a two-dimensional Fourier transform (FFT: Fast Fourier Transform) operation for each color RGB (Red Green Blue) component to obtain a frequency spectrum (step S101). ).

Subsequently, after extracting the data of one side frequency of the frequency spectrum by the process of step S101 (step S102), the intensity of the frequency spectrum is calculated (step S103) to obtain an image.

Then, noise removal processing is performed on the obtained image (step S104), followed by contrast enhancement processing (step S105) and the like. After that, the color balance of the image is adjusted (step S106) and output as a photographed image.

Thus, the image processing by the image processing circuit 106 is completed.

<Shooting Principle of Imaging Device>
Next, the imaging principle of the imaging apparatus 101 will be described.

First, the concentric lattice patterns 104 and 105 whose pitch becomes finer in inverse proportion to the radius from the center are defined as follows. Assume that a spherical wave, which is close to a plane wave, is caused to interfere with a plane wave used as a reference light in a laser interferometer or the like.

Let r be the radius from the reference coordinate, which is the center of the concentric circles, and φ(r) be the phase of the spherical wave at that point.

と表せる。

can be expressed as

The reason why the spherical wave is represented by the square of the radius r is that the spherical wave is close to a plane wave and can be approximated only by the lowest order of expansion. When a plane wave is allowed to interfere with light with this phase distribution,

のような干渉縞の強度分布が得られる。

An intensity distribution of interference fringes such as is obtained.

this is,

を満たす半径位置で明るい線を持つ同心円の縞となる。縞のピッチをｐとすると、

Concentric fringes with bright lines at radial positions satisfying . If p is the pitch of the fringes, then

が得られ、ピッチは、半径に対して反比例して狭くなっていくことがわかる。

is obtained, and it can be seen that the pitch becomes narrower in inverse proportion to the radius.

Such fringes are called Fresnel zone plates. A grid pattern having a transmittance distribution proportional to the intensity distribution defined in this way is used as the grid patterns 104 and 105 shown in FIG.

Assume that parallel light is incident at an angle θ0 as shown in FIG. Assuming that the angle of refraction in the modulator 102 is θ, in terms of geometric optics, the light multiplied by the transmittance of the grating on the front surface is incident on the rear surface with a shift of δ=t·tan θ. If they were formed to be aligned, the transmittance of the grating on the back surface would be shifted by δ and multiplied.

At this time,

のような強度分布が得られる。

An intensity distribution such as is obtained.

It can be seen that the fourth term of this expansion produces a pattern of equally spaced stripes straight in the direction of displacement of the two gratings over the overlapping area. A fringe produced at a relatively low spatial frequency by superposition of such fringes is called a moire fringe.

Such straight, equally spaced fringes produce sharp peaks in the spatial frequency distribution obtained by the two-dimensional Fourier transform of the detected image. It becomes possible to obtain the value of δ, that is, the incident angle θ of the ray from the value of the frequency.

It is clear that the moire fringes that are obtained uniformly and at equal intervals over the entire surface are generated at the same pitch regardless of the direction of displacement due to the symmetry of the concentric lattice arrangement. Such fringes are obtained because the grid pattern is formed by the Fresnel zone plate, and it is considered impossible to obtain uniform fringes over the entire surface with any other grid pattern.

It can be seen that fringes in which the intensity of the Fresnel zone plate is modulated by moire fringes are also generated in the second term, but the frequency spectrum of the product of the two fringes is the convolution of each Fourier spectrum, so a sharp peak cannot be obtained. do not have.

(5) From the equation, only the components with sharp peaks are

のように取り出すと、そのフーリエスペクトルは、

and its Fourier spectrum is

のようになる。

become that way.

where F represents the Fourier transform operation, u, v are the spatial frequency coordinates in the x and y directions, and δ with brackets is the delta function. From this result, it can be seen that in the spatial frequency spectrum of the detected image, the peak of the spatial frequency of moire fringes occurs at the position of u=±δβ/π.

The situation is shown in FIG. In FIG. 5, from left to right, a layout diagram of the light beam and the modulator 102, moire fringes, and a schematic diagram of the spatial frequency spectrum are shown, respectively. FIG. 5(a) shows vertical incidence, FIG. 5(b) shows a case where a ray is incident from the left at an angle θ, and FIG. 5(c) shows a case where the ray is incident from the right at an angle θ. .

The grid pattern 104 formed on the front side of the modulator 102 and the grid pattern 105 formed on the back side have the same axis. In FIG. 5A, since the shadows of the lattice pattern 104 and the lattice pattern 105 match, moire fringes do not occur.

In FIGS. 5(b) and 5(c), the same moire occurs because the grating pattern 104 and the grating pattern 105 have the same displacement, and the peak positions of the spatial frequency spectra are also the same. 5(b) or FIG. 5(c).

In order to avoid this, two grating patterns 104 and 105 are preliminarily arranged as shown in FIG. It must be offset relative to the axis.

Assuming that the relative displacement of the shadows of the two gratings for a plane wave of normal incidence on axis is δ0, the displacement δ caused by a plane wave of incident angle θ is given by

のように表せる。

can be expressed as

At this time, the peak of the spatial frequency spectrum of the moire fringes of the light beam with the incident angle θ is

の位置となる。

position.

If the size of the image sensor is S, and the number of pixels in the x and y directions of the image sensor is N, the spatial frequency spectrum of the discrete image obtained by fast Fourier transform (FFT) is -N/(2S) to +N/( 2S).

From this, considering that the incident angle on the plus side and the incident angle on the minus side are equally received, the spectral peak position of the moire fringes due to the perpendicularly incident plane wave (θ = 0) is the origin (DC: direct current component) position and, for example, the center position of the + side end frequency position, that is,

の空間周波数位置とするのが妥当である。

is the spatial frequency position of .

Therefore, the relative center misalignment of the two gratings is

とするのが妥当である。

It is reasonable to assume that

FIG. 7 is a schematic diagram for explaining the generation of moire fringes and the frequency spectrum when the grid pattern 104 and the grid pattern 105 are arranged with a shift.

Similar to FIG. 5, the left side shows the layout of the light beams and the modulator 102, the middle row shows the moire fringes, and the right side shows the spatial frequency spectrum. FIG. 7(a) shows the case where the light beam is incident vertically, FIG. 7(b) shows the case where the light beam is incident from the left side at an angle θ, and FIG. This is the case of incidence at θ.

The lattice pattern 104 and the lattice pattern 105 are arranged with a shift of δ0 in advance. As a result, moire fringes also occur in FIG. 7A, and a peak appears in the spatial frequency spectrum.

The shift amount δ0 is set so that the peak position appears in the center of the spectral range on one side from the origin, as described above. At this time, in FIG. 7B, the deviation .delta. can be discriminated from the peak position of the spectrum.

The spectral image of this peak is a bright spot indicating a luminous flux at infinity, and is nothing but an image captured by the imaging device 101 in FIG.

Letting θmax be the maximum incident angle of parallel light that can be received,

より、撮像装置１０１にて受光できる最大画角は、

Therefore, the maximum angle of view that can be received by the imaging device 101 is

で与えられる。

is given by

By analogy with image formation using a general lens, when it is assumed that parallel light with an angle of view θmax is focused at the edge of the image sensor and received, the effective focal length of the imaging apparatus 101 that does not use a lens is

に相当すると考えることができる。

can be considered to be equivalent to

As shown in equation (2), it is assumed that the transmittance distribution of the grating pattern basically has sinusoidal characteristics. For example, it is conceivable to binarize the transmittance of the grid pattern, change the duty of the grid area with high transmittance and that of the area with low transmittance, and increase the transmittance by widening the width of the area with high transmittance.

In the above description, the incident light beams have only one incident angle at the same time. There must be.

Such light with a plurality of incident angles already overlaps images of a plurality of front-side gratings at the point of incidence on the rear-side grating pattern. If these mutually generate moire fringes, there is a concern that they will become noise that interferes with the detection of the moire fringes with the lattice pattern 105, which is the signal component.

However, in reality, the overlapping of the images of the lattice pattern 104 does not produce a peak of the moire image, and only the overlapping with the lattice pattern 105 on the rear surface side produces a peak.

The reason will be explained below.

First, the major difference is that the overlap of the shadows of the grating pattern 104 on the surface side caused by light rays of a plurality of incident angles is not a product but a sum. In the overlap of the shadow of the grid pattern 104 and the grid pattern 105 by light of one incident angle, the intensity distribution of the light that is the shadow of the grid pattern 104 is multiplied by the transmittance of the grid pattern 105 to obtain the grid on the back side. A light intensity distribution after passing through the pattern 105 is obtained.

On the other hand, the overlapping of the shadows caused by the plurality of light beams incident on the grating pattern 104 on the surface side at different angles is the overlapping of light beams, so that the result is not the product but the sum. For the sum,

のように、もとのフレネルゾーンプレートの格子の分布に、モアレ縞の分布を乗算した分布となる。

, the distribution is obtained by multiplying the distribution of the original fresnel zone plate grid by the distribution of moire fringes.

Therefore, its frequency spectrum is represented by the overlap integral of each frequency spectrum. Therefore, even if the moire spectrum has a single sharp peak, in practice, there is only a ghost of the frequency spectrum of the Fresnel zone plate at that location. That is, no sharp peaks appear in the spectrum.

Therefore, the spectrum of the moire image detected even when light of a plurality of incident angles is input is always only the moire of the product of the grid pattern 104 on the front side and the grid pattern 105 on the back side, and the grid pattern 105 is single. Therefore, there is only one detected spectral peak for one incident angle.

<Confirmation of shooting principle>
8 and 9 show the results of simulations performed to confirm the principle.

FIG. 8 is an explanatory diagram showing calculation results of spatial frequency spectrum images when irradiated with a total of 10 light beams, that is, a normal incident plane wave and 9 other plane waves with different incident angles. FIG. 9 is a bird's-eye view showing the calculation result of the spatial frequency spectrum image when irradiated with a total of 10 lights, ie, a vertically incident plane wave and 9 other plane waves with different incident angles.

In both cases, the sensor size of the image sensor 103 is 20 mm square, the viewing angle θmax=±70°, the incident side and output side lattice coefficient β=50 (rad/mm2), δ0=0.8 mm, the number of pixels 1024×1024, and the modulator 102. When the substrate thickness is 1 mm and the substrate refractive index is 1.5, the normal incident plane wave, the incident light of θx = 50° and θy = 30°, and the incident light of θx = -30° and θy = 70°, Incident light at θx=10°, θy=−20°, incident light at θx=20°, θy=30°, incident light at θx=30°, θy=−40°, θx=−10°, Incident light at θy=40°, incident light at θx=-20°, θy=-30°, incident light at θx=-30°, θy=0°, and incident light at θx=40°, θy=50° It is a spectrum when a total of 10 plane waves are incident on the incident light.

FIG. 8 is a black-and-white inverted image of the spectrum image, and FIG. 9 is a bird's-eye view showing the luminance of the spectrum image. The original moire image itself has a fine lattice pitch and is not visible even if it is displayed as a drawing of this specification, so it is omitted.

In the figure, the center is the DC component and the periphery is the entire spatial frequency spectrum region of ±N/2S. Since the DC component has a large value, it is masked and removed, and only the peak component to be detected is displayed. Furthermore, since the peak width of the spectrum is narrow as it is and it is difficult to visually recognize it, the contrast is emphasized.

In addition, in FIG. 8, the positions of the signal peaks are indicated by circles. Since the bird's-eye view of FIG. 9 cannot be displayed as it is because the drawn line does not pass through the peak, the result of applying a mesh size averaging filter is displayed.

All of them indicate that basically 10 peaks can be detected as a total of 20 peaks on both positive and negative sides of the origin. In this case, the pitch of the outermost circumference of the grid pattern was about 6 μm, and the effective focal length was about 12.4 mm.

Here, the correspondence between parallel light, which has been described for detection, and light from an actual object will be schematically described with reference to FIG. 10 .

FIG. 10 is an explanatory diagram for explaining angles formed by light from each point forming an object with respect to the image sensor.

Strictly speaking, the light from each point forming the subject 301 is transmitted as a spherical wave from a point light source to the modulator 102 and the image sensor 103 of the imaging device 101 in FIG. Incident.

At this time, if the grid sensor integrated substrate is sufficiently small or sufficiently far from the subject 301, the incident angles of the light that illuminates the grid sensor integrated substrate can be considered to be the same from each point.

The spatial frequency displacement Δu of the moire with respect to the minute angular displacement Δθ obtained from the equation (9) is less than or equal to 1/S, which is the minimum resolution of the spatial frequency of the image sensor.

のように表せる。

can be expressed as

From this, if Δθ<0.18°, this is a condition that can be realized at a distance of 6 m from the subject with a sensor size of 20 mm.

From the analogy of the above results, it can be seen that the imaging apparatus of the present invention can image an object at infinity.

As described above, an object image in the external world can be obtained by a simple calculation such as fast Fourier transform (FFT). As a result, the processing time until acquisition of the object image can be shortened.

In addition, since a high-performance arithmetic processing device is not required, the hardware cost of the imaging device 101 can be reduced. Furthermore, the power consumption of the imaging device 101 can be reduced by shortening the processing time of arithmetic operations. can be done.

(Embodiment 2)
<Overview>
In the first embodiment, the image output from the imaging device 101 is vertically long, but in the second embodiment, a case in which the output image is horizontally long will be described.

In the first embodiment, as described above, the grid pattern 104 and the grid pattern 105 are formed to be shifted in the x (horizontal) direction of the image sensor 103 . In other words, the rectangular image output from the image processing circuit 106 is formed so as to be shifted in the long side direction.

<Example of lattice pattern formation>
FIG. 11 is an explanatory diagram showing an example of the spatial frequency spectrum when the grid patterns 104 and 105 are shifted in the lateral direction.

At this time, the shape of the image sensor 103 is square, and the pixel pitch is the same in both the x direction and the y direction. In this case, as shown on the right side of FIG. 11, the spatial frequency spectrum of the output of the image sensor shows that the image is reproduced separately to the left and right within the frequency range of ±N/S for both x and y. Become.

However, in the example shown in FIG. 11, the image is basically limited to a vertically long area. Generally, an image acquired by a digital camera or the like is a horizontally long rectangle with an aspect ratio of 3:2 or 4:3, for example. Therefore, it is preferable to arrange the lattice patterns 104 and 105 suitable for horizontally long rectangles as shown in FIG. 12, for example.

FIG. 12 is an explanatory diagram showing an example of the spatial frequency spectrum when the grid patterns 104 and 105 are shifted in the vertical direction.

As shown in FIG. 12, the lattice pattern 104 and the lattice pattern 105 are shifted in the vertical direction of the image sensor, that is, in the y direction of the image sensor. In other words, the lattice pattern 104 and the lattice pattern 105 are shifted in the short side direction of the rectangular image output from the image processing circuit 106 . As a result, the image formed in the spatial frequency space of the image sensor output is separated vertically as shown on the right side of FIG.

As described above, the image output from the imaging device 101 can be horizontally long. As a result, an image can be obtained in the same manner as with a general digital camera, so the versatility of the imaging apparatus 101 can be enhanced.

(Embodiment 3)
<Overview>
In the modulator 102 of the first and second embodiments, the grating pattern 104 and the grating pattern 105 having the same shape are formed on the front surface and the rear surface of the grating substrate 102a, respectively, so that the angle of the incident parallel light is changed to that of the moire fringes. The image was constructed by detecting from the spatial frequency spectrum of

The grid pattern 105 on the back side is an optical element that modulates the intensity of incident light in close contact with the image sensor 103 . Therefore, by effectively setting the sensitivity of the image sensor in consideration of the transmittance of the grid pattern 105 on the back side, it is possible to virtually generate moiré in the processed image.

<Configuration example of imaging device>
FIG. 13 is an explanatory diagram showing an example of the configuration of the imaging device 101 according to the third embodiment.

The imaging device 101 of FIG. 13 differs from the imaging device 101 of Embodiment 1 of FIG. 1 in that the lattice pattern 105 shown in FIG. 1 is not formed on the back side of the lattice substrate 102a. Since other configurations are the same as those in FIG. 1, description thereof is omitted.

By adopting the configuration shown in FIG. 13, the number of lattice patterns formed on the lattice substrate 102a can be reduced by one surface. Thereby, the manufacturing cost of the modulator 102 can be reduced.

However, in this case, the pitch of the pixels 103 a of the image sensor 103 may be fine enough to reproduce the pitch of the grid pattern, or coarse enough to reproduce the pitch of the grid pattern with the pixel pitch of the image sensor 103 . is necessary.

When the lattice pattern is formed on both sides of the lattice substrate 102a, the pitch of the lattice pattern does not necessarily have to be resolved by the pixels 103a of the image sensor 103, and only the moiré image needs to be resolved. Therefore, the pitch of the grid pattern can be determined independently of the pixel pitch.

However, when the grid pattern is reproduced by the image sensor 103, the resolution of the grid pattern and the image sensor 103 must be the same. Therefore, the image processing circuit 106 is provided with an intensity modulation circuit 106c corresponding to the lattice pattern 105 (FIG. 1) on the back side for generating moiré on the output image of the image sensor 103. FIG.

<Example of image processing by image processing circuit>
FIG. 14 is a flow chart showing an outline of image processing by the image processing circuit 106 included in the imaging apparatus 101 of FIG.

14 differs from the flowchart of FIG. 3 of the first embodiment in the process of step S201. In the process of step S201, the intensity modulation circuit 106c described above generates a moire fringe image corresponding to the lattice pattern on the back side of the image output from the image sensor 103. FIG.

After that, the processing of steps S202 to S208 in FIG. 14 is the same as the processing of steps S101 to S107 in FIG. 3 of the first embodiment, so the description is omitted here.

Thus, by providing the intensity modulation circuit 106c, it is possible to obtain the same effect as making the grid pattern 105 (FIG. 1) on the rear surface side variable, and the detected light does not necessarily have to be parallel light. It is possible.

<About focusing>
FIG. 15 is an explanatory diagram showing that the projection of the grid pattern 104 on the front side onto the back side is enlarged from the grid pattern 104 when the object to be imaged is at a finite distance.

As shown in FIG. 15, when a spherical wave from a point 1301 constituting an object irradiates the grid pattern 104 on the surface side and its shadow 1302 is projected on the lower surface, the image projected on the lower surface is , is scaled almost uniformly.

Therefore, if the transmittance distribution of the lattice pattern on the back side (corresponding to the lattice pattern 105 in FIG. 1) designed for parallel light is multiplied as it is, linear moire fringes at equal intervals will not occur. However, if the grid on the bottom surface is enlarged in accordance with the uniformly enlarged shadows of the grid pattern 104 on the front side, the enlarged shadows 1302 again cause linear moire fringes at regular intervals. be able to.

This makes it possible to selectively reproduce light from a point 1301 at a distance that is not necessarily infinite. This makes it possible to perform focusing, and instead of photographing at infinity as shown in the first embodiment, it is possible to photograph by focusing on an arbitrary position.

As described above, the convenience of the imaging device 101 can be enhanced.

(Embodiment 4)
In the fourth embodiment, a technique for making the grid pattern 104 on the surface side of FIG. 1 variable will be described.

<Configuration example and operation example of imaging device>
FIG. 16 is an explanatory diagram showing an example of the configuration of the imaging device 101 according to the fourth embodiment.

The imaging device 101 of FIG. 16 differs from the imaging device 101 of the first embodiment shown in FIG. where it was established. Note that the configuration of the image sensor 103 in FIG. 16 is the same as that in FIG. 1, so description thereof will be omitted.

The liquid crystal portion 108 has a structure in which a liquid crystal layer (not shown) is provided on a glass substrate (not shown) on which a transparent electrode or the like is formed. ing.

An arbitrary lattice pattern 1403 is displayed on this liquid crystal layer, and the lattice pattern 1403 acts as the lattice pattern 104 on the surface side. A lattice pattern 105 is formed on the back side of the lattice substrate 102a in the modulator 102, as in FIG.

A focus position specification input unit 109 is an input unit for setting a focus position, which is information such as a distance to an object, and is connected to the image processing circuit 106 . The image processing circuit 106 is also provided with a liquid crystal drive circuit 106a and a grid pattern generation circuit 106b.

The lattice pattern generation circuit 106b generates an optimum lattice pattern for focusing based on the focus position input from the focus position specification input unit 109. FIG. The liquid crystal drive circuit 106a controls display by applying voltage to the transparent electrodes formed on the glass substrate so that the grid pattern generated by the grid pattern generation circuit 106b is displayed on the liquid crystal layer of the liquid crystal section 108. FIG.

Basically, the light from a point 1301 at a finite distance, which is closer than infinity, is divergent light. This means that the pattern 105 should be displayed in a slightly reduced size.

As described above, it is possible to achieve faster focusing.

(Embodiment 5)
In the fifth embodiment, another example of the grid pattern on the surface side formed on the double-sided grid substrate will be described.

<Example of lattice pattern formation>
FIG. 17 is an explanatory diagram showing an example of the configuration of modulator 102 according to the fifth embodiment.

In the first embodiment, the lattice pattern 104 (FIG. 1) of the modulator 102 is formed by printing, sputtering, or the like, but the modulator 102 in FIG. The lattice pattern 105 on the rear surface side of the lattice substrate 102a is the same as that of the first embodiment shown in FIG.

In this case, cylindrical lenses 110 are arranged on the surface of the grating substrate 102a in the modulator 102 so as to form a pattern similar to the grating pattern 104 in FIG. The cylindrical lens 110 is a lens whose surface is formed of a cylindrical surface, and has a curvature of a convex lens in the vertical direction and no curvature in the horizontal direction.

By forming the lattice pattern with the cylindrical lens 110 in this way, the loss of the amount of light can be greatly reduced. For example, as described in the first embodiment, in a grid pattern that is shaded by a printed pattern or the like, the printed portion of the grid pattern blocks light, resulting in a large loss of light. become.

On the other hand, in the case of the cylindrical lens 110, light utilization efficiency can be improved without blocking light.

As described above, the S/N ratio (Signal-to-Noise ratio) of the imaging apparatus 101 can be increased, so that the drawing performance can be improved.

(Embodiment 6)
<Configuration example of portable information terminal>
In the sixth embodiment, a mobile information terminal configured using the imaging device 101 in the fifth embodiment will be described.

FIG. 18 is an external view showing an example of a mobile information terminal 200 according to the sixth embodiment.

The mobile information terminal 200 is, for example, a smart phone. Note that the mobile information terminal 200 is not limited to a smart phone, and may be a mobile terminal having a built-in camera, such as a tablet.

The portable information terminal 200 incorporates an imaging device 101 . An opening window 202 is provided on the rear surface of this mobile information terminal 200, and the modulator 102 of FIG.

A knob 201 for focus adjustment is provided on one long side of the portable information terminal 200 . This knob 201 corresponds to the focus position designation input section 109 of the fourth embodiment.

A focus position is set by turning the knob 201, and an arbitrary lattice pattern 1403 is displayed on the liquid crystal layer of the liquid crystal section 108 in FIG. 16 according to the set focus position. As a result, an image of an object at any distance can be captured.

The imaging apparatus 101 can increase the effective focal length according to the formula (14) shown in the first embodiment. Therefore, the aperture can be enlarged while the imaging device 101 is kept thin.

In the case of a smartphone digital camera using a general lens, the lens opening must be made small in order to reduce the thickness of the information portable device. As a result, the focal length becomes short, and the image becomes flat and blurry.

On the other hand, in the imaging device 101, the aperture can be enlarged as described above, so that beautiful blurring can be produced.

The invention made by the present inventor has been specifically described above based on the embodiment, but the present invention is not limited to the above embodiment, and can be variously modified without departing from the scope of the invention. Needless to say.

In addition, the present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations.

Also, part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. . Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

101 Imaging device 102 Modulator 102a Grid substrate 103 Image sensor 103a Pixel 104 Grid pattern 105 Grid pattern 106 Image processing circuit 106a Liquid crystal drive circuit 106b Grid pattern generation circuit 106c Intensity modulation circuit 107 Monitor display 108 Liquid crystal unit 109 Focus position designation input unit 110 Cylindrical lens 200 Personal digital assistant 201 Knob 202 Opening window

Claims (6)

An imaging device,
an image sensor that converts light received by a plurality of pixels arranged on a light receiving surface into an electrical signal and outputs the electrical signal;
an optical element arranged on the light receiving surface side of the image sensor and transmitting light;
an image processing circuit that performs image processing on an electrical signal output from the image sensor;
with
A plurality of concentric cylindrical lenses are formed on one surface of the optical element, and converge light incident on the concentric cylindrical lenses,
The image sensor receives the light condensed by the optical element and outputs an image signal containing the concentrically condensed image;
The image processing circuit is
inputting an image signal including the concentrically condensed image output from the image sensor;
generating a moire fringe image when combining the concentrically focused image included in the input image signal with another concentric image data;
performing processing including a two-dimensional Fourier transform on the moire fringe image to generate an image;
Imaging device. The imaging device according to claim 1,
The cylindrical lens has a convex lens curvature on one surface and a lens shape with no curvature on the other surface,
Imaging device. The imaging device according to claim 1,
The concentric cylindrical lens is formed with a narrower width from the center toward the outside,
Imaging device. The imaging device according to claim 1,
The another concentric circular image data is a narrow pattern toward the outside from the center,
Imaging device. The imaging device according to claim 1,
the other concentric circular image data can enlarge or reduce the size of the concentric circular pattern;
Imaging device. In the imaging device according to claim 5,
the other concentric circular image data enables focusing by enlarging or reducing the size of the concentric circular pattern;
Imaging device.

Images