JP7287283B2 - Imaging device and imaging method - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to an imaging device and an imaging method, and more particularly to an imaging device and an imaging method capable of capturing a bright image without using an expensive large-aperture lens.

Optical efficiency (light utilization efficiency) in an imaging device can be improved by increasing the size of the lens.

However, as the size of the lens increases, the cost of the lens increases, and the increase in cost is particularly large for lenses for infrared light.

Therefore, as a technology to improve the light utilization efficiency without increasing the size of the lens, the incident light from the subject is split by a plane mirror, divided into multiple areas on the image sensor, and projected as multiple images. A technique has been proposed for generating an image with high light utilization efficiency by reconstructing an image using a plurality of images after performing independent filter processing (see Patent Document 1).

U.S. Publication No. 2016/0065938

However, in the technique disclosed in Patent Document 1, the reconstructed image is generated from a part of the image having the resolution of the imaging device, so the resolution is degraded.

The present disclosure has been made in view of such circumstances, and in particular, enables a bright image to be captured without using an expensive large-aperture lens.

An imaging device according to a first aspect of the present disclosure includes a guiding section that guides incident light from a subject to an imaging element, a modulating section that modulates the incident light guided by the guiding section, and an imaging unit that captures an image of the incident light as a pixel signal by the imaging element ; and a coefficient that is set according to the incident angle of the incident light to the imaging element that has been modulated by the modulating unit for each distance to the subject. and a signal processing unit that reconstructs the pixel signals as a final image by signal processing using the set .

An imaging method according to a first aspect of the present disclosure includes a guiding process for guiding incident light from a subject to an imaging device, a modulation process for modulating the incident light guided by the guiding process, and Imaging processing for imaging the incident light as a pixel signal by the imaging device , and a coefficient set according to an incident angle of the incident light to the imaging device modulated by the modulation processing for each distance to the subject. and signal processing for reconstructing the pixel signals as a final image by signal processing using the set .

In a first aspect of the present disclosure, incident light from a subject is guided to an imaging device, the guided incident light is modulated, the modulated incident light is imaged by the imaging device as pixel signals, and the subject is The pixel signals are reconstructed as a final image by signal processing using a coefficient set that is set according to the incident angle of the modulated incident light to the imaging element for each distance to.
An imaging device according to a second aspect of the present disclosure includes a guiding section that guides incident light from a subject to an imaging element, a modulating section that modulates the incident light guided by the guiding section, and and a signal processing unit for reconstructing the pixel signals into a final image by signal processing. The modulating unit collects the incident light guided by the guiding unit. The imaging device includes a lens that emits light, and a cylindrical mirror surface that reflects incident light condensed by the lens and guides the incident light to the imaging unit.
In a second aspect of the present disclosure, incident light from a subject is guided to an imaging device, the guided incident light is modulated, the modulated incident light is imaged as a pixel signal, and the pixel signal is subjected to signal processing. The light is reconstructed as a final image by the lens, and the guided incident light is collected by the lens, and the incident light collected by the lens is reflected by the cylindrical mirror surface and guided to the imaging unit.

According to one aspect of the present disclosure, it is possible to capture a bright image without using an expensive large-aperture lens.

It is a figure explaining the structural example of a general imaging device. 2 is a diagram for explaining the principle of imaging in the imaging device of FIG. 1; FIG. 1 is an external perspective view illustrating a configuration example of an imaging device according to a first embodiment of the present disclosure; FIG. BRIEF DESCRIPTION OF THE DRAWINGS It is side sectional drawing explaining the structural example of 1st Embodiment of the imaging device of this indication. 5 is a diagram for explaining the imaging principle of the imaging device of FIGS. 3 and 4; FIG. It is a figure explaining the principle of reconstruction. FIG. 5 is a flowchart for explaining imaging processing by the imaging device of FIGS. 3 and 4; FIG. It is a figure explaining the modification of the 1st embodiment of this indication. FIG. 10 is an external perspective view illustrating a configuration example of a second embodiment of an imaging device of the present disclosure; FIG. 10 is a side cross-sectional view illustrating a configuration example of a second embodiment of an imaging device of the present disclosure; 11A and 11B are diagrams for explaining a subject plane imaged in FIGS. 9 and 10, an image imaged by an imaging element, and a reconstructed final image; FIG. FIG. 10 is a side cross-sectional view illustrating a configuration example of a second embodiment of an imaging device of the present disclosure; It is a figure explaining the relationship between the aperture angle of a mirror surface, and light utilization efficiency. It is a figure explaining the relationship between the aperture angle of a mirror surface, and light utilization efficiency. FIG. 11 is a flowchart for explaining imaging processing by the imaging device of FIGS. 9 and 10; FIG. It is a figure explaining the modification of the 2nd Embodiment of this indication.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description.

Embodiments for implementing the present technology will be described below. The explanation is given in the following order.
1. General imaging device 2 . First embodiment 3. Modified example of the first embodiment 4. Second embodiment 5. Modified example of the second embodiment

<<1. General imaging device>>
<Composition of general imaging device>
Before describing the configuration of the present disclosure, the configuration of a general imaging device will be described.

FIG. 1 is a side cross-sectional view of the configuration of a general imaging device. The imaging device 11 of FIG. 1 includes a lens 31, an imaging element 32, and an output unit 33, which are integrally housed in the case 21. As shown in FIG.

The lens 31 collects incident light within a field of view (FOV) and focuses it on the imaging surface of the imaging device 32 . More specifically, the lens 31 converges and projects the light within the field of view FOV at the angle of view θ onto the imaging element 32 to form an image on the imaging plane of the imaging element 32 .

The imaging device 32 is composed of a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, captures an image formed by condensing light through the lens 31, and outputs it as a signal for each pixel. Output to unit 33 .

The output unit 33 performs signal processing based on the pixel-by-pixel signal output from the imaging device 32 and outputs the processed signal as an image signal.

The center positions of the lens 31 and the imaging element 32 with respect to the light transmission direction are aligned with the optical axis AX.

<Image pickup principle of general image pickup device>
Next, with reference to FIG. 2, the principle of imaging by the imaging device 11 of FIG. 1 will be described. 2, only the lens 31, the image sensor 32, and the output unit 33, which are required for explanation, are shown in the configuration of the imaging device 11. As shown in FIG.

A plane on which an object exists, which is a predetermined distance from the imaging position of the imaging device 11 and is perpendicular to the optical axis AX, is defined as an object plane S, and points P and Q on the object plane S are considered.

When the point P on the object plane S is a point light source, the diffused light emitted from the point P, which is the point light source, passes through the lens 31 as indicated by the solid line and is condensed on the imaging surface of the imaging element 32. is incident on the point P'.

Similarly, diffused light emitted from point Q, which is a point light source, passes through lens 31 as indicated by the dotted line, is condensed on the imaging surface of imaging element 32, and enters point Q'.

Similarly, other points are condensed by the lens 31 and incident on the imaging surface of the imaging element 32 .

That is, the lens 31 converges the diffused light from each point light source on the object surface S, converges the light on the corresponding point on the imaging surface of the imaging element 32, and forms an image.

As a result, the image of the subject plane S is projected onto the imaging plane of the imaging element 32 , and as a result, the imaging element 32 captures an image of the subject plane.

That is, points on the object plane S and points on the imaging plane of the imaging device 32 have a one-to-one correspondence. For this reason, in order to improve the light utilization efficiency, it is necessary to allow more diffused light emitted from each point light source to enter the lens 31 and condense it, so the diameter of the lens 31 must be increased.

However, increasing the diameter of the lens 31 not only increases the cost, but also increases the size of the device configuration and reduces the portability.

<<2. First Embodiment>>
Next, a configuration example of the imaging device according to the first embodiment of the present disclosure will be described with reference to FIGS. 3 and 4. FIG. 3 is a schematic perspective view of the imaging device 41, and FIG. 4 is a side cross-sectional view.

The imaging device 41 of FIGS. 3 and 4 includes a truncated cone-shaped mirror surface 51 , a random mask 52 , an imaging element 53 , a reconstruction section 54 and an output section 55 .

The truncated cone-shaped mirror surface 51 is formed into a truncated cone shape and has a curved mirror surface on the inner side. The respective center positions of the portion 51a and the small opening portion 51b are aligned with the optical axis AX.

In addition, incident light from the subject plane G1 enters the large aperture 51a and enters the imaging element 53 through the random mask 52 provided in the small aperture 51b. The small opening 51b has substantially the same size as the random mask 52 and the imaging element 53, but is large enough to contain the random mask 52 and the imaging element 53 in the small opening 51b.

More specifically, as shown by the solid line in FIG. 5, the optical path of the incident light from the point P on the object plane G1 passes through the large opening 51a, passes through the random mask 52, and passes directly onto the imaging device 53. , and after passing through the opening 51a and being reflected on the mirror surface 51, as indicated by the dashed line and the dotted line in FIG. There is an optical path incident on P'.

That is, in the imaging device 41 of FIGS. 3 and 4, among the diffused light from the point P, the light that would not enter the imaging device 53 if the truncated cone-shaped mirror surface 51 was not provided is a truncated cone-shaped light. By being reflected by the mirror surface 51, the light is incident on a point P' on the imaging element 53 as diffused light from the virtual point P1.

As a result, provision of the truncated cone-shaped mirror surface 51 makes it possible to improve the light utilization efficiency.

Specifically, the light condensing ability is approximately (area of the large opening 51a)/(area of the large opening 51b) times that of the case where the truncated cone-shaped mirror surface 51 is not provided. That is, when the large opening 51a has an area, for example, five times the area of the small opening 51b, the amount of light incident on the imaging device 53 through the random mask 52 is The amount of light is approximately five times the amount of light when it is not on. That is, in this case, the imaging device 41 can capture an image five times brighter.

The random mask 52 modulates the light incident from the subject plane G<b>1 via the truncated cone-shaped mirror surface 51 and causes the light to enter the imaging plane of the imaging element 53 .

Here, in the random mask 52, as shown in the upper right part of FIG. A mask pattern is formed.

The unit size Δ is at least larger than the pixel size of the image sensor 53 . A small gap of distance d is provided between the imaging element 53 and the random mask 52 .

For example, as shown in the upper left part of FIG. 6, from the point light sources PA, PB, and PC on the object plane G1, each of the positions Pa, Pb, and Pc on the imaging element 53 that is incident after passing through the random mask 52. Assume that rays of light intensities a, b, and c are incident.

As shown in the upper left part of FIG. 6, the detection sensitivity of each pixel has directivity according to the incident angle by modulating the incident light through the randomly set apertures in the random mask 52. become. Giving incident angle directivity to the detection sensitivity of each pixel referred to here means giving different light receiving sensitivity characteristics according to the incident angle of incident light depending on the area on the image sensor 53 . be.

That is, assuming that the light source constituting the object plane G1 is a point light source, light rays with the same light intensity emitted from the same point light source are incident on the imaging device 53. , the incident angle changes for each area on the imaging surface of the imaging device 53 by being modulated by the random mask 52 . Since the random mask 52 changes the incident angle of the incident light according to the area on the image sensor 53, the light receiving sensitivity characteristic, that is, the incident angle directivity, is obtained. However, the random mask 52 provided in front of the image pickup surface of the image pickup device 53 detects with different sensitivities for each region on the image pickup device 53, and detection signals with different detection signal levels are detected for each region. be.

More specifically, as shown in the upper right part of FIG. 6, detection signal levels DA, DB, and DC of pixels at positions Pa, Pb, and Pc on the image sensor 53 are expressed by the following equations (1) to (3).

DA=α1×a+β1×b+γ1×c
... (1)
DB=α2×a+β2×b+γ2×c
... (2)
DC=α3×a+β3×b+γ3×c
... (3)

Here, α1 is a coefficient for the detection signal level a set according to the incident angle of the light beam from the point light source PA on the object plane G1 to be restored at the position Pa on the imaging device 53 . The light rays from the point light source PA include both those that are directly incident on the imaging element 53 and those that are reflected by the truncated cone-shaped mirror surface 51 .

β1 is a coefficient for the detection signal level b that is set according to the incident angle of the light beam from the point light source PB on the object plane G1 to be restored at the position Pa on the imaging device 53 . The light rays from the point light source PB include both those that directly enter the imaging element 53 and those that are reflected by the truncated cone-shaped mirror surface 51 .

Furthermore, γ1 is a coefficient for the detection signal level c set according to the incident angle of the light beam from the point light source PC on the object plane G1 to be restored at the position Pa on the image sensor 53 . The light rays from the point light source PC include both those that directly enter the imaging device 53 and those that are reflected by the truncated cone-shaped mirror surface 51 .

Therefore, (α1×a) of the detection signal level DA indicates the detection signal level due to the light beam from the point light source PA at the position Pc.

Also, (β1×b) of the detection signal level DA indicates the detection signal level due to the light beam from the point light source PB at the position Pc.

Further, (γ1×c) of the detection signal level DA indicates the detection signal level due to the light beam from the point light source PC at the position Pc.

Therefore, the detection signal level DA is expressed as a composite value obtained by multiplying each component of the point light sources PA, PB, and PC at the position Pa by respective coefficients α1, β1, and γ1. Hereinafter, the coefficients α1, β1, and γ1 are collectively referred to as a coefficient set.

Similarly, the coefficient sets α2, β2, γ2 for the detected signal level DB at the point light source PB correspond to the coefficient sets α1, β1, γ1 for the detected signal level DA at the point light source PA, respectively. Also, the coefficient sets α3, β3, γ3 for the detection signal level DC at the point light source PC correspond to the coefficient sets α1, β1, γ1 for the detection signal level DA at the point light source PA, respectively.

However, the detection signal levels of the pixels at the positions Pa, Pb, and Pc are values expressed by the product sum of the light intensities a, b, and c of the light beams emitted from the point light sources PA, PB, and PC, respectively, and the coefficients. is. For this reason, these detection signal levels are a mixture of the light intensities a, b, and c of the light beams emitted from the point light sources PA, PB, and PC, respectively. is different from An image composed of detection signal levels DA, DB, and DB of pixels at positions Pa, Pb, and Pc corresponds to image G2 in FIG.

That is, a set of coefficients α1, β1, γ1, a set of coefficients α2, β2, γ2, a set of coefficients α3, β3, γ3, and detection signal levels DA, DB, DC are used to construct simultaneous equations, and light intensities a, b , c, the pixel values of the respective positions Pa, Pb, and Pc are obtained as shown in the lower right part of FIG. As a result, a restored image (final image), which is a set of pixel values, is reconstructed and restored.

Also, when the distance between the imaging device 53 and the object plane G1 shown in the upper left part of FIG. By changing this coefficient set, it is possible to reconstruct restored images (final images) of object planes at various distances.

Therefore, by changing the coefficient set to correspond to various distances in one imaging, images of the object plane at various distances from the imaging position can be reconstructed.

As a result, in imaging using the imaging device 41 of FIGS. 3 and 4, it is necessary to be aware of a phenomenon such as so-called out-of-focus, in which imaging is performed with the imaging device 11 using a lens in an out-of-focus state. If the image is captured so that the desired subject is included in the field of view FOV, images of the subject plane at various distances can be reconstructed after imaging by changing the coefficient set according to the distance. .

Note that the detection signal level shown in the upper right portion of FIG. 6 is not the detection signal level corresponding to the image in which the image of the object is formed, and therefore is not the pixel value. Also, the detection signal level shown in the lower right part of FIG. becomes. That is, the restored image (final image) of this object plane G1 corresponds to the image G3.

Such a configuration enables the imaging device 41 to function as a so-called lensless imaging device. As a result, since the imaging lens is not an essential component, it is possible to reduce the height of the imaging device, that is, to reduce the thickness in the light incident direction in the configuration that realizes the imaging function. Also, by varying the coefficient set, it is possible to reconstruct and restore the final image (restored image) on the object plane at various distances.

Hereinafter, the image captured by the image pickup device 53 before reconstruction will be simply referred to as the captured image, and the image reconstructed and restored by signal processing the captured image will be referred to as the final image (restored image). image). Therefore, from one shot image, by variously changing the coefficient set described above, images on the object plane G1 at various distances can be reconstructed as the final image.

The reconstruction unit 54 has the above-described coefficient set, and uses the coefficient set according to the distance from the imaging position of the imaging device 41 to the object plane G1, based on the captured image captured by the imaging device 53, The final image (restored image) is reconstructed and output to the output unit 55 .

The output unit 55 applies signal processing to the final image supplied from the reconstruction unit 54 and outputs the result as an image signal.

<Imaging Processing by the Imaging Apparatuses in FIGS. 3 and 4>
Next, imaging processing by the imaging device 41 of FIGS. 3 and 4 will be described with reference to the flowchart of FIG.

In step S<b>11 , the truncated cone-shaped mirror surface 51 collects the light from the object plane G<b>1 and transmits the light to the random mask 52 .

In step S<b>12 , the random mask 52 modulates the light from the subject plane G<b>1 condensed by the mirror surface 51 and causes the light to enter the imaging element 53 .

In step S13, the image sensor 53 captures an image of light from the object plane G1, which is condensed by the truncated cone mirror surface 51 and modulated by the random mask 52. It is output to the reconstruction unit 54 as an image. That is, in this case, the image of the object plane G1 in FIG. captured as a captured image. That is, the image G2 is obtained by collecting diffused light from each point on the object plane G1 as a point light source by the truncated cone-shaped mirror surface 51 and receiving the light in a state of being diffused to various pixels of the imaging device 53. Then, by superimposing various lights, the pixel value of each pixel is smoothed, and a blurry captured image G2 is captured as a whole.

In step S14, the reconstruction unit 54 calculates the distance from the imaging position of the imaging device 41 to the object plane G1 based on the image G2, which is a captured image in which the image formed by the modulated light output from the imaging element 53 is captured. Using a predetermined coefficient set according to the distance, the image is reconstructed and output to the output unit 55 as a final image (restored image). That is, for image G2, by constructing and solving simultaneous equations using the coefficient set described with reference to equations (1) to (3) above, the final An image (restored image) is required.

In step S15, the output unit 55 performs signal processing and outputs an image signal.

That is, by using the truncated cone-shaped mirror surface 51 through the series of processes described above, it is possible to increase the light utilization efficiency without using an expensive lens. In addition, since a final image (restored image) is reconstructed using a coefficient set after modulation is applied using a random mask without using a lens, it is possible to achieve a low profile. .

Since the random mask 52 modulates the incident light, it may have another configuration as long as it is a pattern mask that can modulate the incident light. For example, it may be a diffraction grating or a diffusion plate. good too.

<<3. Modified example of the first embodiment >>
In the above description, an example in which the truncated conical mirror surface 51 is formed of a curved mirror surface has been described, but instead of the curved mirror surface, a planar mirror surface may be used.

FIG. 8 shows a modification of the imaging device 11 of the first embodiment, in which instead of the truncated cone mirror surface 51 composed of curved mirror surfaces, a truncated pyramid mirror surface 71 composed of four flat mirror surfaces is provided. An example configuration is shown.

The truncated pyramid-shaped mirror surface 71 in FIG. 8 is provided with four plane mirrors 71p to 71s, and the random mask 52 and the imaging device 53 are provided in the large opening 71a through which the incident light from the object surface G11 is incident. It is configured to be larger than the small opening 71b.

Also in the imaging device 41 of FIG. 8, it is possible to increase the light utilization efficiency according to the magnification of the area of the large opening 71a with respect to the area of the small opening 71b.

When the truncated pyramidal mirror surface 71 is used, the use of a planar mirror surface facilitates the definition by the matrix described above, so that the calculation load of the reconstruction unit 54 can be reduced.

<<4. Second Embodiment>>
In the above, an example of the configuration of an imaging apparatus configured using a mirror surface without using a lens has been described. The resulting image may be captured as a captured image, and the final image may be reconstructed based on the captured image.

9 and 10 show a configuration example of an imaging device 41 in which a lens 101 is provided in place of the random mask 52. FIG. 9 is an external perspective view of the image pickup device 41, and FIG. 10 is a side cross-sectional view of the plane mirror 71q in the truncated pyramid mirror surface 71 of the image pickup device 41 of FIG.

The imaging device 41 of FIGS. 9 and 10 includes a truncated pyramidal mirror surface 71 , a lens 101 , a prismatic light absorbing section 91 , a prismatic mirror surface 92 , an imaging device 53 , a reconstruction section 54 , and an output section 55 . . 9 and 10, components having the same functions as those of the imaging device 41 shown in FIG. 8 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

9 and 10 differs from the imaging device 41 of FIG. That is the point.

The lens 101 has the same configuration as the lens 31 in FIG. Project on the imaging plane.

The prismatic light-absorbing portion 91 is provided behind the lens 101 and is a portion composed of a prismatic light-absorbing member composed of four planar light-absorbing members 91p to 91s having the same size as the diameter of the lens 101. By absorbing the incident light by the planar light absorbing members 91p to 91s inside the prismatic shape, incident light from a range other than the field of view FOV is prevented from entering the imaging device 53 . The principle of preventing incident light from entering the imaging element 53 from a range other than the field of view FOV will be described later in detail with reference to FIG.

The prismatic mirror surface 92 is provided behind the prismatic light absorbing part 91, and the light transmitted from the lens 101 is reflected by four plane mirrors 92p to 92p provided inside the prismatic shape (cylindrical shape with a square opening cross section). The light is reflected by 92 s and made incident on the imaging surface of the imaging device 53 .

More specifically, as shown in FIG. 10, out of the diffused light from the point P on the object plane G21 as a point light source, the light indicated by the solid line is condensed by the lens 101 and projected onto the image sensor 53. It is condensed and projected at a point P'.

Among the diffused light from the point P on the object plane S as the point light source, the light indicated by the dashed line is reflected by the plane mirror 71p of the truncated pyramid mirror surface 71 and diffuses from the virtual point P1 indicated by the dotted line. It passes through the lens 101 as light. Furthermore, when the diffused light from the virtual point P1 passes through the lens 101, it is virtually condensed and projected onto a point P'' on a virtual image pickup device 53' different from the image pickup device 53. However, in reality, when the diffused light from the virtual point P1 passes through the lens 101, it is reflected by the prismatic mirror surface 92, condensed and projected onto the point P1' on the imaging device 53.

As a result, the diffused light from the point P on the object plane G21 is dispersed to a plurality of points on the imaging device 53, and is condensed by the lens 101 and projected and imaged.

That is, the condensing of light by the truncated pyramidal mirror surface 71 guides the diffused light with the point P on the object plane G21 as the point light source into the prismatic mirror surface 92 (to the lens 101) in a diffused state. is. On the other hand, the condensing by the lens 101 is condensing of diffused light with the point P on the object plane G21 as a point light source to a predetermined focal position corresponding to the focal length.

In the imaging device 41 of FIGS. 9 and 10, the diffused light with each point on the object surface G21 as a point light source is guided to the lens 101 in the state of diffused light by the mirror surface 71p of the truncated pyramid-shaped mirror surface 71. Further, while being condensed by the lens 101 to any one focal position on the imaging element 53 , the light is reflected within the prismatic mirror surface 92 and projected onto the imaging element 53 .

At this time, as shown in FIG. 10, the diffused light with each point on the object plane G21 as a point light source is directly incident on the lens 101, condensed, condensed on the imaging element 53, and projected. The pixel position in the case where the light is reflected by the truncated pyramidal mirror surface 71 , enters the lens 101 , is further reflected by the prismatic mirror surface 92 , and is projected onto the image sensor 53 is different. That is, on each pixel of the imaging element 53, light of a plurality of points on the object plane G21 is superimposed and projected.

Therefore, when the image on the subject plane G21 is, for example, (the image of) the subject plane G21 as shown in the left part of FIG. , the image G22 shown in the center of FIG. 11 is blurred as a whole. This is the same state as the state in which the incident light from the subject plane G11 is modulated by using the random mask 52 and captured as the image G12, as described with reference to FIG. That is, the lens 101 and the prismatic mirror surface 92 create a state similar to the state in which the guided incident light from the subject plane G21 is modulated, and the imaging device 53 picks up the image G22.

As a result, the reconstruction unit 54 reconstructs the image output from the image sensor 53 using the coefficient set described above, for example, converting the image G23 shown in the right part of FIG. 11 into the final image (restored image). ).

Further, as described above, each point on the object plane G21 is reflected by the truncated pyramid mirror surface 71 and dispersed into a plurality of points, and then condensed by the lens 101 and projected onto each pixel on the imaging device 53. Then, the added pixel signal is obtained. As a result, the image of the object plane G21 is captured in a smoothed state as a whole by the image sensor 53, so the image of the image G22 is a blurry image with a suppressed dynamic range.

For example, when the large opening 71a is 2.8 times as large as the small opening 71b, that is, when the amount of light received by the imaging element 53 is increased by 2.8 times as a whole, the object plane (upper image) G21 , the amount of light is about 2.8 times that when the truncated pyramid mirror surface 71 is not provided. That is, an image that is 2.8 times brighter is captured.

However, the dynamic range of the captured image G22 at this time is about 1/2.15 times the image captured by the imaging element 53 when the truncated pyramid mirror surface 71 is not provided.

This is because light from a point light source on the object plane G21 is reflected by the truncated pyramid mirror surface 71 and then condensed by the lens 101, thereby being dispersed to several points on the imaging device 53, and further This is because light from a plurality of points is added, smoothing is performed in all pixels of the image pickup device 53, thereby averaging detection signal levels as a whole and reducing the dynamic range.

However, when the image G23, which is the final image, is restored using the image G22 by the coefficient set described with reference to equations (1) to (3), dynamic Restored to range.

As a result, the dynamic range of the image picked up by the imaging device 53 is reduced, and the exposure time is lengthened, so that the dynamic range of the reconstructed final image is made larger than when the truncated pyramid mirror surface 71 is not provided. becomes possible. Moreover, even if the dynamic range of the imaging device 53 is small, it is possible to reconstruct an image with a dynamic range equivalent to that obtained when the imaging device 53 having a large dynamic range is used.

The total length of the prismatic light absorbing portion 91 and the prismatic mirror surface 92 (horizontal length in FIG. 10) corresponds to the focal length of the lens 101 . For this reason, the reconstruction unit 54 uses a coefficient set corresponding to the focal length of the lens 101 (a coefficient set corresponding to the distance to the object plane S21 when the lens 101 is used in the imaging device 41) to convert the image G22 into An image G23, which is the final image (restored image), is reconstructed.

<About the action of the prismatic light absorbing part>
Next, with reference to FIG. 12, the operation of the prismatic light absorbing portion 91 will be described.

The imaging device 41 in FIGS. 9 and 10 captures an image of the subject plane G21 within the field of view FOV. For example, as shown in FIG. By being reflected by the truncated pyramid mirror surface 71, the light is transmitted through the lens 101 as diffused light with the virtual point S1 as the light source.

Here, when the prismatic light absorbing portion 91 has a mirror surface structure, the light transmitted through the lens 101 as diffused light with the virtual point S1 as the light source is repeatedly reflected within the prismatic mirror surface 92, and originally, the image sensor 53 is received as light that should not be received at.

As shown in FIG. 12, the prismatic light absorbing portion 91 prevents the light from entering the prismatic mirror surface 92 by absorbing the light transmitted through the lens 101 and having the virtual point S1 as the light source.

For this reason, incident light from outside the field of view FOV is absorbed by the prismatic light absorbing portion 91 and therefore does not enter the prismatic mirror surface 92 and is not received by the imaging device 53 .

As a result, the light outside the field of view FOV is not received by the imaging element 53 due to the prismatic light absorbing portion 91. Therefore, the utilization efficiency of the light from the object plane G21 within the field of view FOV is increased, and unnecessary light does not enter. can be prevented.

The length of the prismatic light-absorbing portion 91 in the horizontal direction in FIG. is set to be The prismatic light absorbing portion 91 may be made of any material that does not reflect light onto the prismatic mirror surface 92 , and may be made of a light shielding film or the like, or may transmit light so as not to enter the prismatic mirror surface 92 . Any material such as

<Regarding the aperture angle of the truncated pyramid-shaped mirror surface>
Next, the aperture angle of the truncated pyramid mirror surface 71 will be described.

Here, for example, as shown in FIG. 13, the angle θ11 formed by the opposing mirror surfaces in the cross section of the truncated pyramidal mirror surface 71 is defined as the aperture angle. Further, in FIG. 10, when diffused light having a point light source at a virtual point P1 is transmitted through the lens 101 and reflected by the prismatic mirror surface 92 to enter a point P1' on the imaging element 53, the point P1' A virtual opening 71b′ is defined as a virtual opening corresponding to the small opening 71b when it is assumed that the light is directly incident on the light beam.

As described above, the amount of light that can be collected can be increased by adjusting the area ratio of the large opening 71a and the small opening 71b.

In other words, as shown in FIG. 10, by increasing the area of the virtual opening 71b' corresponding to the small opening 71b, it can be considered that more diffused light can be collected.

For example, as shown in FIG. 13, when the aperture angle of the plane mirrors 71p and 71r is the aperture angle θ11, the actual small aperture 71b and the virtual apertures 71b′-1 and 71b′-2 produce the The truncated pyramidal mirror surface 71 can converge the light in the range of the angle θ1 among the diffused light with the point Px as the point light source.

On the other hand, as shown in FIG. 14, when the plane mirrors 71p and 71r have an aperture angle of θ11 (<θ12), the actual aperture 71b and the virtual apertures 71b′-11 to 71b′-16 give: The plane mirrors 71p and 71r in FIG. 14 can converge the light in the range of the angle θ2 (>θ1) among the diffused light with the point Px as the point light source.

That is, even if the aperture angle is small, more light can be condensed by increasing the virtual aperture by securing the length with respect to the incident direction of the incident light on the truncated pyramidal mirror surface 71. can be considered. In other words, even if the aperture angle is small, it is possible to increase the ratio between the large aperture portion 71a and the small aperture portion 71b by securing the length of the truncated pyramid mirror surface 71 with respect to the incident direction of the incident light. You can think that you can.

<Imaging Processing by the Imaging Apparatus of FIGS. 9 and 10>
Next, imaging processing by the imaging device 41 of FIGS. 9 and 10 will be described with reference to the flowchart of FIG.

In step S<b>31 , the truncated pyramid mirror surface 71 collects the light from the object surface G<b>21 and transmits the light to the lens 101 .

In step S<b>32 , the lens 101 condenses the light from the subject plane G<b>21 condensed by the truncated pyramidal mirror surface 71 and causes it to enter the prismatic mirror surface 92 . At this time, incident light from outside the field of view FOV is absorbed by the prismatic light absorbing portion 91 .

In step S<b>33 , the square mirror surface 92 reflects the light from the object surface G<b>21 condensed by the lens 101 and causes the light to enter the imaging device 53 .

In step S<b>34 , the imaging element 53 captures an image of light from the object plane G<b>21 and condensed by the truncated pyramid mirror surface 71 and the lens 101 , and outputs the image to the reconstruction unit 54 . That is, in this case, the image of the subject plane G21 in FIG. 10 is condensed by the truncated pyramid mirror surface 71 and the lens 101, and the image G22 is captured by the imaging device 53, for example. The image G22 is received in a state in which the light condensed by the truncated pyramidal mirror surface 71 and the lens 101 is diffused in various pixels of the imaging element 53, and furthermore, by superimposing various lights on a pixel-by-pixel basis, The pixel value of each pixel is smoothed, and an overall blurry image (an image with a reduced dynamic range) is captured.

In step S35, the reconstruction unit 54 reconstructs an image using a predetermined coefficient set based on the pixel signals obtained by imaging the image composed of the condensed light output from the image sensor 53, and finalizes the image. It is output to the output unit 55 as an image (restored image). That is, by constructing and solving simultaneous equations using a coefficient set for the image G22, for example, a final image (restored image) as shown by the image G23 is obtained.

In step S36, the output unit 55 performs signal processing and outputs an image signal.

That is, by using the truncated pyramidal mirror surface 71 and the lens 101 through the series of processes described above, it is possible to capture a bright image without using an expensive large-aperture lens. Further, by using the lens 101, the image G22 captured by the image sensor 53 is closer to the final image (restored image) G23 reconstructed using the coefficient set than when the lens 101 is not used. It becomes possible to pick up as an image. In other words, by using the lens 101, it is possible to capture the image G22 as an image having a higher sparsity and being easily reconstructed into the final image G23.

<<5. Modified example of the second embodiment >>
For the truncated pyramidal mirror surface, two types of openings, large and small corresponding to the large opening and the small opening connected to the prismatic mirror surface, are provided, and the incident light from the subject surface is guided to the cylindrical mirror surface. As long as the mirror surface is provided on the inner side, the mirror surface may have another shape, for example, a truncated cone-shaped mirror surface 51 as shown in FIG. 16 may be used. However, in the case of the truncated cone-shaped mirror surface 51, it is necessary to provide a cylindrical light absorbing portion 121 and a cylindrical mirror surface 122 having a corresponding shape (cylindrical shape with a circular opening cross section). In addition, as long as two types of openings, large and small, are provided for guiding incident light, the shapes of the large and small openings may be asymmetrical or similar. It may be nothing.

In this specification, a system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device housing a plurality of modules in one housing, are both systems. .

Further, the embodiments of the present disclosure are not limited to the embodiments described above, and various modifications are possible without departing from the gist of the present disclosure.

Furthermore, each step described in the flowchart above can be executed by one device, or can be shared by a plurality of devices.

Further, when one step includes a plurality of processes, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.

It should be noted that the present disclosure can also take the following configuration.

<1> a guide unit that guides incident light from a subject to an imaging device;
an imaging unit that captures the incident light guided by the guiding unit as a pixel signal;
and a signal processing unit that reconstructs the pixel signals into a final image by signal processing.
<2> The guiding portion has a truncated cone shape provided with a large opening and a small opening, the inner side of which is a mirror surface, and the incident light incident from the large opening is directed to the small opening. The image pickup apparatus according to <1>, in which the image pickup device is guided to the image pickup element.
<3> The imaging device according to <2>, wherein the guide section has a truncated cone shape with the large opening and the small opening, and the inner side is a curved mirror surface.
<4> The imaging device according to <2>, wherein the guide section has a truncated pyramid shape in which the large opening and the small opening are provided, and the inner side is a planar mirror surface.
<5> The imaging element is sized to be included in the small opening,
The guiding section guides the incident light from the subject, so that the light amount of the incident light in the imaging section is reduced by the magnification of the area of the large opening when the area of the small opening is used as a reference. The imaging device according to <2>, which is increased.
<6> further comprising a modulating unit that modulates the incident light before the imaging element;
The imaging device according to any one of <1> to <5>, wherein the imaging section captures the incident light guided by the guiding section and modulated by the modulating section as the pixel signal.
<7> The imaging device according to <6>, wherein the modulation unit is a random mask or a diffraction grating.
<8> The modulation unit
a lens for condensing the incident light guided by the guiding part;
The imaging apparatus according to <6>, further comprising a cylindrical mirror surface that reflects the incident light condensed by the lens and guides it to the imaging unit.
<9> The modulation unit
The imaging apparatus according to <8>, further comprising a cylindrical mirror surface that reflects the incident light condensed by the lens and guides it to the imaging unit.
<10> The imaging device according to <9>, wherein the opening cross-section of the cylindrical mirror surface is circular.
<11> The imaging device according to <9>, wherein the opening cross section of the cylindrical mirror surface is square.
<12> The image pickup apparatus according to <9>, further including a cylindrical light absorbing portion that absorbs the incident light, downstream of the lens and upstream of the cylindrical mirror surface.
<13> The imaging device according to <12>, wherein the cylindrical light absorbing portion is provided at a position where incident light outside the field of view of the lens enters.
<14> The imaging device according to <13>, wherein the sum of the cylindrical length of the cylindrical light absorbing portion and the cylindrical length of the cylindrical mirror surface is the focal length of the lens.
<15> Guidance processing for guiding incident light from a subject to an imaging device;
Imaging processing for imaging the incident light guided by the guiding processing as a pixel signal;
and signal processing for reconstructing the pixel signals into a final image by signal processing.

Reference Signs List 11 imaging device, 31 lens, 32 imaging device, 33 output unit, 41 bandpass filter, 51 truncated cone mirror surface, 51a large aperture, 51b small aperture, 52 random mask, 53 imaging device, 54 reconstruction unit, 55 output section 71 truncated pyramid mirror surface 71a large opening 71b small opening 71p to 71s plane mirror 71b', 71b'-1, 71b'-2, 71b'-11 to 71b'-16 virtual opening, 91 Prismatic light absorbing part 92 Prismatic mirror surface 101 Lens 121 Cylindrical light absorbing part 122 Cylindrical mirror surface

Claims (15)

a guide section that guides incident light from a subject to an imaging element;
a modulating section that modulates incident light guided by the guiding section;
an imaging unit for imaging the incident light modulated by the modulating unit as a pixel signal by the imaging device ;
The pixel signals are reconstructed as a final image by signal processing using a coefficient set that is set according to the incident angle of the incident light to the imaging element modulated by the modulating unit for each distance to the subject. and a signal processing unit. The guide section has a frustum shape provided with a large opening and a small opening, and has a mirror surface inside, and the incident light entering through the large opening is captured through the small opening. The imaging device according to claim 1, wherein the element is guided. 3. The imaging device according to claim 2, wherein the guiding portion has a truncated cone shape with the large opening and the small opening, and the inner side is a curved mirror surface. 3. The imaging device according to claim 2, wherein the guide section has a truncated pyramid shape in which the large opening and the small opening are provided, and the inner side is a flat mirror surface. The imaging element is sized to be included in the small opening,
The guiding section guides the incident light from the subject, so that the light amount of the incident light in the imaging section is reduced by the magnification of the area of the large opening when the area of the small opening is used as a reference. The imaging device according to claim 2, which is increased. The imaging device according to claim 1 , wherein the modulation section is a pattern mask, a diffraction grating, or a diffusion plate. The modulation unit
The imaging device according to claim 1 , further comprising a lens that collects the incident light guided by the guiding section. The modulation unit
8. The imaging apparatus according to claim 7 , further comprising a cylindrical mirror surface that reflects incident light condensed by said lens and guides it to said imaging unit. The imaging device according to claim 8 , wherein the opening cross section of the cylindrical mirror surface is circular. The imaging device according to claim 8 , wherein the opening cross section of the cylindrical mirror surface is square. 9. The image pickup apparatus according to claim 8 , further comprising a cylindrical light absorbing portion that absorbs the incident light after the lens and before the cylindrical mirror surface. 12. The imaging device according to claim 11 , wherein the cylindrical light absorbing portion is provided at a position where incident light outside the field of view of the lens is incident. 13. The imaging device according to claim 12 , wherein the sum of the cylinder length of the cylindrical light absorbing portion and the cylinder length of the cylindrical mirror surface is the focal length of the lens. Guidance processing for guiding incident light from a subject to an imaging device;
a modulation process that modulates incident light induced by the induction process;
an imaging process of imaging the incident light modulated by the modulation process with the imaging element as a pixel signal;
The pixel signals are reconstructed as a final image by signal processing using a coefficient set that is set according to the incident angle of the incident light modulated by the modulation processing to the imaging device for each distance to the subject. and an imaging method comprising: a guide section that guides incident light from a subject to an imaging element;
a modulating section that modulates incident light guided by the guiding section;
an imaging unit that captures the incident light modulated by the modulating unit as a pixel signal;
a signal processing unit that reconstructs the pixel signals as a final image by signal processing;
The modulation unit
a lens for condensing the incident light guided by the guiding part;
and a cylindrical mirror surface that reflects the incident light condensed by the lens and guides it to the imaging unit.
Imaging device.

Images