JP2017022491A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which easily suppresses a lack of image information, for example, a halation and black saturation.SOLUTION: An imaging apparatus 100 comprises: imaging lenses 111, 112; an imaging element 120 which receives a beam L1 having passed through the imaging lenses 111, 112; a spatial modulation optical element 130 which comprises plural modulation surfaces 132a arranged flatly in a space allowing the beam L1 to pass therethrough and is disposed between the imaging lenses 111, 112 and the imaging element 120; and a modulation control part which individually controls the permeability D of the modulation surfaces 132 on the basis of intensity information I of a beam L2 received by the imaging element 120.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus using a spatial light modulation element.

  In an imaging apparatus represented by a digital camera, the range of sensitivity of an imaging element used is narrower than the range of sensitivity of human vision, and therefore, black areas in dark places and white areas in bright areas are likely to occur. For this reason, it is known that when there is an extremely bright place on the same screen, the lack of image information such as other parts being crushed darkly occurs.

  In order to avoid such a lack of image information, a high dynamic range (HDR) method is known in which a plurality of photographs having different exposure values are taken and the missing image information is restored by image synthesis (for example, patents). Reference 1 etc.).

  However, in this method, it is necessary to shoot the same scene with a plurality of exposure values and further synthesize an image, and particularly those that move at a high speed or high-precision images that require a high frame rate. Shooting was difficult.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging apparatus that can easily suppress lack of image information such as overexposure or blackout.

  In order to solve the above-described problems, an imaging apparatus according to the present invention includes an imaging lens, an imaging element that receives a light beam that has passed through the imaging lens, and a plurality of planarly arranged spaces in which the light beam is transmitted. A modulation surface, a spatial modulation optical element disposed between the imaging lens and the imaging device, and the transmittance of the plurality of modulation surfaces based on intensity information of the light beam received by the imaging device. And a modulation control unit for individually controlling.

  According to the imaging apparatus of the present invention, lack of image information such as overexposure or blackout is easily suppressed.

1 is a schematic diagram illustrating an example of an overall configuration of an imaging apparatus according to an embodiment of the present invention. It is the top view and sectional drawing which show an example of a structure of the spatial modulation optical element and imaging device in embodiment of this invention. It is a perspective view showing an example of composition of an imaging device in an embodiment of the present invention. It is a flowchart which shows an example of the imaging method in embodiment of this invention. It is a conceptual diagram which shows an example of the image obtained with an image pick-up element. It is a conceptual diagram which shows an example of the relationship between the image obtained in embodiment of this invention, and the opening pattern of a spatial modulation optical element. It is a conceptual diagram which shows an example of the opening pattern of the transmittance | permeability of the spatial modulation optical element in the case of imaging | photography the image of FIG.

The imaging apparatus 100 shown in FIG. 1 as an example of the present embodiment acquires a spatial modulation optical element 130 that applies spatial modulation to light from the imaging target 102 that is a scene, and image information about the imaging target 102. And an image sensor 120 for the purpose.
The imaging apparatus 100 includes lenses 111 and 112 as imaging lenses constituting an imaging optical system 110 that emits incident light as a light beam L1 toward the image sensor 120, and an aperture stop 115 that controls the total amount of the incident light beam L1. And have.
The imaging apparatus 100 includes a spacer 140 that is a fixing unit for fixing and supporting the spatial modulation optical element 130, and a control unit 190 that is a control unit for controlling the operation of each unit in the imaging apparatus 100. Yes.

As shown in FIG. 2A, the spatial modulation optical element 130 is a plurality of two-dimensionally arranged so as to divide an irradiation area where the light beam L <b> 1 from the imaging target 102 is irradiated to the spatial modulation optical element 130. A cell 132 is provided.
The spatial modulation optical element 130 also has a support beam 133 formed around each cell 132.
The spatial modulation optical element 130 is a transmissive liquid crystal device, and is a code that forms a coded aperture pattern, in other words, a grayscale pattern that is a mosaic pattern, by independently changing the transmittance at the position of each cell 132. It is an optical element having a function as an aperture element.

As shown in FIG. 2B, the cell 132 has a modulation surface 132a arranged in a plane in a space through which the light beam L1 passes, and is independently formed by a modulation control unit 191 described later along FIG. Then, the intensity of the transmitted light L2 is changed by changing the transmittance.
The cell 132 has a square shape of 10 μm square here, but is not limited to this shape and size. It is desirable that the modulation surface 132a has a sufficiently small influence on information other than the intensity of the light beam L1, that is, on the direction, wavelength, and the like.

The support beam portion 133 is a beam-like support portion formed around the cells 132 and is a portion that does not contribute to the modulation of the transmittance that is the peripheral portion located between the cells 132. In order to suppress the influence of the support beam portion 133 itself as a shadow on the intensity information of the transmitted light L2, the support beam portion 133 is configured so that the shadow of the support beam portion 133 itself does not appear on the image sensor 120. Provided.
Specifically, the distance Z1 between the spatial light modulator 130 and the image sensor 120 is separated to such an extent that the shadow of the support beam 133 itself does not appear on the image sensor 120.
The distance in which the shadow of the support beam 133 is reflected also varies depending on the width Z2 of the support beam 133, in other words, the interval between the adjacent cells 132. Therefore, the distance Z1 between the spatial light modulation element 130 and the imaging element 120 may be appropriately determined according to the numerical value of the width Z2 that is the interval between the plurality of adjacent cells 132.
The spatial modulation optical element 130 is a transmission type liquid crystal device here, but may be a wet capillary type spatial modulation optical element, or may be a liquid crystal element combined with a polarizing plate, an electrochromic element, or a MEMS micro-device. A shutter array may be used.

The imaging element 120 is an image sensor using a CCD (Charge Coupled Device) as a photodetector array that acquires the incident light quantity I of the transmitted light L2 that has passed through the spatial modulation optical element 130 as intensity information.
The image sensor 120 is provided with a photodetector array in which photodiodes serving as light receiving elements, which are a plurality of light detectors 121, are arranged two-dimensionally, and an imaging surface 122 serving as a light receiving surface is provided on the surface of the light receiving element. Is formed.
The imaging element 120 and the spatial modulation optical element 130 are provided so that the imaging surface 122 and the modulation surface 132a face each other, in other words, close to each other, separated by a predetermined distance Z1 by the spacer 140. .
The distance Z1 between the image sensor 120 and the spatial modulation optical element 130 is provided such that all the transmitted light L2 is received by the opposite imaging surface 122, and when a 36 mm × 24 mm full size image sensor 120 is used, 1 mm or less is desirable (confirmation is required from another application).

The photodetector 121 converts the intensity information of the transmitted light L2 incident on the imaging surface 122, that is, the light amount I that is the amount of the transmitted light L2 into an electrical signal. Although a CCD is used here, a CMOS (Complementary Metal-Oxide Semiconductor) sensor or the like may be used as long as it is an image sensor that can acquire intensity information.
The imaging surface 122 is a virtual surface that is two-dimensionally arranged in a two-dimensional manner with a unit in which the transmitted light L2 transmitted through one cell 132 is incident at a position facing the cell 132. .
In other words, the imaging surface 122 is provided as a pair with each of the cells 132, and has a function as a light receiving surface that receives the transmitted light L2 transmitted through the corresponding cell 132.
Further, in the present embodiment, the imaging surface 122 has an area equivalent to the area of the cell 132 because the imaging element 120 and the spatial modulation optical element 130 are provided close to each other at a distance Z1.

As shown in FIG. 3, the spacer 140 is a fixing member that is disposed so as to surround the imaging device 120 and is attached to a housing portion of the imaging device 100 that fixes the imaging device 120.
The spacer 140 adjusts the distance Z <b> 1 according to the height of the spacer 140 by bonding and fixing the spatial modulation optical element 130 to the housing portion of the imaging device 100.
Here, the spacer 140 is fixed to the housing portion of the imaging device 100. However, the spacer 140 is disposed between the spatial modulation optical element 130 and the imaging element 120, and the spatial modulation optical element 130 and the imaging element 120 are bonded and fixed to each other. You may do.

As shown in FIG. 1, the control unit 190 forms an image based on the modulation control unit 191 that controls the transmittance of each cell 132 of the spatial modulation optical element 130 and the intensity information of the transmitted light L2 acquired by the image sensor 120. And an image generation unit 192 for reconstruction.
When the modulation control unit 191 sets the transmittance of the cell 132, the control unit 190 includes a correction coefficient calculation unit 193 that is a correction coefficient calculation unit that calculates the correction coefficient E of the light receiving sensitivity of the imaging surface 122 corresponding to the transmittance. Have.
The control unit 190 corrects the output image using the light amount value calculating unit 194 serving as a light amount value calculating unit for calculating the light amount I of the transmitted light L2 incident on the imaging surface 122, the correction coefficient E, and the light amount I. And a correction unit 195 as correction means.
The control unit 190 is a control unit realized by a microprocessor built in the imaging apparatus 100 or a program operating on the microprocessor.

The modulation control unit 191 independently sets the transmittance in each cell 132 of the spatial modulation optical element 130 based on the electrical signal based on the distribution information of the light quantity I from the image sensor 120.
The image generation unit 192 combines image information from the light amount I of the transmitted light L2 input from the image sensor 120 and generates an image to be output from the intensity distribution of the light amount I.

An imaging method until an image is obtained using the imaging apparatus 100 having such a configuration will be described with reference to FIG.
First, when shooting is started, the light incident on the imaging device 100 is deflected by the imaging optical system 110 so as to form an image on the imaging element 120 and passes through the aperture stop 115 to be a luminous flux. The total amount of L1 is adjusted.
In such an initial state, the spatial modulation optical element 130 is set to have no aperture pattern, that is, all of the cells 132 are in a transmissive state, that is, the maximum transmittance (step S201).

In the image sensor 120, a 2D image is formed as intensity information of the transmitted light L2 incident on the corresponding cell 132, as in a normal digital camera, and the image generation unit 192 acquires the intensity information as image data. Convert and save (step S202).
Such processing is an image acquisition step in which the image sensor 120 acquires image information from light transmitted through the spatial modulation optical element 130.

  The light amount value calculation unit 194 calculates the light amount I of the transmitted light L2 in the image sensor 120 for each imaging surface 122, in other words, for each pixel, and acquires the brightness on the imaging surface 122 (step S203).

By the way, when the image obtained by imaging the light from the imaging target 102 is captured under ideal exposure conditions, the overall brightness of the imaging element is exemplified as shown in FIG. Images are taken so that they are within the sensitivity range of 120.
However, the imaging target 102 includes a plurality of subjects with various conditions. For example, the amount of incident light varies greatly depending on conditions such as reflectance and color, and the image cannot be captured under ideal exposure conditions. ) As shown in FIG.
Under such exposure conditions, the amount of light I incident on the imaging surface 122 exceeds the upper limit of sensitivity of the photodetector 121 (out-of-white), or the amount of light I falls below the lower limit of sensitivity of the photodetector 121. It is considered to be in a state (black crushing).

In a conventional imaging apparatus, a method of suppressing the occurrence of overexposure or blackout by adjusting exposure conditions, i.e., the sensitivity of an aperture or an image sensor, or an HDR method that synthesizes images obtained by shooting under a plurality of exposure conditions. Etc. are known.
However, at present, it has not been possible to solve problems such as shooting of a subject that moves at high speed, which causes limitations on blurring and processing time depending on conditions.
Also, if the total light amount is reduced to match the part with a large amount of light, the black part will be crushed in the part where the light amount is originally low, and the observation target cannot be identified particularly with a surveillance camera. May occur.

  In the present embodiment, in order to solve the problem that the image information is missing depending on the exposure condition as described above, the following operation is performed based on the brightness information acquired in step S203.

In order to simplify the description, as shown in FIG. 6, the sections a1 to h8 when the entire imaging region of the imaging target 102 projected onto the imaging surface 122 is divided into 64 × 8 × 8 will be described. Note that the division into 64 is only a large division for simplicity of explanation, and in the actual imaging apparatus 100, the imaging surface 122 may arbitrarily determine the number of divisions with each pixel as a minimum unit. .
In FIG. 6, in particular, sections b3, b4, c3, c4, e5, e6, f5, and f6 of the imaging surface 122 are whiteout portions, in other words, portions where image information is missing.
As already described, the amount of light I incident on the imaging surface 122 corresponding to the sections b3, b4, c3, c4, e5, e6, f5, and f6 is larger than the predetermined value I _Lim. . Specifically, it is considered that the light amount I is in a saturated state in which the upper limit value of the sensitivity of the photodetector 121 is exceeded.

The light quantity value calculation unit 194 measures the magnitude of the light quantity I incident on each imaging surface 122 and compares it with the upper limit value of the sensitivity of the photodetector 121, so that the photodetector 121 corresponding to the imaging surface 122 can be detected. It is determined whether or not it is saturated (step S204). This step S204 is a saturation determination step for determining the lack of image information on each imaging surface 122.
Note that the predetermined value I _{Lim to be} compared in the saturation determination step may be set to a predetermined value lower than the upper limit value of the sensitivity of the photodetector 121.

  If a portion lacking image information is found in step S204, a lower limit determination step (step S205) is performed to determine whether the current transmittance D is a lower limit value that can be set in the cell 132, that is, a minimum value.

When it is determined in step S205 that the transmittance D is not the minimum value, the modulation control unit 191 determines the transmittance D in the cell 132 at the position corresponding to the imaging surface 122 where the saturation state is determined as the transmittance D. '= D-ΔD is lowered (step S206).
Specifically, the transmittance D of the cell 132 at the position corresponding to the sections b3, b4, c3, c4, e5, e6, f5, f6 is lowered.
In the present embodiment, the transmittance decrease value ΔD is a transmittance obtained by dividing the difference between the maximum transmittance and the minimum transmittance of the spatial modulation optical element 130 into 64 equal parts, but the transmittance that can be set by the spatial modulation optical element 130 An arbitrary value may be set according to the number of gradations.
Step S206 is a transmittance setting step for setting the transmittance D of the cell 132.

The imaging apparatus 100 returns to step S202 again according to the new transmittance D ′ pattern set in step S206, and acquires intensity information of the light amount I incident on the imaging element 120 as image information.
When Steps S202 to S206 are repeated until there is no more compartment where the saturation state is determined, a maximum of 64 floors is formed on the cell 132 of the spatial modulation optical element 130 as schematically shown in FIG. A modulation pattern which is a tone transmittance distribution is formed.

Since the modulation pattern is known, the value of the amount of light I incident on the imaging surface 122 of the transmitted light L2 that has passed through the modulation pattern is divided by the correction coefficient E calculated based on the transmittance D of the cell 132 at the corresponding position. Thus, the maximum light amount I _max in the initial state is obtained.
Specifically, the correction coefficient E = D _b3 / D when the light amount I _b3 incident on the section b3 of the imaging surface 122, the transmittance D _b3 of the section b3 of the cell 132, and the maximum transmittance D _max of the cell 132 are set. _max , maximum light intensity I _max = I _b3 / E.
The correction coefficient calculation unit 193 calculates a correction coefficient E for correcting each light quantity I in each section of the imaging surface 122 based on the modulation pattern of the transmittance D (step S207).

In step S205, there may be a case where there is still no compartment in which the saturated state is determined even if the transmittance D is reduced to the lower limit value that can be set in the cell 132. In such a case, it is desirable to further reduce the aperture stop 115 (step S208) and repeat steps S201 to S206 while suppressing the total amount of the light beam L1. When such a light quantity suppression step is passed, it is desirable to consider the light quantity suppressed by the aperture stop 115 as the correction coefficient E in step S207.
However, step S205 and step S208 may be omitted, and in this case, when a saturated state is determined in step S204, the process may transition to step S206 to adjust the transmittance to D ′. .

  When there is no imaging surface 122 that is saturated in step S204 at the stage where such a modulation pattern is formed, the image generation unit 192 reconstructs an image from the image information acquired in step S202 (step S209). ).

The correction unit 195 corrects the brightness of the entire image by correcting the image reconstructed in step S208 using the correction coefficient E and the light amount I (step S210).
In this way, by performing correction by the correction unit 195, an image with a wide dynamic range in which whiteout and blackout are reduced can be obtained.

  When the image generation unit 192 generates the output image again (step S211), the imaging apparatus 100 ends the shooting.

In the present embodiment, the imaging apparatus 100 includes a modulation control unit 191 that individually controls the transmittance D of the cell 132 based on the intensity information of the transmitted light L2 received by the imaging element 120.
With this configuration, the imaging apparatus 100 suppresses lack of image information such as whiteout or blackout with a simple structure.

In the present embodiment, the spatial modulation optical element 130 is provided in close proximity to the imaging element 120.
With this configuration, the transmitted light L2 that has passed through the cell 132 is incident on the imaging surface 122 of the opposing imaging device 120, so that the imaging surface 122 does not detect excess light. Therefore, the imaging apparatus 100 suppresses lack of image information such as whiteout or blackout with a simple structure.

In the present embodiment, the image sensor 120 is provided at a position facing each of the plurality of cells 132, and includes a plurality of light receiving surfaces 122 having an area equivalent to that of the cells 132.
In addition, a light amount value calculating unit 194 that calculates the light amount I received by the image sensor 120 for each light receiving surface 122 is provided, and the modulation control unit 191 is based on the light amount I and the cell 132 facing the light receiving surface 122. The transmittance D is controlled.
With this configuration, the transmitted light L2 that has passed through the cell 132 is incident only on the imaging surface 122 of the opposing imaging device 120, so that the imaging surface 122 does not detect excess light. Therefore, the imaging apparatus 100 suppresses lack of image information such as whiteout or blackout with a simple structure.

In the present embodiment, the modulation control unit 191 reduces the transmittance D of the cell 132 facing the imaging surface 122 when the light amount I is larger than a predetermined value.
With such a configuration, it is possible to reduce the amount of light only at a position where the amount of light I is greater than a predetermined value. Therefore, since whiteout is suppressed at a pinpoint, the occurrence of black crushing is suppressed without reducing the overall amount of light. While suppressing whiteout.

In the present embodiment, the imaging apparatus 100 includes the correction coefficient calculation unit 193 that calculates the correction coefficient E according to the transmittance D, the correction coefficient E, and the light amount I when the modulation control unit 191 sets the transmittance D. And a correction unit 195 that corrects an image by using the correction unit.
With this configuration, correction is performed by the correction unit 195 to calculate the maximum amount of light I _max that should originally be incident at the overexposed position and perform imaging with a wide dynamic range of the image sensor 120.
In the present embodiment, the distance Z1 between the image sensor 120 and the spatial modulation optical element 130 is set so that the shadow of the support beam 133 located between the adjacent modulation surfaces 132a is not reflected on the image sensor 120. Be released.
With this configuration, since the shadow of the support beam 133 does not appear in the image sensor 120, a decrease in intensity information of the light beam incident on the image sensor 120 is suppressed.

In the present embodiment, the aperture stop 115 is disposed downstream of the lenses 111 and 112 in the optical axis direction of the light beam L1 and upstream of the spatial modulation optical element 130.
With such a configuration, even if the transmittance D is reduced to the lower limit value that can be set in the cell 132, even if the section where the saturated state is still determined does not disappear, image information such as overexposure or blackout may be displayed with a simple structure. Suppress the lack.

  These imaging methods are realized by a program recorded in the control unit 190, and are executed by calculation means such as a CPU. Such a program may be configured to be recorded and provided in a computer-readable recording medium such as a CD-ROM, FD, CD-R, or DVD in an installable or executable format file.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the specific embodiments, and the present invention described in the claims is not specifically limited by the above description. Various modifications and changes are possible within the scope of the above.

  For example, in the above-described embodiment, the imaging apparatus 100 may be an imaging apparatus of various forms such as a compact digital camera or a small camera mounted on a portable device.

  In the above-described embodiment, only the light amount is used as the intensity information. However, it is also possible to obtain the intensity information for each color by attaching a filter of each basic color to the photodetector.

  The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

DESCRIPTION OF SYMBOLS 100 Imaging device 102 Imaging target 110 Imaging optical system 111, 112 Imaging lens (lens)
115 Aperture (aperture stop)
120 Image sensor 121 Photo detector 122 Image pickup surface (light receiving surface)
130 Spatial Modulation Optical Element 132 Cell 132a Modulation Surface 140 Spacer 190 Control Unit 191 Modulation Control Unit 192 Image Generation Unit 193 Correction Coefficient Calculation Means (Correction Coefficient Calculation Unit)
194 Light quantity value calculating means (light quantity value calculating section)
195 Correction means (correction unit)
D Transmittance E Correction factor I Amount of luminous flux (light quantity)
Z1 Distance between the imaging element and the spatial modulation optical element Z2 Spacing between a plurality of modulation surfaces

JP 2005-173493 A

Claims (7)

An imaging lens;
An image sensor for receiving a light beam transmitted through the imaging lens;
A plurality of modulation surfaces arranged in a plane in a space through which the light beam passes, and a spatial modulation optical element disposed between the imaging lens and the imaging element;
An image pickup apparatus comprising: a modulation control unit that individually controls transmittances of the plurality of modulation surfaces based on intensity information of the light flux received by the image pickup element. The imaging device according to claim 1,
The image pickup apparatus, wherein the spatial modulation optical element is provided in close proximity to the image pickup element. The imaging device according to claim 1 or 2,
The imaging device is provided at a position facing each of the plurality of modulation surfaces, and includes a plurality of light receiving surfaces having an area equivalent to each of the modulation surfaces,
A light amount value calculating means for calculating the amount of the light flux received by the image sensor for each of the light receiving surfaces;
The imaging apparatus according to claim 1, wherein the modulation control unit controls the transmittance of the modulation surface facing the light receiving surface based on the amount of the light flux. The imaging device according to claim 3.
The imaging apparatus, wherein the modulation control unit lowers the transmittance of the modulation surface facing the light receiving surface when the amount of the light beam is larger than a predetermined value. In the imaging device according to claim 3 or 4,
Correction coefficient calculating means for calculating a correction coefficient corresponding to the transmittance when the modulation control unit lowers the transmittance;
An image pickup apparatus comprising: a correction unit configured to correct an image formed on the image pickup device using the correction coefficient and the amount of the light flux. The imaging apparatus according to any one of claims 1 to 5,
The distance between the imaging element and the spatial modulation optical element is separated so that a shadow of a peripheral edge located between the plurality of adjacent modulation surfaces is not reflected on the imaging element. apparatus. The imaging device according to any one of claims 1 to 6,
An imaging apparatus comprising: a stop portion disposed downstream of the imaging lens in the optical axis direction of the light beam and upstream of the spatial modulation optical element.

Images