JP2008167154A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus easily suppressing generation of cross talk in using an array-like optical element for imaging. <P>SOLUTION: A liquid crystal shutter array 10 including a plurality of liquid crystal shutters arranged corresponding to each micro lens 12A is arranged at one side (between the micro lens array 12 and an imaging lens 11) of the micro lens array 12. In addition, passage and blocking of an imaging beam traveling toward an image pickup device 13 are switched by each micro lens 12A by the liquid crystal shutter array 10. Overlap of beams on the image pickup device 13 is suppressed by image forming light beams by each micro lens 12A. An aperture array 18 having a plurality of apertures 18A may be positioned instead of liquid crystal shutter array 10. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus using a microlens array.

  Conventionally, various imaging devices have been proposed and developed. There has also been proposed an imaging apparatus that performs predetermined image processing on imaging data obtained by imaging and outputs the data.

  For example, Patent Document 1 and Non-Patent Document 1 propose an imaging apparatus using a technique called “Light Field Photography”. For example, like the imaging device 100 shown in FIG. 19, an imaging lens 111 that is a main lens for imaging the imaging target 131, a microlens array 112 in which a plurality of microlenses 112A are arrayed, The imaging element 113 and an image processing unit (not shown) are configured so that imaging data obtained from the imaging element 113 includes information on the light traveling direction in addition to the light intensity distribution on the light receiving surface. It has become. In the image processing unit, an observation image from an arbitrary viewpoint or direction can be reconstructed.

  Further, for example, Patent Document 2 and Non-Patent Document 2 disclose a technique called “Compound Eye” using an ultra-thin compound eye optical system as another example of an imaging device using a microlens array or the like and image processing. An imaging device has been proposed. This is configured by a microlens array 222 having a plurality of microlenses 222A, an image sensor 223, and an image processing unit (not shown), for example, as in the imaging device 200 shown in FIG. 232 is imaged on the image sensor 223 by the microlens array 222, and a captured image from each microlens 222A is received by a plurality of imaging pixels (for example, n × n (n: integer of 2 or more)). It has become so. The captured image obtained by photographing in this way is equal to the case where an image observed at the position of each microlens 222A is photographed at an n × n resolution, and the method disclosed in Patent Document 2 is used. By rearranging the element outputs and performing pixel interpolation processing, a reconstructed image with high resolution can be obtained.

International Publication No. 06/039486 Pamphlet Japanese Patent No. 3699921 Ren.Ng and 7 others, “Light Field Photography with a Hand-Held Plenoptic Camera”, Stanford Tech Report CTSR 2005-02 Y. Kitamura and 10 others “Reconstruction of High-Resolution Image on a Compound-Eye Image-Capturing System”, Appl. Opt. 2004, Vol. 43, p. 1719-1727

  By the way, in the imaging apparatus using the technique called “Light Field Photography” described above, the F-numbers are made to coincide between the main lens (for example, the imaging lens 111) and the microlens array (for example, the microlens array 112). The necessity is described in the above-mentioned Patent Document 1. For example, when the F number of the main lens is smaller than the F number of the microlens array (in the case of F (ML) <F (MLA)), for example, the reference numbers P101 to P103 in FIG. As described above, the overlap occurs between the imaged light beams by the adjacent microlenses 112A, and this causes crosstalk, so that the image quality of the reconstructed image is deteriorated. On the other hand, when the F number of the main lens is larger than the F number of the microlens array (when F (ML)> F (MLA)), for example, the reference numbers P104 to P106 in FIG. As described above, an imaging pixel in which the imaging light beam from the microlens 112A is not received is generated, so that the imaging pixel cannot be sufficiently used, and the resolution of the reconstructed image is lowered. Among these, in particular, regarding the occurrence of the former crosstalk, a part of the information of the observation image is lost, so it is necessary to improve so as not to occur. However, Patent Document 1 and Non-Patent Document 1 do not describe any specific improvement method, so an improvement method for this point is desired.

  On the other hand, in the above-described imaging apparatus called “Compound Eye”, as in the imaging apparatus 200 shown in FIG. 20, for example, each microlens 222A is not provided with an aperture restriction, so that it is necessarily left as it is. Crosstalk occurs on the image sensor 223, and as a result, the resolution of the reproduced image is significantly reduced. In Non-Patent Document 2, for example, as shown in FIG. 20, a technique for solving the problem of crosstalk by arranging a light-shielding body 221 called a signal separator between a microlens array 222 and an image sensor 223. Has been proposed. However, since it is difficult to integrally form such a light shield, and since it is difficult to completely prevent reflection of incident light within the light shield, leakage light is generated. It is difficult to avoid the occurrence of talk, and a more realistic improvement method is desired.

  The present invention has been made in view of such problems, and an object thereof is to provide an imaging apparatus capable of easily suppressing the occurrence of crosstalk during imaging using an arrayed optical element. is there.

  The imaging device of the present invention is arranged so that a plurality of imaging elements are arrayed in a two-dimensional array, and imaging light from each imaging element is received by a plurality of imaging pixels. An image sensor and a switching unit that is disposed on at least one side of the imaging element array unit and that can switch the passage and blocking of the imaging light beam toward the image sensor for each imaging element, and controls the switching operation by the switching unit And a control unit.

  In the imaging apparatus of the present invention, the imaging light from each imaging element in the imaging element array unit is received by a plurality of imaging pixels in the imaging element. At that time, the switching means arranged on at least one side of the imaging element array unit switches the passage and blocking of the imaging light beam toward the imaging element for each imaging element. Therefore, the overlapping of light rays on the image pickup element can be suppressed between the image forming lights by the respective image forming elements.

  According to the imaging apparatus of the present invention, the switching unit is disposed on at least one side of the imaging element array unit, and the switching unit switches the passage and blocking of the imaging light beam toward the imaging element in units of imaging elements. Therefore, it is possible to suppress the overlapping of light rays on the image pickup element between the image forming lights by the respective image forming elements. Therefore, it is possible to easily suppress the occurrence of crosstalk during imaging using an arrayed optical element.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 shows the overall configuration of an imaging apparatus (imaging apparatus 1) according to a first embodiment of the present invention. The imaging device 1 captures an imaging target 31 and outputs imaging data Dout, and includes an imaging lens 11, a liquid crystal shutter array 10, a microlens array 12, an imaging element 13, and an image processing unit 14. And a liquid crystal shutter array driving unit 15, an image sensor driving unit 16, and a control unit 17.

  The imaging lens 11 is a main lens for imaging the imaging target 31, and is configured by a general imaging lens used in a video camera, a still camera, or the like, for example.

  The microlens array 12 includes a plurality of microlenses 12A arranged in a matrix (two-dimensional array), and the focal plane of the imaging lens 11 (reference numeral f1 in the drawing represents the focal length of the imaging lens 11). ). The microlens 12A corresponds to a specific example of “imaging element” in the present invention, and the microlens array 12 corresponds to a specific example of “imaging element array section” in the present invention.

  The image pickup device 13 receives light from the microlens array 12 and generates image pickup data. The focal plane of the microlens array 12 (reference numeral f2 in the figure represents the focal length of the microlens array 12). Arranged). The imaging element 13 is configured by, for example, a plurality of CCDs (Charge Coupled Devices) arranged in a matrix (two-dimensional array). Each CCD or the like constitutes an imaging pixel that is a unit pixel at the time of imaging.

  The liquid crystal shutter array 10 is disposed on one side of the microlens array 12, specifically, between the microlens array 12 and the imaging lens 11, and passes and blocks the imaging light beam toward the imaging element 13 in units of microlenses 12A. It can be switched. The detailed configuration of the liquid crystal shutter array 10 will be described later.

  The liquid crystal shutter array drive unit 15 controls the switching operation by driving the liquid crystal shutter array 10. Specifically, the switching operation of the liquid crystal shutter array 10 is controlled by changing the magnitude of the voltage applied to each liquid crystal shutter included in the liquid crystal shutter array 10. The image sensor driving unit 16 controls the light receiving operation by driving the image sensor 13.

  The image processing unit 14 performs predetermined image processing on the image data obtained by the image sensor 13 and outputs the image data as image data Dout. Specifically, for example, a refocusing calculation process using a technique called “Light Field Photography” is performed, whereby an observation image from an arbitrary viewpoint or direction can be reconstructed.

  The control unit 17 controls operations of the image processing unit 14, the liquid crystal shutter array driving unit 15, and the image sensor driving unit 16. Specifically, the driving operation of the image sensor driving unit 16 and the driving operation of the liquid crystal shutter array driving unit 10 are appropriately controlled, and the operation of the image processing unit 14 is controlled. The control unit 17 is configured by a microcomputer, for example. The controller 17 and the liquid crystal shutter array driver 15 described above correspond to a specific example of “a controller” in the present invention.

  Next, the detailed configuration of the liquid crystal shutter array 10 will be described with reference to FIGS. Here, FIG. 2 is an enlarged perspective view of a part of the imaging apparatus 1 (specifically, the imaging lens 11, the liquid crystal shutter array 10, the microlens array 12, and the imaging element 13). FIG. 3A shows a cross-sectional configuration of the liquid crystal shutter array 10.

  As shown in FIG. 2, in the liquid crystal shutter array 10, a plurality of liquid crystal shutters are arranged in a matrix (two-dimensional array) corresponding to each microlens 12 </ b> A in the microlens array 12. Each of the plurality of liquid crystal shutters constitutes a switching area, which is a unit area in the above-described imaging light beam switching operation, and the switching operation is periodically performed between the switching areas, as will be described in detail later. ing. Further, in each cycle of the switching operation, for example, a switching region 10B in which the imaging light beam L1 passes through the liquid crystal shutter as in the switching region 10B in FIG. 2, and an imaging light beam L2 in the switching region 10A in FIG. The cut-off switching areas where the liquid crystal shutters are cut off are arranged in a staggered manner, and the pass switching area and the cut-off switching area are interchanged between each cycle, as will be described in detail later.

  As shown in FIG. 3A, in the liquid crystal shutter array 10, a liquid crystal layer 104 is formed between a pair of opposing substrates 101 and 106, and the liquid crystal layer 104 is interposed between the substrates 101 and 106. Electrodes 102 and 105 and alignment films 103A and 103B are formed. Polarizing filters 107A and 107B are formed on the opposite sides (outside) of the liquid crystal layer 104 on the substrates 101 and 106, respectively.

  Each of the substrates 101 and 106 is made of a transparent substrate such as a glass substrate, and can transmit incident light. A voltage is supplied to the electrodes 102 and 105 from the liquid crystal shutter array driving unit 15. Each of the electrodes 102 and 105 is configured by a transparent electrode such as ITO (Indium Tin Oxide), and can transmit incident light as in the case of the substrates 101 and 106. The alignment films 103A and 103B are for uniformly aligning the alignment directions of the liquid crystal molecules in the liquid crystal layer 104. In this embodiment, the alignment directions for the liquid crystal molecules are between the alignment films 103A and 103B. However, they are orthogonal to each other in the formation plane. The liquid crystal layer 104 is made of, for example, a liquid crystal material such as nematic liquid crystal, and the alignment direction of the liquid crystal molecules changes according to the voltage applied between the electrodes 101 and 106.

  Each of the polarizing films 107A and 107B selectively transmits polarized light in a direction along a predetermined polarization axis among incident light rays. Among these, as shown in FIG. 3B, for example, the polarizing film 107A has a polarization axis common to the liquid crystal shutters (in this case, the horizontal direction in the figure) regardless of the difference between the switching areas 10A and 10B. Have. On the other hand, in the polarizing film 107B, for example, as shown in FIG. 3C, the polarization axes of the liquid crystal shutters are orthogonal to each other between the switching region 10A and the switching region 10B. Specifically, for example, in this case, the switching region 10A has a polarization axis (in this case, the horizontal direction in the figure) along the same direction as the polarization axis of the corresponding region in the polarizing film 107, and the switching region 10B has In the polarizing film 107, there is a polarization axis along a direction orthogonal to the polarization axis of the corresponding region. Although the liquid crystal shutter array 10 having such a configuration will be described in detail later, different switching operations are performed between the switching regions 10A and 10B, and the presence or absence (or magnitude) of the voltage applied between the electrodes 102 and 105 is determined. Accordingly, the switching operations are changed (changed) with each other.

  Next, the operation of the imaging apparatus 1 according to the present embodiment will be described in detail with reference to FIGS.

  First, the basic operation of the imaging apparatus 1 will be described with reference to FIGS. 1, 4, and 5.

  In the imaging apparatus 1, an image of the imaging target 31 by the imaging lens 11 is formed on the microlens array 12. Light rays incident on the microlens array 12 reach the image sensor 13 via the microlens array 12, and image data is obtained from the image sensor 13 in accordance with the drive operation by the image sensor drive unit 16.

  Next, the imaging data obtained by the imaging element 13 is input to the image processing unit 14. In the image processing unit 14, predetermined image processing is performed on the imaging data in accordance with the control of the control unit 17, thereby outputting imaging data Dout.

  Here, an example of image processing (refocus calculation processing) by the image processing unit 14 will be described in detail with reference to FIGS. 4 and 5.

First, as shown in FIG. 4, an orthogonal coordinate system (u, v) is considered on the imaging lens surface of the imaging lens 11, and an orthogonal coordinate system (x, y) is considered on the imaging surface of the imaging element 13. Assuming that the distance between the imaging lens surface of the lens 11 and the imaging surface of the imaging device 13 is F, the light beam passing through the imaging lens 11 and the imaging device 13 as shown in the figure is a four-dimensional function L _F (x, y, Since it is expressed by u, v), it is expressed in a state where the traveling direction of the light beam is held in addition to the position information of the light beam.

In this case, as shown in FIG. 5, when the positional relationship among the imaging lens surface 110, the imaging surface 130, and the refocus surface 120 is set (the refocus surface 120 is set so that F ′ = αF), The detected intensity LF _′ of the coordinates (s, t) on the refocus plane 120 on the imaging plane 130 is expressed by the following equation (1). Further, the image E _{F ′} (s, t) obtained on the refocus plane 120 is obtained by integrating the detected intensity L _{F ′} with respect to the lens aperture, and is represented by the following equation (2). Therefore, by performing the refocus calculation from the equation (2), an observation image from an arbitrary viewpoint or direction is reconstructed based on the imaging data Dout after the image processing.

  Next, with reference to FIGS. 3, 6, and 7, an imaging light beam switching operation by the liquid crystal shutter array 10, which is one of the characteristic parts of the present invention, will be described in detail. Here, FIG. 6 is for explaining the operation of the liquid crystal shutter array 10, and is represented by a plan view in which each liquid crystal shutter and each microlens 12A are overlapped. FIG. 7 is a sectional view showing the operation of the liquid crystal shutter array 10. 6A and 7A show the case where no voltage is applied between the electrodes 102 and 105 by the liquid crystal shutter array driving unit 15 (or when the applied voltage is lower than a predetermined threshold voltage). FIG. 6B and FIG. 7B show the case where a voltage is applied between the electrodes 102 and 105 by the liquid crystal shutter array driving unit 15 (or when the applied voltage is higher than a predetermined threshold voltage). Represents each one.

  First, as shown in FIGS. 6A and 7A, when no voltage is applied between the electrodes 102 and 105, the imaging light beam of the imaging lens 11 enters the polarizing filter 107. As shown in FIG. 3B, the polarized light in the horizontal direction is selectively transmitted, and the polarized light in the horizontal direction is modulated so that the polarization direction is changed by 90 degrees in the liquid crystal layer 104. And reaches the polarizing filter 107B. Then, as illustrated in FIG. 3C, in the polarization filter 107B, the polarization axes are orthogonal to each other in the switching regions 10A and 10B. Therefore, in the switching region 10A, as illustrated in FIG. In the polarizing film 107B, the vertically polarized light (imaging light beam) is blocked, while in the switching region 10B, the vertically polarized light (imaging light beam) passes through the polarizing film 107B. Here, since the switching region 10A and the switching region 10B are arranged in a staggered manner, for example, as shown in FIG. 7A, the imaging light beams that have passed through the polarizing film 107B are reflected on the imaging device 13. Overlap (overlap of rays adjacent to each other in the vertical and horizontal directions of the drawing) is suppressed (in this case, overlap is avoided).

  On the other hand, as shown in FIGS. 6B and 7B, when a voltage is applied between the electrodes 102 and 105, the imaging light beam of the imaging lens 11 enters the polarizing filter 107. As shown in FIG. 3B, the polarized light in the left-right direction selectively passes through, and the polarized light in the left-right direction is not modulated in the liquid crystal layer 104, and remains the polarized light in the left-right direction. Reach 107B. Therefore, contrary to the case where no voltage is applied, as shown in FIG. 6B, in the switching region 10B, the polarized light (imaging light beam) in the horizontal direction is blocked in the polarizing film 107B, while the switching region In 10A, left-right polarized light (imaging light beam) passes through the polarizing film 107B. Also in this case, for example, as shown in FIG. 7B, the overlapping of the imaging light rays that have passed through the polarizing film 107B on the imaging element 13 is suppressed (in this case, the overlapping is avoided).

  The control unit 17 and the liquid crystal shutter array driving unit 15 periodically perform a period in which no voltage is applied between the electrodes 102 and 105 and a period in which a voltage is applied between the electrodes 102 and 105 in this manner. In this way, the imaging operation of the imaging object 32 is required to be performed twice (two times when no voltage is applied and two times when no voltage is applied). On the element 13, the overlapping of the imaging light rays is always suppressed.

  As described above, in the present embodiment, a liquid crystal including a plurality of liquid crystal shutters arranged corresponding to each microlens 12A on one side of the microlens array 12 (between the microlens array 12 and the imaging lens 11). Since the shutter array 10 is disposed and the liquid crystal shutter array 10 is used to switch the passage and blocking of the imaging light beam toward the image sensor 13 in units of the microlenses 12A, the imaging light by the microlenses 12A is imaged with each other. Overlapping of light rays on the element 13 can be suppressed. Therefore, it is possible to easily suppress the occurrence of crosstalk during imaging using the arrayed optical element (in this case, the microlens array 12). Therefore, it is possible to improve the image quality of the captured image, and thus the reconstructed image after the image processing by the image processing unit 14.

  Further, in the liquid crystal shutter array 10, the passage switching region through which the imaging light beam passes and the blocking switching region through which the imaging light beam is blocked are arranged in a staggered manner, and the passage switching region and the blocking switching region between each cycle. Are periodically changed (replaced), so that multiple imaging operations (in this case, two times) are required, but it is possible to always suppress overlapping of the imaging light beams on the imaging device 13. . Therefore, the image quality of the captured image and the reconstructed image can be further improved.

  Further, for example, as shown in FIG. 8, each microlens 12 </ b> A is within a range where imaging light beams that have passed through the switching region 10 </ b> A or the switching region 10 </ b> B do not overlap on the light receiving element 13 (within a range in which crosstalk does not occur). When the setting is made so that the number of imaging pixels in the imaging device 13 allocated to the reception of the imaging light by (1) increases (increases), the increased number of imaging pixels (in this case, the number of imaging pixels is 2). Can be used to improve the resolution in the depth direction and the parallax direction. Therefore, in such a configuration, the image quality of the reconstructed image after image processing in the image processing unit 14 can be further improved.

  Further, for example, like the microlens array 12B (having a plurality of microlenses 12C) and the liquid crystal shutter array 10C illustrated in FIG. 9, the imaging light beams that have passed through the switching region 10A or the switching region 10B overlap on the light receiving element 13. Within a range that does not match (within a range in which crosstalk does not occur), the number of microlenses 12A in the microlens array 12 and the number of liquid crystal shutters (switching areas 10A and 10B) in the liquid crystal shutter array 10 increase (increase). ), The resolution of the captured image can be increased. In this case, the resolution can be doubled at the maximum. Therefore, in such a configuration, the image quality of the captured image and the reconstructed image can be further improved.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the same thing as the component in 1st Embodiment, and description is abbreviate | omitted suitably.

  FIG. 10 illustrates the overall configuration of the imaging apparatus (imaging apparatus 1A) according to the present embodiment. This imaging device 1A is provided with an aperture array 18 having a plurality of apertures 18A instead of the liquid crystal shutter array 10 in the imaging device 1 described in the first embodiment, and in place of the liquid crystal shutter array driving unit 15. An aperture array driving unit 19 is provided.

  In the aperture array 18, for example, as shown in FIG. 11A, the apertures 18A correspond to the microlenses 12A and are arranged in a staggered manner. Further, in this case, an array pattern of the openings 18A is provided in one row in the left-right direction on the drawing and extra at the lowermost end. Note that this extra arrangement pattern may be provided at the uppermost end. The aperture array 18 corresponds to a specific example of “switching means” in the present invention.

  The aperture array driving unit 19 controls the switching operation by driving the aperture array 18. Specifically, the switching operation of the aperture array 10 is controlled by displacing the relative position (relative position in the in-plane direction) of the aperture array 18 with respect to the microlens array 12. The aperture array drive unit 19 and the control unit 17 correspond to a specific example of “control unit” in the present invention.

  Next, with reference to FIG. 10 and FIG. 11, the switching operation of the imaging light beam by the aperture array 18 which is a characteristic operation in the imaging apparatus 1A of the present embodiment will be described in detail. Here, FIG. 11 is for explaining the operation of the aperture array 18, and is represented by a plan view in which the apertures 18A and the microlenses 12A are overlapped. Note that the basic operation (imaging operation) of the imaging apparatus 1A is the same as that of the imaging apparatus 1 described in the first embodiment, and thus the description thereof is omitted.

  In the imaging apparatus 1A of the present embodiment, when the relative position (in the in-plane direction) of the aperture array 18 with respect to the microlens array 12 is as shown in FIG. 11A, for example, the aperture 18A is provided. The switching area that functions as a passage switching area, and the imaging light beam from the imaging lens 11 passes through the aperture array 18, while the switching area that is not provided with the opening 18A functions as a blocking switching area. The imaging light beam is blocked by the aperture array 18. The openings 18A are arranged in a staggered pattern on the opening array 18. Therefore, as in the first embodiment, the overlapping of the imaging light rays that have passed through the opening 18A on the imaging element 13 (the overlapping of the light rays adjacent to each other in the vertical and horizontal directions in the drawing) is suppressed (this). If overlap is avoided).

  On the other hand, the aperture array drive unit 19 and the control unit 17 displace the aperture array 18 upward in the plane, and the relative position (in the plane) of the aperture array 18 with respect to the microlens array 12 is, for example, as shown in FIG. As shown, since the openings 18A are arranged in a staggered pattern in the opening array 18, the pass switching region and the blocking switching region are interchanged with each other for each microlens 12A. Therefore, in this case as well, the imaging light beams that have passed through the opening 18A can be prevented from overlapping on the image sensor 13 (the overlapping of the light beams adjacent to each other in the vertical and horizontal directions in the drawing) (in this case, the overlapping is avoided). )

  Further, the control unit 17 and the aperture array driving unit 19 are controlled so that the aperture array 18 is periodically displaced in the vertical direction, whereby the imaging operation of the imaging object 32 is performed twice (FIG. 11A). Although it is necessary to perform the imaging in the arrangement and the imaging in the arrangement of FIG. 11B twice, the overlapping of the imaging light rays is always suppressed on the imaging element 13.

  As described above, in the present embodiment, one side of microlens array 12 (between microlens array 12 and imaging lens 11) includes a plurality of openings 18A arranged corresponding to each microlens 12A. The aperture array 18 is arranged, and the aperture array 18 is used to switch the passage and blocking of the imaging light beam toward the image sensor 13 in units of microlenses 12A. Therefore, each microlens is the same as in the first embodiment. Overlapping of light rays on the image sensor 13 can be suppressed between the imaged lights by 12A. Therefore, it is possible to easily suppress the occurrence of crosstalk during imaging using the arrayed optical element (in this case, the microlens array 12). Therefore, it is possible to improve the image quality of the captured image, and thus the reconstructed image after the image processing by the image processing unit 14.

  Further, by arranging the openings 18A in a staggered pattern in the aperture array 18, the passage switching area and the blocking switching area are arranged in a staggered pattern, and the aperture array 18 is periodically displaced in the vertical direction. Since the pass switching area and the cut-off switching area are periodically changed (replaced) between each cycle, the imaging operation is performed a plurality of times (in this case, twice) as in the first embodiment. However, it is possible to always suppress the overlapping of the imaging light rays on the image sensor 13. Therefore, the image quality of the captured image and the reconstructed image can be further improved.

  Further, for example, as in the case of FIGS. 8 and 9 in the first embodiment, the imaging light beams that have passed through the opening 18A do not overlap on the light receiving element 13 (within a range in which crosstalk does not occur). Thus, the number of image pickup pixels in the image pickup device 13 assigned to the reception of the imaging light by each microlens 12A is set to increase (increase) or the microlens 12A in the microlens array 12 is set. When the number and the number of openings 18A in the aperture array 18 are set to increase (increase), the image quality can be further improved as in the first embodiment.

  11A and 11B, an extra array pattern of the openings 18A is provided at the lowermost end or the uppermost end in the opening array 18, and the opening array 18 is in-plane with respect to the microlens array 12. However, for example, the aperture array 18B shown in FIGS. 12A and 12B has a plurality of apertures 18C arranged in a staggered manner. In the aperture array 18B, an extra array pattern of the openings 18C is provided at the leftmost end or the rightmost end, and the aperture array 18B is periodically displaced in the horizontal direction in the plane with respect to the microlens array 12. It may be. Even when configured in this manner, the same effects as in the present embodiment can be obtained.

  In the opening array of the present embodiment, the case where the shape of the opening is rectangular has been described. For example, the opening array 18D (having a plurality of openings 18E) illustrated in FIG. As shown in the aperture array 18F (having a plurality of apertures 18G) shown in FIG. 13B, the apertures are formed in a circular shape in accordance with the shape of each microlens 12A. The aperture array may be periodically displaced in the vertical direction (example shown in FIG. 13A) or in the left-right direction (example shown in FIG. 13B). Even when configured as described above, the same effects as in the present embodiment can be obtained.

  Although the present invention has been described with reference to the first and second embodiments, the present invention is not limited to these embodiments, and various modifications can be made.

  For example, in the above-described embodiment, the case where each microlens 12A and each switching region are arranged in a matrix in the microlens array 12 and the liquid crystal shutter array or the aperture array has been described. As another example of arrangement when the regions are arranged in a two-dimensional array, for example, a liquid crystal shutter array 10D shown in FIG. 14A or an aperture array 18H shown in FIG. Each switching area may have a hexagonal shape, and each microlens 12A and each switching area may be in a close-packed arrangement. Specifically, in the liquid crystal shutter array 10D, one switching group region is configured by three switching regions (for example, the switching regions 10E to 10G), and the passing switching region and the blocking switching region are mutually connected during each period. The passage switching area is changed periodically, for example, in the order of the switching area 10E, the switching area 10F, and the switching area 10G. Further, in the opening array 18H, one switching group region is configured by the switching region provided with the opening 18I that is one opening switching region and the two blocking switching regions, and the passage switching region between each cycle. And the cut-off switching area are interchanged (for example, the opening array 18H is periodically displaced in the plane so that the position of the opening 18I is displaced as indicated by an arrow in FIG. 14B). It has become. When configured in this way, in addition to the effects of the above-described embodiment, three imaging operations are required, but the number of microlenses 12A in the microlens array 12 can be increased and one pass switching region All the switching areas surrounding the can be set as the cutoff switching area. Therefore, each microlens 12A and each switching area are arranged in a matrix, and each microlens 12A has a switching area adjacent to a single passing area in the vertical and horizontal directions as a blocking area. It is possible to further suppress the overlapping of the light rays on the image sensor 13 between the imaging lights, and to further improve the image quality of the captured image and the reconstructed image.

  In the above embodiment, in the liquid crystal shutter array or the aperture array, the pass switching areas are arranged in a staggered pattern every period, and the pass switching areas are switched between the periods. Although the case has been described, more generally, one switching group region is configured from one passage switching region and a plurality of cutoff switching regions in each cycle of the switching operation, and each switching group region is between each cycle. The positions of the passage switching areas may be periodically displaced so as not to be adjacent to each other between the switching group areas. Specifically, for example, a liquid crystal shutter array 10H shown in FIG. 15A and an aperture array 18J shown in FIG. The position of the passage switching area in each switching group area may be periodically displaced as indicated by the arrows in the figure. Specifically, in the liquid crystal shutter array 10H, one switching group region 10I is configured by four switching regions (for example, the switching regions 10I1 to 10I4), and a passing switching region and a blocking switching region are provided between each cycle. They are switched to each other (the passage switching area changes periodically in the order of, for example, the switching area 10I1, the switching area 10I2, the switching area I3, and the switching area 10I4). Further, in the opening array 18J, one switching group region is configured by the switching region provided with the opening 18K which is one opening switching region and the three blocking switching regions, and the passage switching region between the periods. And the cut-off switching area are interchanged (for example, the opening array 18J is periodically displaced in the plane so that the position of the opening 18K is displaced as indicated by the arrow in FIG. 15B). It has become. When configured in this way, in addition to the effects in the above-described embodiment, a plurality of imaging operations (four times in the case of FIG. 15) are required, but all switching regions surrounding one passage switching region are all blocked switching regions. It can be. Therefore, compared to the above-described embodiment in which the switching region adjacent to the upper, lower, left, and right with respect to one passing region is the blocking region, the overlapping of the light rays on the image pickup device 13 between the imaged light by each microlens 12A is further increased. Thus, the image quality of the captured image and the reconstructed image can be further improved.

  In the above embodiment, the case where the liquid crystal shutter array or the aperture array is provided on one side of the micro lens array 12 (specifically, between the imaging lens 11 and the micro lens array 12) has been described. The liquid crystal shutter array and the aperture array may be provided on at least one side along the optical axis direction with respect to the microlens array 12. Specifically, the liquid crystal shutter array 10J may be provided between the microlens array 12 and the image sensor 13 as in the image pickup apparatus 1B shown in FIG. 16, for example, as shown in FIG. Like the imaging device 1C, liquid crystal shutter arrays (liquid crystal shutter arrays 10K1 and 10K2) may be provided on both ends of the microlens array 12. In the former case (FIG. 16), the same effect as in the above embodiment can be obtained. However, from the viewpoint of blocking part of the imaging light, a liquid crystal shutter array is arranged as close to the microlens array 12 as possible. Is preferred. Further, in the latter case (FIG. 17), part of the imaging light beam by the imaging lens 11 can be double blocked, so that the imaging light by the microlenses 12A is compared with the imaging element in comparison with the above embodiment. It is possible to further suppress the overlapping of light rays on the image 13, and to further improve the image quality of the captured image and the reconstructed image.

  In the above-described embodiment, the microlens 12A is exemplified as an example of the imaging element, and the case where the imaging element array unit is configured by the microlens array 12 has been described. For example, a liquid crystal lens having imaging characteristics Alternatively, the imaging element array unit may be configured by imaging elements such as a diffraction grating having imaging characteristics. When configured with a liquid crystal lens having imaging characteristics, it is possible to perform imaging with spectral characteristics in addition to the effects of the above embodiment. In addition, since a problem of chromatic aberration occurs when a diffraction grating having imaging characteristics is formed, the imaging apparatus is suitable particularly when the imaging light beam is monochromatic light. Compared to the above configuration, the imaging element array portion can be made thinner and can be manufactured at a lower cost.

  Furthermore, in the above embodiment, the refocus calculation process using “Light Field Photography” has been described as an example of the image processing method in the image processing unit 14, but the image processing method in the image processing unit 14 is not limited to this. The image processing method is not limited and may be other image processing methods. Further, in the conventional imaging device 200 (imaging device using “Compound Eye”) shown in FIG. 20, the liquid crystal shutter array and the aperture array described in the above embodiment may be provided. Specifically, for example, like the imaging device 2 illustrated in FIG. 18, predetermined image processing is performed with the microlens array 22 having a plurality of microlenses 22 </ b> A and the imaging element 23 in order to image the imaging target 32. An image processing unit (not shown) and the liquid crystal shutter array 10 (or the aperture array 18) may be provided. When configured in this manner, similar to the above-described embodiment, the overlapping of light rays on the image sensor 23 can be further suppressed between the imaged light beams formed by the microlenses 22A. Therefore, compared to the conventional imaging apparatus 200 shown in FIG. 20, it is possible to easily suppress the occurrence of crosstalk and improve the image quality of the reconstructed image.

It is a block diagram showing the structure of the imaging device which concerns on the 1st Embodiment of this invention. It is the perspective view which expanded and represented a part of imaging device shown in FIG. FIG. 2 is a cross-sectional view and a schematic view for explaining the configuration of the liquid crystal shutter array shown in FIG. 1. It is a model perspective view for demonstrating an example of the image process in the imaging device shown in FIG. It is a schematic cross section for demonstrating an example of the image process in the imaging device shown in FIG. It is a top view for demonstrating the effect | action of the liquid-crystal shutter array shown in FIG. It is sectional drawing for demonstrating an effect | action of the liquid-crystal shutter array shown in FIG. It is sectional drawing for demonstrating the effect | action of the liquid-crystal shutter array which concerns on the modification of 1st Embodiment. It is sectional drawing for demonstrating the structure and effect | action of the liquid-crystal shutter array and microlens array which concern on the other modification of 1st Embodiment. It is a block diagram showing the structure of the imaging device which concerns on the 2nd Embodiment of this invention. It is a top view for demonstrating an effect | action of the opening array shown in FIG. It is a top view for demonstrating the effect | action of the opening array which concerns on the modification of 2nd Embodiment. It is a top view for demonstrating the structure and effect | action of an opening array which concern on the other modification of 2nd Embodiment. It is a top view for demonstrating the structure and effect | action of a liquid-crystal shutter array and opening array which concern on the modification of this invention. It is a top view for demonstrating the structure and effect | action of the liquid-crystal shutter array and opening array which concern on the other modification of this invention. It is a block diagram showing the structure of the imaging device which concerns on the other modification of this invention. It is a block diagram showing the structure of the imaging device which concerns on the other modification of this invention. It is a perspective view showing the structure of the imaging device which concerns on the other modification of this invention. It is sectional drawing showing the structural example of the conventional imaging device. It is a perspective view showing the other structural example of the conventional imaging device. It is sectional drawing for demonstrating an effect | action of the conventional imaging device shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1,1A-1C, 2 ... Imaging device 10, 10C, 10D, 10H, 10J, 10K1, 10K2 ... Liquid crystal shutter array, 10A, 10B, 10E-10G, 10I1-10I4 ... Liquid crystal shutter, 10I ... Switching group, 101 , 106 ... Substrate, 102, 105 ... Electrode, 103A, 103B ... Alignment film, 104 ... Liquid crystal layer, 107A, 107B ... Change filter, 11 ... Imaging lens, 110 ... Imaging lens surface, 12, 12B, 22 ... Micro lens array , 12A, 12C, 22A ... microlens, 120 ... refocus plane, 13, 23 ... imaging element, 130 ... imaging plane, 14 ... image processing unit, 15, 15B, 15C ... liquid crystal shutter array driving unit, 16 ... imaging element Drive unit, 17 ... control unit, 18, 18B, 18D, 18F, 18H, 18J ... opening array 18A, 18C, 18E, 18G, 18I, 18K ... opening, 19 ... opening array driver, 31, 32 imaged object, f1, f2 ... focal length, Dout ... imaging data, L1, L2, L4 ... light.

Claims (12)

An imaging element array unit formed by arranging a plurality of imaging elements in a two-dimensional array;
An imaging device arranged to receive the imaging light from each imaging device by a plurality of imaging pixels;
Switching means arranged on at least one side of the imaging element array section and capable of switching the imaging light beam passing and blocking toward the imaging element in units of the imaging element;
An image pickup apparatus comprising: a control unit that controls a switching operation by the switching unit. The switching means has a plurality of switching areas which are arrangement corresponding to the imaging elements and are unit areas of the switching operation,
The imaging apparatus according to claim 1, wherein the control unit performs control so that the switching operation is periodically performed between the switching regions. The control unit has a staggered arrangement of a passage switching region through which the imaging light beam passes and a blocking switching region through which the imaging light beam is blocked in each cycle of the switching operation, and the passage switching between each cycle. The imaging apparatus according to claim 2, wherein the switching operation is controlled so that the area and the cutoff switching area are switched with each other. The control unit includes one switching group region composed of one passing switching region through which the imaging light beam passes and a plurality of blocking switching regions in which the imaging light beam is blocked in each cycle of the switching operation. The switching operation is controlled so that the position of the pass switching area in each switching group area is periodically displaced so as not to be adjacent to each other between the switching group areas during the period. Item 3. The imaging device according to Item 2. The switching means is a liquid crystal shutter array including a plurality of liquid crystal shutters arranged corresponding to the imaging elements,
The imaging apparatus according to claim 1, wherein the control unit controls the switching operation by changing a magnitude of an applied voltage to the liquid crystal shutter. The switching means is an aperture array having a plurality of apertures arranged corresponding to the imaging elements,
The imaging apparatus according to claim 1, wherein the control unit controls the switching operation by displacing a relative position of the aperture array with respect to the imaging element array unit. The imaging apparatus according to claim 1, wherein the switching unit is disposed on both sides of the imaging element array unit. Setting is made so that the number of the imaging pixels allocated to the reception of the imaging light by each imaging element increases within a range in which the imaging light beams that have passed through the switching means do not overlap each other between the corresponding imaging elements. The imaging apparatus according to claim 1, wherein: The number of imaging elements in the imaging element array section is set to be large within a range in which light beams that have passed through the switching means do not overlap with each other between corresponding imaging elements. The imaging apparatus according to 1. The imaging device according to claim 1, wherein the imaging element array unit is a microlens array. The imaging apparatus according to claim 1, wherein the imaging element is a liquid crystal lens having imaging characteristics. The imaging apparatus according to claim 1, wherein the imaging element is a diffraction element having imaging characteristics.

Images