US9300885B2 - Imaging device, portable information terminal, and imaging system - Google Patents
Imaging device, portable information terminal, and imaging system Download PDFInfo
- Publication number
- US9300885B2 US9300885B2 US14/162,122 US201414162122A US9300885B2 US 9300885 B2 US9300885 B2 US 9300885B2 US 201414162122 A US201414162122 A US 201414162122A US 9300885 B2 US9300885 B2 US 9300885B2
- Authority
- US
- United States
- Prior art keywords
- mtf
- image
- imaging
- lens
- main
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 166
- 230000014509 gene expression Effects 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 35
- 239000007787 solid Substances 0.000 abstract description 29
- 239000000758 substrate Substances 0.000 abstract description 21
- 239000004065 semiconductor Substances 0.000 abstract description 13
- 230000002194 synthesizing effect Effects 0.000 abstract description 2
- 230000003287 optical effect Effects 0.000 description 74
- 230000000052 comparative effect Effects 0.000 description 18
- 238000000034 method Methods 0.000 description 17
- 238000013461 design Methods 0.000 description 16
- 230000004075 alteration Effects 0.000 description 15
- 230000009467 reduction Effects 0.000 description 13
- 230000000007 visual effect Effects 0.000 description 7
- 230000000149 penetrating effect Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 238000001308 synthesis method Methods 0.000 description 6
- 230000007423 decrease Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 150000001875 compounds Chemical class 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000002834 transmittance Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000010410 layer Substances 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000011347 resin Substances 0.000 description 1
- 229920005989 resin Polymers 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 125000006850 spacer group Chemical group 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B13/00—Optical objectives specially designed for the purposes specified below
- G02B13/001—Miniaturised objectives for electronic devices, e.g. portable telephones, webcams, PDAs, small digital cameras
-
- H04N5/335—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/42—Diffraction optics, i.e. systems including a diffractive element being designed for providing a diffractive effect
- G02B27/4205—Diffraction optics, i.e. systems including a diffractive element being designed for providing a diffractive effect having a diffractive optical element [DOE] contributing to image formation, e.g. whereby modulation transfer function MTF or optical aberrations are relevant
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/0006—Arrays
- G02B3/0037—Arrays characterized by the distribution or form of lenses
- G02B3/0056—Arrays characterized by the distribution or form of lenses arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/1462—Coatings
- H01L27/14621—Colour filter arrangements
-
- H04N13/0271—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/133—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements including elements passing panchromatic light, e.g. filters passing white light
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
-
- H04N5/2254—
-
- H04N5/2257—
Definitions
- Embodiments described herein relate generally to solid state imaging devices, portable information terminals, and solid state imaging systems.
- imaging techniques for obtaining two-dimensional information on the depth-direction distance to a subject.
- imaging techniques for obtaining two-dimensional information on the depth-direction distance to a subject.
- Use of such image information for obtaining a distance (hereinafter also referred to as the “distance image information”) enables subject recognition that is more sophisticated than subject recognition based on image information obtained from regular cameras.
- Devices that can obtain distance image information are therefore increasingly needed as relatively inexpensive products used as consumer appliances, games, and industrial devices.
- a compound-eye imaging device having an imaging lens and capable of obtaining a large number of parallax images and measuring a distance based on triangulation with a single camera.
- Such an imaging device includes, for example, a main imaging lens, and a plurality of optical systems serving as re-imaging optical systems between the main imaging lens and an imaging element.
- An example of the aforementioned optical systems is a micro lens array having microlenses on a plane surface. Pixels are arranged under the microlenses at locations corresponding to the respective microlenses in order to obtain images formed thereby.
- An image formed by the main imaging lens is re-imaged on the imaging element by the re-imaging microlenses with viewpoints being shifted due to parallaxes existing at the locations of the respective microlenses (parallax image group).
- the distance to the subject can be estimated based on the principles of triangulation by performing image processing on the parallax image group obtained from the microlenses.
- a two-dimensional image can be reconstructed from the images by performing image processing to combine the images.
- FIG. 1 is a block diagram showing a solid state imaging device according to a first embodiment.
- FIG. 2 is a cross-sectional view showing the solid state imaging device 1 according to the first embodiment.
- FIGS. 3( a ) and 3( b ) are drawings showing the relationship between the distance to a subject and light rays in an optical system of the solid state imaging device according to the first embodiment.
- FIGS. 4( a ) to 4( c ) are drawings showing a method of reconstructing a two-dimensional image with no overlap from microlens images by which the same subject is photographed several times.
- FIGS. 5( a ) to 5( e ) are drawings for explaining an image synthesis method for reconstituting a two-dimensional image.
- FIG. 6 is a drawing showing a frequency characteristic of MTF values.
- FIGS. 7( a ) to 7( c ) are drawings for explaining sensor sampling limit.
- FIG. 8 is a drawing showing the change in image frequency caused by the reduction by means of microlenses.
- FIG. 9 is a drawing showing the image frequency of a sensor (image plane).
- FIG. 10 is a drawing showing the image frequency characteristic of a reconstructed image.
- FIG. 11 is a drawing showing the relationship among the image frequencies in the solid state imaging device according to the first embodiment.
- FIG. 12 is a cross-sectional view showing a specific example of the optical system (with MLA) of the first embodiment.
- FIG. 13 is a drawing showing examples of numerical values of the optical system of the specific example.
- FIG. 14 is a drawing showing the MTF characteristics of the optical system of the specific example.
- FIG. 15 is a drawing showing the MTF through focus characteristics of the optical system of the specific example.
- FIG. 16 is a drawing showing the MTF through focus characteristics of the optical system of the specific example.
- FIG. 17 is a cross-sectional view showing a specific example of an optical system (with MLA) of a comparative example.
- FIG. 18 is a drawing showing examples of numerical values of the optical system of the comparative example.
- FIG. 19 is a drawing showing the MTF characteristics of the optical system of the comparative example.
- FIG. 20 is a drawing showing the MTF through focus characteristics of the optical system of the comparative example.
- FIG. 21 is a drawing showing the MTF through focus characteristics of the optical system of the comparative example.
- FIG. 22 is a drawing showing a portable information terminal according to the second embodiment.
- FIG. 23 is a solid state imaging system according to the third embodiment.
- FIGS. 24( a ) to 24( d ) are drawings showing how images are enlarged and added.
- Imaging modules implemented in such portable terminals need to be small, low-profile (thin), and low-power-consumption as well as capable of acquiring high-resolution two-dimensional visible images.
- the pixel pitch of imaging elements in imaging modules is getting closer to the order of visible light wavelength, and becoming finer.
- the diffraction limit which is the limit of a light-collecting spot diameter size, is proportional to the wavelength or F-number of an optical system. Therefore, in order to achieve a high resolution for a pixel pitch, the F-number of the optical system should also be reduced at the same time.
- the aperture aberration is increased, which makes lens design and manufacturing difficult, and which increases the number of lens elements.
- recently optical systems with the F-number of about 2 having four or five lens elements have been proposed.
- the aberration cannot be sufficiently compensated, which raises problems such as an increase in the number of components, higher assembling costs, and difficulties in downsizing the system, caused by the increase in the number of lenses.
- a solid state imaging device includes: an imaging element formed on a semiconductor substrate, and including pixel blocks each having pixels; a main lens forming an image of a subject on an imaging plane; a microlens array including microlenses corresponding to the pixel blocks, the microlens array reducing an image to be formed on the imaging plane by Nf times or less and forming reduced images on the pixel blocks corresponding to the respective microlenses; and an image processing unit enlarging and synthesizing the reduced images formed by the microlenses, the solid state imaging device meeting conditions of an expression MTF Main (u) ⁇ MTF ML (u) ⁇ MTF Main (u ⁇ Nf) where u denotes an image spatial frequency, MTF ML (u) denotes an MTF function of the microlenses, and MTF Main (u) denotes an MTF function of the main lens.
- FIG. 1 shows a solid state imaging device (hereinafter also referred to as the “camera module”) according to the first embodiment.
- the solid state imaging device 1 according to the first embodiment includes an imaging module unit 10 and an imaging signal processor (hereinafter also referred to as the “ISP”) 20 .
- ISP imaging signal processor
- the imaging module unit 10 includes an imaging optics (hereinafter also referred to as the “main lens”) 12 , a microlens array 14 (hereinafter also referred to as the “MLA”), an imaging element 16 , and an imaging circuit 18 .
- the imaging optics 12 functions as an imaging optical system to capture light from a subject into the imaging element 16 .
- the imaging element 16 functions as an element for converting the light captured by the imaging optics 12 to signal charges, and includes a plurality of pixels (for example, photodiodes serving as photoelectric conversion elements) arranged in a two-dimensional array form.
- the microlens array 14 is, for example, a microlens array including a plurality of microlenses, or a micro-optical system such as a prism.
- the microlens array 14 functions as an optical system for reducing and re-forming images on pixel blocks corresponding to respective microlenses (hereinafter also referred to as the “MLs”) from light rays that are focused on an image plane by the imaging optics 12 .
- the imaging circuit 18 includes a drive circuit unit (not shown) that drives the respective pixels of the pixel array of the imaging element 16 , and a pixel signal processing circuit unit (not shown) that processes signals outputted from the pixel region.
- the drive circuit unit and the pixel signal processing circuit unit may be combined with each other to form a drive and processing circuit.
- the imaging circuit 18 includes a drive and processing circuit.
- the drive circuit unit includes, for example, a vertical selection circuit that sequentially selects pixels to be driven in a vertical direction in units of horizontal line (row), a horizontal selection circuit that sequentially selects pixels to be driven in units of column, and a timing generator (TG) circuit that drives the selection circuits with several types of pulses.
- the pixel signal processing circuit unit includes such circuits as an analog-to-digital conversion circuit that converts analog electric signals from the pixel region to digital signals, a gain adjustment and amplifier circuit that performs gain adjustment and amplifying operations, and a digital signal processing circuit that corrects the digital signals.
- the ISP 20 includes a camera module interface (I/F) 22 , an image capturing unit 24 , a signal processing unit 26 , and a driver I/F 28 .
- a RAW image obtained by the imaging operation performed by the imaging module unit 10 is captured through the camera module I/F 22 into the image capturing unit 24 .
- the signal processing unit 26 also performs a signal processing operation on the RAW image captured into the image capturing unit 24 .
- the driver I/F 28 outputs, to a display driver that is not shown, an image signal having been subjected to the signal processing operation at the signal processing unit 26 .
- the display driver displays the image formed by the solid state imaging device.
- FIG. 2 shows a cross-section of the solid state imaging device 1 according to the first embodiment.
- the imaging element 16 of the solid state imaging device 1 according to the first embodiment includes a semiconductor substrate 16 a , pixels 16 b , color filters 16 c , and microlenses 16 d for collecting light for the pixels.
- the pixels 16 b are formed on the semiconductor substrate 16 a , and have photodiodes.
- Also on the semiconductor substrate 16 a is formed a drive/readout circuit (not shown) that drives the pixels 16 b to read the signals from the pixels 16 b.
- the color filters 16 c including R filters (having a high transmittance for light in the red wavelength range), G filters (having a high transmittance for light in the green wavelength range), and B filters (having a high transmittance for light in the blue wavelength range) are formed above the pixels 16 b , each color filter corresponding to one pixel 16 b .
- the color filters 16 c may include W filters (that passes lights in the red, green and blue wavelength ranges).
- the microlenses 16 d for use in collecting light for pixel are formed so as to correspond to the respective pixels.
- the microlenses 16 d are not necessarily formed.
- a microlens array 14 is arranged above the color filters 16 c .
- the microlens array 14 includes a microlens unit 14 a formed on a visible light transmitting substrate 14 b .
- the microlens unit 14 a is arranged on the imaging element 16 side viewed from the visible light transmitting substrate 14 b , and includes a plurality of microlenses arranged in a two-dimensional array form.
- the arrangement in the array is a square arrangement that has four-fold (90 degrees) symmetry with respect to the center of the array, or hexagonal arrangement that has six-fold (60 degrees) symmetry with respect to the center of the array.
- Each microlens corresponds to a pixel block including a plurality of pixels formed on the semiconductor substrate 16 a , and functions as an optical system for forming a reduced image on the corresponding pixel block.
- the visible light transmitting substrate 14 b is aligned in the optical axis direction (the vertical direction in FIG. 2 ) and bonded with the semiconductor substrate 16 a by a spacer 42 of a resin material formed around the imaging region, on which the pixels 16 b are formed. In bonding the semiconductor substrate 16 a with the visible light transmitting substrate 14 b , the alignment is performed using the alignment markings etc.
- the visible light transmitting substrate 14 b can be formed of a material having a function of, for example, cutting off unnecessary near infrared light in addition to the function of transmitting visible light, or can include a multi-layer film or single-layer film that transmit visible light but reflects near infrared light.
- an optical filter 40 may be formed above the microlens array 14 .
- an optical filter 40 having a function of cutting off near infrared light is provided.
- Electrode pad 44 for reading the pixels 16 b are formed in the semiconductor substrate 16 a , and penetrating electrodes 46 are formed under the electrode pads 44 , the penetrating electrodes 46 penetrating the semiconductor substrate 16 a and connecting to a processing and driving chip 50 under the imaging element 16 .
- the semiconductor substrate 16 a is electrically connected to the processing and driving chip 50 via the penetrating electrodes 46 and bumps 48 formed between the penetrating electrodes 46 and the processing and driving chip 50 .
- the processing and driving chip 50 includes a drive and processing circuit (the imaging circuit 18 shown in FIG. 1 ) that drives the imaging element 16 and processes the read signals.
- the semiconductor substrate 16 a and the processing and driving chip 50 may be electrically connected with each other not only by the penetrating electrodes 46 but also by metal wires etc. connecting the electrode pads formed on the both sides.
- an imaging lens 12 that includes a plurality of lens elements and serves as an imaging optics is arranged.
- the imaging lens 12 is attached to a lens barrel 62 that is attached to a lens holder 64 .
- the position for mounting the imaging lens 12 may be adjusted based on the relationship between the pressing force and the output image when the lens barrel 62 is attached to the lens holder 64 .
- a light shielding cover 52 is attached to the surrounding portions (side surfaces) of the processing and driving chip 50 , the semiconductor substrate 16 a , the visible light transmitting substrate 14 b , and the lens holder 64 .
- Module electrodes 54 that electrically connect the processing and driving chip 50 to the outside are arranged below the processing and driving chip 50 .
- FIGS. 3( a ) and 3( b ) The relationship (relationship in geometric optics) between the distance to the subject and the rays of light in the optical system (virtual image optical system) of the solid state imaging device 1 according to the first embodiment will be described below with reference FIGS. 3( a ) and 3( b ) .
- FIGS. 3( a ) and 3( b ) For simplification, it is assumed herein that the expressions relating to the imaging hold in the paraxial region, and the drawings only illustrate the area near the optical axis.
- FIG. 3( a ) illustrates the solid state imaging device 1 in its entirety.
- FIG. 3( b ) is an extended view of the microlens region shown in FIG. 3( a ) .
- a principal ray 80 from a subject 100 on the optical axis and peripheral rays 82 which are homologous rays of the principal ray 80 , form an image on a virtual imaging plane 70 determined by the effective focal length f of the imaging optics 12 and the distance A between the imaging optics 12 and the subject 100 so as to meet the relationship defined by the expression (1):
- the image magnification (lateral magnification) M of the imaging lens 12 can be expressed by the following expression:
- the virtual imaging plane 70 of the imaging lens 12 is placed after the imaging element 16 (on the side opposite to the subject 100 ). Since the microlenses 14 a are arranged before the virtual imaging plane 70 , the light rays are focused on the plane of the imaging element 16 . This means that the light rays 80 , 82 are caused to form reduced images based on a virtual image relationship.
- the imaging system of the microlenses 14 a can be expressed by the following expression:
- N D C ( 4 )
- the pitch or the distance between the centers of two adjacent microlenses 14 a is denoted by L ML .
- L ML The pitch or the distance between the centers of two adjacent microlenses 14 a.
- f, E, and L ML are the parameters that are determined in design, and therefore are known fixed values.
- the variation in D is very slight relative to the variation in A. Accordingly, D can be regarded as a fixed value D 0 .
- D 0 denotes the distance from the image side principal plane of each microlens 14 a to the plane of the imaging element 16 . Therefore, ⁇ is uniquely determined by A.
- the expression (7) can be transformed into the following expression (8).
- the subject distance A can be calculated if the image shift amount ⁇ can be detected from the imaging element 16 .
- an image matching process is performed on the images formed by the adjacent imaging microlenses 14 a recorded by the element 16 .
- the image matching process can be performed by using a known template matching method for checking, for example, the degree of similarity or dissimilarity between two images.
- the shift position can be obtained more accurately by interpolating the degree of similarity or dissimilarity with a continuing fitting function etc. pixel by pixel to obtain the sub-pixel position for providing a maximum or minimum of the fitting function. In this manner, the shift amount can be obtained more accurately.
- FIGS. 4( a ) to 4( c ) A method of reconstructing a two-dimensional image without any overlap from a group of microlens images for the same subject taken at several points (p 1 , p 2 , p 3 , . . . ) will be described with reference to FIGS. 4( a ) to 4( c ) .
- FIG. 4( a ) shows the relationship between the rays of light and the distance from the optical system (virtual optical system) of the solid state imaging device 1 according to the first embodiment to the subject.
- FIGS. 4( b ) and 4( c ) explain the relationships of the overlapped visual fields among adjacent microlenses.
- microlenses form microlens images 91 a , 91 b , 91 c , respectively, on the imaging element 16 , as shown in FIGS. 4( b ) and 4( c ) .
- the effective f-number of the imaging lens 12 should match the effective f-number of the microlenses 14 a.
- the visual fields of the microlens images 91 a , 91 b , 91 c correspond to the visual fields 93 a , 93 b , 93 c on the virtual imaging plane 70 , which overlap each other as shown in FIG. 4( c ) .
- the image magnification N can be obtained from the group of microlens images, using the relationship between the expressions (4) and (6) to derive the expression (9).
- the pitch L ML of the microlenses 14 a is known, if the shift amount ⁇ of the same subject 100 is obtained from the images, the image magnification N can be determined.
- step S 1 a compound eye RAW image is derived from the imaging element 16 .
- the compound eye RAW image is in a state as outputted from the imaging element 16 .
- FIG. 5( b ) shows an example thereof.
- White balancing for adjusting the signal balance among the B, G, and R pixels is performed on the compound eye RAW image using a known technique (step S 2 ).
- demosaicing is performed in which, since there is no signal information of G and B pixels at the positions of R pixels for example, the G and B signals are estimated and produced by referring to the pixels arranged in the surrounding portions (step S 3 ).
- a process for obtaining an average value from the surrounding pixels is performed.
- Various other methods, including the one broadening the range of pixels to be referred to, can be used depending on the necessity (see FIG. 5( c ) ).
- the demosaicing is also performed on the G pixels and B pixels.
- the image points p 1 , p 2 , . . . , p n corresponding to the point P of the subject 100 shown in FIGS. 3( a ) and 3( b ) are recorded as pixel signal values S p1 , S p2 , . . . , S pn in the imaging element 16 .
- the pixel signal values S p1 , S p2 , . . . , S pn are associated with a synthesized signal S′ p in the n-to-one relationship (step S 4 and FIG. 5( d ) ).
- the association is performed by detecting, from the image, the relationship of the shift amount ⁇ among the image points and the relationship of the overlaps among the visual fields. Since ⁇ can be obtained by assuming the subject distance A, ⁇ obtained from the assumed subject distance A can be used alternatively. Thereafter, an image is synthesized by a two-dimensional image synthesis method at step S 5 to obtain a two-dimensional image, and the flow ends.
- FIG. 5( e ) shows an example of the two-dimensional image obtained in such a manner.
- the pixel signal values of n pixels are denoted by S p1 , S p2 , . . . , and S pn
- the noise values of the pixels are denoted by N p1 , N p2 , . . . , and N pn .
- luminance correction processing is performed on the respective pixel signal values and noise values. The luminance correction processing is performed when the luminance distribution of the periphery portion is more degraded than that of the other portions in each microlens image.
- S pn are multiplied by luminance correction coefficients a 1 , a 2 , . . . , a n , respectively.
- S′ p The noise value N′ p included in the synthesized signal value S′ p is calculated using the expression (11).
- S′ p ⁇ a 1 S p1 +a 2 S p2 + . . .
- N′ p ⁇ ( a 1 N p1 ) 2 +( a 2 N p2 ) 2 + . . . +( a n N pn ) 2 ⁇ 1/2 /n (11)
- the order of the aforementioned steps may be changed depending on which synthesis method is selected.
- the order of the steps S 1 to S 5 may be arbitrarily changed by, for example, performing the demosaicing after the two-dimensional image synthesis processing.
- Modulation transfer function is a value indicating the degree of modulation and the capability of transferring image frequency of an optical system.
- MTF Modulation transfer function
- the smallest light-collecting spot size determined by the diffraction phenomenon of an optical system is limited.
- a diffraction limited spot size is broadened in terms of waves even with an ideal aplanatic lens, and the spot size has a limit at a finite size.
- the bottom of the zero-order diffraction intensity of an aplanatic lens with a circular aperture is in a circular shape, and called “Airy disc.”
- the value of MTF decreases as the image frequency u increases, and reaches 0 at an image frequency determined by the diffraction-limited spot-size.
- This image frequency is called “cutoff frequency” u c , and can be expressed by the following expression (13).
- FIG. 6 shows, in the graphs g 1 -g 9 , the frequency characteristics of the MTF values obtained by the expression (13) in the cases where the wavelength ⁇ is 470 nm, 555 nm, and 660 nm, and F# is 1.4, 2.8, and 8.0 for each wavelength.
- the graph g 1 shows the case where the wavelength ⁇ is 470 nm and F# is 1.4
- the graph g 2 shows the case where the wavelength ⁇ is 555 nm and F# is 1.4
- the graph g 3 shows the case where the wavelength ⁇ is 660 nm and F# is 1.4
- the graph g 4 shows the case where the wavelength ⁇ is 470 nm and F# is 2.8
- the graph g 5 shows the case where the wavelength ⁇ is 555 nm and F# is 2.8
- the graph g 6 shows the case where the wavelength ⁇ is 660 nm and F# is 2.8.
- the graph g 7 shows the case where the wavelength ⁇ is 470 nm and F# is 8.0
- the graph g 8 shows the case where the wavelength ⁇ is 555 nm and F# is 8.0
- the graph g 9 shows the case where the wavelength ⁇ is 660 nm and F# is 8.0. Since the MTF value decreases as the wavelength becomes longer, or F# becomes greater, in order to obtain the resolution covering the red region with the wavelength of 600 nm or more, an optical design with a low F# is needed, as can be understood from FIG. 6 .
- the second factor for limiting the MTF value obtainable by an imaging device is the pixel pitch of the imaging element.
- FIGS. 7( a ) to 7( c ) explain the sensor sampling limit.
- FIG. 7( a ) shows the input (light) and the sensor output (electrical signal) when the spatial frequency is f n /2, FIG.
- FIG. 7( b ) shows the input and the sensor output when the spatial frequency is f n
- FIG. 7( c ) shows the input and the sensor output when the spatial frequency is 2f n
- the Nyquist frequency f n is 357 LP/mm.
- f n is 455 LP/mm.
- FIG. 8 shows the MTF characteristic of the main lens of the solid state imaging device according to the first embodiment and the change in image frequency (spatial frequency) when the image formed by the main lens is reduced by a microlens (hereinafter referred to as “ML”).
- ML microlens
- FIG. 8 shows the relationship when the reduction rate N by the microlens is 1/2.5. At this time, the frequency after the reduction by the microlens increases by 1/N times (2.5 times).
- FIG. 9 shows the image frequency on the sensor plane (image plane), which is the sampling plane.
- image plane which is the sampling plane.
- the frequency reproducibility (MTF) of the microlens is limited by the optical diffraction limit and the aberrations as described above. Accordingly, a low-pass effect, with which only images having a frequency lower than a certain frequency can be reproduced, occurs in the microlens as shown in FIG. 9 , and the increase in frequency is limited.
- the graphs g 2 and g 4 shown in FIG. 9 are calculated with the wavelength ⁇ being 555 nm, and the F# of the microlens being 3.5.
- FIG. 10 shows the image frequency characteristic of a reconstructed image.
- a process is performed to enlarge and add the microlens images to recover the reduction performed by the microlenses, as described above. Accordingly, after the enlargement, the frequency of the reconstructed image is reduced.
- MTF Main (u) of the main lens is limited by MTF ML (u) of the microlens.
- an ideal design could be made by setting MTF Main (u) of the main lens to be lower than MTF ML (u) of the microlens, and setting MTF Main (u) of the main lens after the reduction to be higher than MTF ML (u) of the microlens.
- Designing MTF ML (u) of the main lens to be lower means not requiring the respective aberration amounts as strictly as in the conventional cases. Alleviating the aberration requirement leads to increasing the degree of freedom in lens design. Therefore, designs with a smaller number of lenses and a lower F# are made possible, which has conventionally been impossible.
- FIG. 11 shows the relationship between MTF and the image frequency in the solid state imaging device according to the first embodiment. It is possible to design MTF Main (u) of the main lens to be lower than MTF ML (u) of the microlens. However, in order to set MTF Main after the reduction to be higher than MTF ML of the microlens, the following expression (14) must hold. MTF Main ( u ) ⁇ MTF ML ( u ) ⁇ MTF Main ( u ⁇ Nf ) (14) where u denotes the image spatial frequency, and Nf denotes the reduction rate of the microlens.
- Nf is preferably less than 0.87, and more preferably less than 0.5. The reason for this will be described with reference to FIGS. 24( a ) to 24( d ) .
- FIGS. 24( a ) to 24( d ) show the enlargement and the addition of images in the case where the main lens aperture is circular, each microlens image is also circular, and the images are arranged without any gap.
- the images shown in FIGS. 24( a ) to 24( d ) correspond to the images of the imaging element 16 shown in FIGS. 4( a ) to 4( c ) .
- FIG. 24( d ) shows the case where the value of 1/Nf is 2.00.
- the value of 1/Nf be more than 2, i.e., Nf be 0.5 or less.
- the expression (14) holds for any image height and any wavelength even if the imaging performance of the main lens degrades.
- the expression (14) should hold in the wavelength range of 380 nm to 780 nm, or in the wavelength range of light passing through the color filters 16 c .
- the color filters 16 c include R (red), G (green) and B (blue) color filters
- the expression (14) should hold in the wavelength range of more than the peak wavelength of light passing through the B color filters, and less than the peak wavelength of light passing through the R color filters.
- the ideal image height meets the conditions of the expression up to 100%, which corresponds to the four corners of the imaging area, when the center of the imaging area (center of the optical axis) is 0%.
- a case is also preferable, where the ratio of the vertical line and the horizontal line of the rectangular of the sensor area is 3:4, and the expression holds for up to 80% of the image height with which the horizontal width of the region is covered since a good characteristic can be exerted in most of the screen.
- the wavelength range covers 380 nm to 780 nm to cover the visible light wavelength range.
- the aperture of the main lens is preferably in a circular shape, and Nf is preferably 0.001 or more and 0.87 or less in the first embodiment.
- the aperture of the main lens is also preferably in a hexagonal shape, and Nf in this case is preferably 0.001 or more and 1 or less. If the aperture of the main lens is in a hexagonal shape, the shape of a microlens image is also in a hexagonal shape. Accordingly, the microlens images are arranged without any space. If such images are expanded even slightly, adjacent images overlap each other. Therefore, the aforementioned partial image matching using the periphery portions of the microlens images can be performed.
- each lens has a simple three-lens-element in three-group structure (L 1 , L 2 , L 3 ), each lens element being a spherical lens. It is assumed that the optical axis direction of the lens is the z-axis direction, and the normal directions relative to the optical axis are the y-axis direction (vertical direction in the drawing) and the x-axis direction (the direction perpendicular to the drawing surface), as shown in FIG. 12 .
- the positive direction on the z-axis is a direction from the left side (subject side) of the main lens group to the right side (image plane).
- c the curvature of each lens plane (1/mm)
- z′ 0.
- FIG. 13 shows examples of numerical values of the specific example.
- n d denotes the refractive index with respect to the d-line (587.6 nm) of the optical material forming the lens
- v d denotes the Abbe number with respect to the d-line of the material forming the lens
- R denotes the effective radius (mm), i.e., the radius of the circular region through which the light rays pass
- f denotes the focal length (mm).
- the lenses having the shapes represented by the values in FIG. 13 are arranged on a plane.
- FIG. 14 shows the white color MTF characteristic of the optical system shown in FIGS. 12 and 13 with the angles of view of 0 degree and 10 degrees.
- the MTF characteristic means the value in the white region (MTF obtained with respect to the range of 0.4047 ⁇ m-0.6563 ⁇ m).
- the 10-degree angle of view in the tangential plane is expressed by “T,” and in the sagittal plane is expressed by “S.”
- T The 10-degree angle of view in the tangential plane
- S in the sagittal plane
- the respective aberrations that could not have been fully corrected by the imaging optical lens (main lens) are reduced in both the vertical direction and the horizontal direction due to the optical image reduction by means of the microlens array in the vertical (z) direction and the horizontal (x, y) directions. Accordingly, at the 0-degree angle of view and the 10-degree angle of view in the sagittal plane (S), the performance substantially reaches the diffraction limit.
- FIG. 15 shows the MTF through focus characteristic of the optical system shown in FIGS. 12 and 13 .
- the through focus characteristic indicates the change in MTF at a frequency when the imaging plane position (imaging element position) is shifted.
- FIG. 15 shows the characteristic values at 89 LP/mm (1/4 Nyquist frequency of the pixels with the pitch of 1.4 ⁇ m) when the angle of view is 0 degree and 10 degrees.
- FIG. 16 also shows the MTF through focus characteristic of the optical system shown in FIGS. 12 and 13 .
- the characteristic values are at 179 LP/mm (1/2 Nyquist frequency of the pixels with the pitch of 1.4 ⁇ m) when the angle of view is 0 degree and 10 degrees.
- FIG. 17 shows an optical system of a comparative example.
- no microlens array is arranged after the third lens of the optical system shown in FIG. 12 .
- the optical system of the comparative example only has an imaging optical lens (main lens) and does not have any microlens array. Since no microlens array is included, the light rays converge to one point, and form an image there.
- FIG. 18 shows examples of numerical values of the comparative example.
- FIG. 19 shows the MTF characteristic of the optical system of the comparative example (with no MLA) at the angles of view of 0 degree and 10 degrees.
- the 10-degree angle of view in the tangential plane is expressed by “T,” and in the sagittal plane is expressed by “S.”
- T The 10-degree angle of view in the tangential plane
- S in the sagittal plane
- the values of the MTF characteristic are lower than the values of the diffraction limit performance, i.e., lower than the values of the specific example of the first embodiment shown in FIG. 14 .
- the increase in MTF as shown in FIG. 8 can, be achieved by inserting a microlens array and reducing and re-forming images as in the first embodiment.
- FIG. 20 shows the MTF through focus characteristic of the optical system of the comparative example (with no MLA) at 89 LP/mm (1/4 Nyquist frequency of pixels with the pitch of 1.4 ⁇ m) when the angle of view is 0 degree and 10 degrees.
- FIG. 21 also shows the MTF through focus characteristic of the optical system of the comparative example (with no MLA) at 179 LP/mm (1/2 Nyquist frequency of pixels with the pitch of 1.4 ⁇ m) when the angle of view is 0 degree and 10 degrees.
- the MTF values of the comparative example are lower than those of the specific example of the first embodiment. It can also be understood from this that the MTF values can be increased by inserting a microlens array and reducing and re-forming images as in the first embodiment.
- the MTF value at the 1/4 Nyquist frequency (89 LP/mm) for the image height of 0 degree of the specific example of the first embodiment including the microlens array increases from 59% to 79% as compared to the MTF value of the comparative example including the same main lens, and the MTF value at the 1/2 Nyquist frequency (179 LP/mm) increases from 34% to 59%.
- a small lens for portable use requires a relatively wide angle of view, a thinner size (lower profile), and a higher resolution. Accordingly, the number of lens elements tends to increase to correct aberrations and control incident light angle.
- the optical system according to the first embodiment re-reduces an image by means of the microlens array arranged after the imaging optics (main lens) to increase the image frequency. Accordingly, the main lens does not necessarily have a high frequency image reproducibility like conventional cases, which increases the degree of freedom in design. As a result, the optical system according to the first embodiment can be formed with a lower number of lens elements.
- the number of lens elements in optical systems can be reduced, and the degree of freedom in designing the main lens is increased due to the relaxation in requirement of high frequency image reproducibility. Accordingly, a slim design can be made. Moreover, since the light rays from the main lens are converged by the microlenses, the length of the entire optical system can be shortened as compared to the case of the comparative example including only the main lens and the imaging element.
- the microlenses have a geometric image reduction function with respect to an image formed by the main lens. Accordingly, the optical system has an effect of reducing geometric aberrations. Therefore, it is possible to prevent the increase in aberration on the sensor image plane with the main lens being designed to have a low F#.
- the first embodiment it is possible to provide a solid state imaging device that can be easily designed or manufactured even if the high resolution imaging module is downsized.
- FIG. 22 shows a portable information terminal according to the second embodiment.
- the portable information terminal 200 according to the second embodiment includes the solid state imaging device according to the first embodiment.
- the portable information terminal shown in FIG. 22 is just an example, and the reference numeral 10 denotes the imaging module of the solid state imaging device according to the first embodiment.
- the solid state imaging device of the first embodiment can be applied not only to portable information terminals but also to still cameras.
- the second embodiment it is possible to provide a portable information terminal that can be designed or manufactured easily even if the high resolution imaging module is downsized.
- FIG. 23 shows a solid state imaging system according to the third embodiment.
- the solid state imaging system according to the third embodiment includes a plurality of imaging module units 10 1 - 10 m and an imaging signal processor (ISP) 20 A.
- ISP imaging signal processor
- the IPS 20 A has a configuration obtained by replacing the camera module interface (I/F) 22 of the IPS 20 in the first embodiment with a camera module interface (I/F) 22 A.
- the ISP 20 A includes the camera module I/F 22 A, an image capturing unit 24 , a signal processing unit 26 , and a driver I/F 28 .
- the signal processing unit 26 performs a signal processing operation on the RAW image captured by the image capturing unit 24 .
- the driver I/F 28 outputs image signals having been subjected to the signal processing performed by the signal processing unit 26 to a display driver that is not shown in FIG. 23 .
- the display driver displays the image formed by the solid state imaging device.
- distance information either by using a single imaging module, e.g., imaging module unit 10 1 , or by performing stereo measurement using two or more imaging modules.
- the accuracy of distance information can be improved by, for example, obtaining distance information in a short distance or narrow area with a single module, and distance information in a long distance by performing measurement using two or more imaging modules.
- the optical system according to the third embodiment re-reduces an image with the microlens array arranged after the imaging optics (main lens) to increase the image frequency. Accordingly, the main lens is not required to have high frequency image reproducibility as in the conventional cases. This improves the degree of freedom in design. As a result, the optical system can be formed with a lower number of lens elements.
- the number of lens elements of each lens in the optical system according to the third embodiment can be reduced, and the requirement of high-frequency image reproducibility is relaxed, which improves the degree of freedom in designing the main lens. As a result, a slim design can be made. Moreover, since the light rays from the main lens are converged by means of the microlenses, the entire length of the optical system can be shortened as compared to an optical system including only a main lens and an imaging element like the comparative example.
- the microlens in the third embodiment has a geometric image-reducing function on an image formed by the main lens, which provides an effect of reducing geometric aberrations. It is therefore possible to prevent an increase in aberration on the sensor image plane while designing the main lens to have a low F#.
- the third embodiment it is possible to provide a solid state imaging system capable of being designed or manufactured even if the high resolution imaging module is downsized, like the first embodiment.
- an imaging device capable of obtaining a visible image and a depth image at a time with a single device.
- the downsizing of the device makes it possible to mount the device on various apparatuses, which contributes to the downsizing of such products as portable information terminals and home appliances.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Engineering & Computer Science (AREA)
- Solid State Image Pick-Up Elements (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Lenses (AREA)
- Automatic Focus Adjustment (AREA)
- Focusing (AREA)
- Optical Elements Other Than Lenses (AREA)
Abstract
Description
where f denotes the effective focal length of the
where g denotes the focal length of each microlens 14 a, C denotes the distance from the object side principal plane of each microlens 14 a to the virtual imaging plane, and D denotes the distance from the image side principal plane of each microlens 14 a to the imaging point thereof. At this time, the image magnification of the imaging system of the
E=B−C (5)
S′ p ={a 1 S p1 +a 2 S p2 + . . . +a n S pn }/n (10)
N′ p={(a 1 N p1)2+(a 2 N p2)2+ . . . +(a n N pn)2}1/2 /n (11)
ra=1.22λF
where λ denotes the wavelength of light, and F denotes the F-number.
MTF Main(u)≦MTF ML(u)≦MTF Main(u×Nf) (14)
where u denotes the image spatial frequency, and Nf denotes the reduction rate of the microlens.
Claims (12)
MTF Main(u)≦MTF ML(u)≦MTF Main(u×Nf)
MTF Main(u)≦MTF ML(u)≦MTF Main(u×Nf)
MTF Main(u)≦MTF ML(u)≦MTF Main(u×Nf)
MTF Main(u)≦MTF ML(u)≦MTF Main(u×Nf)
MTF Main(u)≦MTF ML(u)≦MTF Main(u×Nf)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013035946A JP2014164174A (en) | 2013-02-26 | 2013-02-26 | Solid-state image pickup device, portable information terminal and solid-state imaging system |
JP2013-035946 | 2013-02-26 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20140240559A1 US20140240559A1 (en) | 2014-08-28 |
US9300885B2 true US9300885B2 (en) | 2016-03-29 |
Family
ID=50230852
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/162,122 Expired - Fee Related US9300885B2 (en) | 2013-02-26 | 2014-01-23 | Imaging device, portable information terminal, and imaging system |
Country Status (3)
Country | Link |
---|---|
US (1) | US9300885B2 (en) |
EP (1) | EP2770727A3 (en) |
JP (1) | JP2014164174A (en) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9188839B2 (en) * | 2012-10-04 | 2015-11-17 | Cognex Corporation | Component attachment devices and related systems and methods for machine vision systems |
JP6230222B2 (en) * | 2012-10-12 | 2017-11-15 | キヤノン株式会社 | Lens array optical system and method of manufacturing lens array |
JP2014099696A (en) * | 2012-11-13 | 2014-05-29 | Toshiba Corp | Solid state image pickup device |
JP6189061B2 (en) | 2013-03-19 | 2017-08-30 | 株式会社東芝 | Solid-state imaging device |
JP2015185947A (en) * | 2014-03-20 | 2015-10-22 | 株式会社東芝 | imaging system |
US20150346115A1 (en) * | 2014-05-30 | 2015-12-03 | Eric J. Seibel | 3d optical metrology of internal surfaces |
WO2017013054A1 (en) * | 2015-07-17 | 2017-01-26 | Leica Microsystems Cms Gmbh | Light sheet microscope for simultaneously imaging a plurality of object planes |
KR102547821B1 (en) * | 2016-11-25 | 2023-06-26 | 삼성전자주식회사 | 3d display apparatus |
CN108240829B (en) * | 2016-12-23 | 2022-01-21 | 奥特斯奥地利科技与系统技术有限公司 | Sensor system |
JP7030592B2 (en) * | 2018-03-27 | 2022-03-07 | キヤノン株式会社 | Distance measuring device, imaging device, mobile device, robot device, program |
EP3936837A4 (en) * | 2019-03-06 | 2022-04-27 | Panasonic Intellectual Property Management Co., Ltd. | Light detection device, light detection system, and filter array |
WO2021074861A1 (en) * | 2019-10-17 | 2021-04-22 | 3M Innovative Properties Company | Optical system and methods including microlenses and light-blocking layer |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5485204A (en) * | 1993-08-02 | 1996-01-16 | Nec Corporation | Solid state image sensor and imaging method using spatial pixel offset |
US20040223229A1 (en) * | 2003-05-06 | 2004-11-11 | Hiromi Ishikawa | Projecting exposure apparatus |
US20060139476A1 (en) * | 2004-11-22 | 2006-06-29 | Toru Sasaki | Optical low pass filter and image pickup apparatus having the same |
JP2007322844A (en) | 2006-06-02 | 2007-12-13 | Nidec Copal Corp | Imaging lens |
US20080118867A1 (en) * | 2004-05-12 | 2008-05-22 | Morimasa Sato | Pattern Forming Material, Pattern Forming Apparatus, And Pattern Forming Process |
US20100165134A1 (en) * | 2006-04-17 | 2010-07-01 | Dowski Jr Edward R | Arrayed Imaging Systems And Associated Methods |
JP2011232772A (en) | 2008-08-25 | 2011-11-17 | Konica Minolta Opto Inc | Imaging lens, imaging apparatus and mobile tarminal |
US20120218448A1 (en) | 2011-02-25 | 2012-08-30 | Kabushiki Kaisha Toshiba | Solid-state imaging device and portable information terminal |
US20120229683A1 (en) * | 2011-03-08 | 2012-09-13 | Kabushiki Kaisha Toshiba | Solid-state imaging device and portable information terminal |
US20130240709A1 (en) * | 2012-03-15 | 2013-09-19 | Risako Ueno | Solid-state imaging device and portable information terminal |
US20130242161A1 (en) * | 2012-03-15 | 2013-09-19 | Mitsuyoshi Kobayashi | Solid-state imaging device and portable information terminal |
US20140285693A1 (en) * | 2013-03-19 | 2014-09-25 | Kabushiki Kaisha Toshiba | Microlens array unit and solid state imaging device |
US9100635B2 (en) * | 2012-06-28 | 2015-08-04 | Pelican Imaging Corporation | Systems and methods for detecting defective camera arrays and optic arrays |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0229072A (en) * | 1988-04-14 | 1990-01-31 | Ricoh Co Ltd | Picture correction device for digital picture processing unit |
JP4538766B2 (en) * | 2008-08-21 | 2010-09-08 | ソニー株式会社 | Imaging device, display device, and image processing device |
JP5634946B2 (en) * | 2011-06-06 | 2014-12-03 | 富士フイルム株式会社 | Imaging device |
US8998411B2 (en) * | 2011-07-08 | 2015-04-07 | Carl Zeiss Meditec, Inc. | Light field camera for fundus photography |
-
2013
- 2013-02-26 JP JP2013035946A patent/JP2014164174A/en active Pending
-
2014
- 2014-01-23 US US14/162,122 patent/US9300885B2/en not_active Expired - Fee Related
- 2014-02-19 EP EP14155723.1A patent/EP2770727A3/en not_active Withdrawn
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5485204A (en) * | 1993-08-02 | 1996-01-16 | Nec Corporation | Solid state image sensor and imaging method using spatial pixel offset |
US20040223229A1 (en) * | 2003-05-06 | 2004-11-11 | Hiromi Ishikawa | Projecting exposure apparatus |
US20080118867A1 (en) * | 2004-05-12 | 2008-05-22 | Morimasa Sato | Pattern Forming Material, Pattern Forming Apparatus, And Pattern Forming Process |
US20060139476A1 (en) * | 2004-11-22 | 2006-06-29 | Toru Sasaki | Optical low pass filter and image pickup apparatus having the same |
US20100165134A1 (en) * | 2006-04-17 | 2010-07-01 | Dowski Jr Edward R | Arrayed Imaging Systems And Associated Methods |
JP2007322844A (en) | 2006-06-02 | 2007-12-13 | Nidec Copal Corp | Imaging lens |
JP2011232772A (en) | 2008-08-25 | 2011-11-17 | Konica Minolta Opto Inc | Imaging lens, imaging apparatus and mobile tarminal |
US20120218448A1 (en) | 2011-02-25 | 2012-08-30 | Kabushiki Kaisha Toshiba | Solid-state imaging device and portable information terminal |
US20120229683A1 (en) * | 2011-03-08 | 2012-09-13 | Kabushiki Kaisha Toshiba | Solid-state imaging device and portable information terminal |
US20130240709A1 (en) * | 2012-03-15 | 2013-09-19 | Risako Ueno | Solid-state imaging device and portable information terminal |
US20130242161A1 (en) * | 2012-03-15 | 2013-09-19 | Mitsuyoshi Kobayashi | Solid-state imaging device and portable information terminal |
US9100635B2 (en) * | 2012-06-28 | 2015-08-04 | Pelican Imaging Corporation | Systems and methods for detecting defective camera arrays and optic arrays |
US20140285693A1 (en) * | 2013-03-19 | 2014-09-25 | Kabushiki Kaisha Toshiba | Microlens array unit and solid state imaging device |
Non-Patent Citations (2)
Title |
---|
K. Fife et al., "A 3D multi-aperture image sensor architecture", Custom Integrated Circuits Conference, Sep. 2006, pp. 281-284, 4 pages. |
U.S. Appl. No. 14/165,925, filed Jan. 28, 2014, Kobayashi, et al. |
Also Published As
Publication number | Publication date |
---|---|
US20140240559A1 (en) | 2014-08-28 |
EP2770727A2 (en) | 2014-08-27 |
EP2770727A3 (en) | 2017-03-01 |
JP2014164174A (en) | 2014-09-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9300885B2 (en) | Imaging device, portable information terminal, and imaging system | |
US7924327B2 (en) | Imaging apparatus and method for producing the same, portable equipment, and imaging sensor and method for producing the same | |
US9048157B2 (en) | Solid-state imaging device and portable information terminal | |
US7718940B2 (en) | Compound-eye imaging apparatus | |
TWI534465B (en) | Imaging lens and solid state imaging device | |
US20130222546A1 (en) | Solid-state image pickup element and image pickup apparatus | |
JP5619294B2 (en) | Imaging apparatus and focusing parameter value calculation method | |
US8937274B2 (en) | Solid-state imaging device and portable information terminal having a proportion of W pixels increases toward an outer periphery of each pixel block | |
CN110636277B (en) | Detection apparatus, detection method, and image pickup apparatus | |
JP2011176715A (en) | Back-illuminated image sensor and imaging apparatus | |
KR20170032227A (en) | Compound-eye imaging device | |
JP2014057231A (en) | Imaging apparatus | |
US11889186B2 (en) | Focus detection device, focus detection method, and image capture apparatus | |
JP2015210329A (en) | Array optical system | |
WO2014196322A1 (en) | Compound eye imaging optical system and compound eye imaging device | |
JP5568150B2 (en) | Solid-state imaging device and portable information terminal | |
JPH01119178A (en) | Image pickup device | |
JP5978570B2 (en) | Imaging device | |
JP2017026814A (en) | Imaging optical system and imaging system | |
JP2015111782A (en) | Solid-state image sensor and imaging device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:UENO, RISAKO;HONDA, HIROTO;KOBAYASHI, MITSUYOSHI;AND OTHERS;SIGNING DATES FROM 20140109 TO 20140113;REEL/FRAME:032031/0287 |
|
ZAAA | Notice of allowance and fees due |
Free format text: ORIGINAL CODE: NOA |
|
ZAAB | Notice of allowance mailed |
Free format text: ORIGINAL CODE: MN/=. |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20240329 |