US20150077524A1 - Image sensor and imaging device - Google Patents

Image sensor and imaging device Download PDF

Info

Publication number
US20150077524A1
US20150077524A1 US14/476,367 US201414476367A US2015077524A1 US 20150077524 A1 US20150077524 A1 US 20150077524A1 US 201414476367 A US201414476367 A US 201414476367A US 2015077524 A1 US2015077524 A1 US 2015077524A1
Authority
US
United States
Prior art keywords
film
parallax
photoelectric converter
reflection rate
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/476,367
Other languages
English (en)
Inventor
Satoshi Suzuki
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nikon Corp
Original Assignee
Nikon Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nikon Corp filed Critical Nikon Corp
Assigned to NIKON CORPORATION reassignment NIKON CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SUZUKI, SATOSHI
Publication of US20150077524A1 publication Critical patent/US20150077524A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G02B7/34—Systems for automatic generation of focusing signals using different areas in a pupil plane
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14601—Structural or functional details thereof
    • H01L27/1463—Pixel isolation structures
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14601—Structural or functional details thereof
    • H01L27/1462—Coatings
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14601—Structural or functional details thereof
    • H01L27/1462—Coatings
    • H01L27/14621—Colour filter arrangements
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14601—Structural or functional details thereof
    • H01L27/1462—Coatings
    • H01L27/14623—Optical shielding
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14683—Processes or apparatus peculiar to the manufacture or treatment of these devices or parts thereof
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14683—Processes or apparatus peculiar to the manufacture or treatment of these devices or parts thereof
    • H01L27/14685—Process for coatings or optical elements
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L31/00—Semiconductor devices sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation; Processes or apparatus specially adapted for the manufacture or treatment thereof or of parts thereof; Details thereof
    • H01L31/02—Details
    • H01L31/0216—Coatings
    • H01L31/02161—Coatings for devices characterised by at least one potential jump barrier or surface barrier
    • H01L31/02162—Coatings for devices characterised by at least one potential jump barrier or surface barrier for filtering or shielding light, e.g. multicolour filters for photodetectors
    • H04N13/0282—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
    • H04N13/218—Image signal generators using stereoscopic image cameras using a single 2D image sensor using spatial multiplexing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/257—Colour aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems

Definitions

  • the present invention relates to an image sensor and an imaging device.
  • An imaging device which captures two parallax images having parallax at one imaging using a single image-capturing optical system has been known.
  • Patent Document 1 Japanese Patent Application Publication No. 2003-7994
  • a light-blocking member for blocking incident luminous fluxes is provided for each pixel, for generating a parallax image.
  • the light-blocking member is provided apart from the photoelectric converter element, and so unnecessary light such as diffracted light generated at the boundary between the light-blocking member and the aperture portion sometimes reaches the photoelectric converter element.
  • a first aspect of the innovations may include an image sensor including: photoelectric converter elements aligned two dimensionally, and photoelectric converting incident light into an electric signal; and reflection rate adjusted films, each of which is formed on a light receiving surface of a photoelectric converter element of at least a part of the photoelectric converter elements and at least includes a first portion having a first reflection rate and a second portion having a second reflection rate different from the first reflection rate.
  • a second aspect of the innovations may include imaging device, including: the image sensor described above; and an image processor that generates, from an output of the image sensor, a plurality of pieces of parallax image data having parallax to each other and 2D no-parallax image data.
  • a third aspect of the innovations may include a method of manufacturing reflection rate adjusted films formed on light receiving surfaces of photoelectric converter elements aligned two dimensionally and photoelectric converting incident light into an electric signal, the method including: depositing a first film on a substrate on which the photoelectric converter elements are formed; adjusting a film thickness of the first film so that a first portion and a second portion resulting from dividing a light receiving surface of each of the photoelectric converter elements have film thicknesses different from each other; depositing a second film different from the first film, on the first film; and adjusting a film thickness of the second film so that the first portion and the second portion have film thicknesses different from each other.
  • a fourth aspect of the innovations may include a method of manufacturing reflection rate adjusted films formed on light receiving surfaces of photoelectric converter elements aligned two dimensionally and photoelectric converting incident light into an electric signal, the method including: depositing a first film on a substrate on which the photoelectric converter elements are formed; masking a first portion, out of the first portion and a second portion resulting from dividing a light receiving surface of each of the photoelectric converter elements; etching the first film; depositing a second film different from the first film, on the first film; masking one of the first portion and the second portion; and etching the second film.
  • FIG. 1 explains a configuration of a digital camera according to the present embodiment.
  • FIG. 2 is an overview showing a cross section of an image sensor according to the present embodiment.
  • FIG. 3A explains a configuration of a reflection rate adjusted film according to the present embodiment.
  • FIG. 3B explains a configuration of a reflection rate adjusted film according to the present embodiment.
  • FIG. 4 is an overview showing an enlarged view of a part of an image sensor.
  • FIGS. 5A-5C are conceptual diagrams explaining the relation between a parallax pixel and a subject.
  • FIG. 6 is a conceptual diagram explaining a process of generating a parallax image.
  • FIG. 7 explains a Bayer array.
  • FIG. 8 explains an array of a repeating pattern 110 in a first embodiment example.
  • FIG. 9 explains an array of a repeating pattern 110 in a second embodiment example.
  • FIG. 10 explains an example of a generation process of RGB plane data as 2D image data.
  • FIG. 11 explains an example of a generation process of two pieces of G plane data as parallax image data.
  • FIG. 12 explains an example of a generation process of two pieces of B plane data as parallax image data.
  • FIG. 13 explains an example of a generation process of two pieces of R plane data as parallax image data.
  • FIG. 14 is a conceptual diagram showing the relation of resolutions of respective planes.
  • FIG. 15 explains a shape of a first portion 106 .
  • FIG. 16 schematically shows a cross section of an image sensor according to a first modification example.
  • FIG. 17 schematically shows a cross section of an image sensor according to a second modification example.
  • FIG. 18A and FIG. 18B explain a configuration of a reflection rate adjusted film adjusted to the incident light characteristic.
  • FIG. 19A and FIG. 19B explain a configuration of a reflection rate adjusted film according to another variation.
  • FIG. 20 shows a process flow according to a first manufacturing process.
  • FIG. 21 shows a process flow according to a second manufacturing process.
  • FIG. 22 shows a simulation result of a reflection rate of each film composition with respect to an incident wavelength.
  • a digital camera relating to the present embodiment which is a form of an image processing apparatus and an imaging device, is configured to be able to produce a plurality of images of a plurality of viewpoints for a single scene, with a single image-capturing operation.
  • the images from different viewpoints are referred to as parallax images.
  • FIG. 1 illustrates the configuration of a digital camera 10 according to the present embodiment.
  • the digital camera 10 includes an image-capturing lens 20 , which is an image-capturing optical system, and guides incoming subject luminous flux along an optical axis 21 to an image sensor 100 .
  • the image-capturing lens 20 may be a replaceable lens that is attachable and detachable to/from the digital camera 10 .
  • the digital camera 10 includes the image sensor 100 , a controller 201 , an A/D converter circuit 202 , a memory 203 , a drive unit 204 , an image processor 205 , a memory card IF 207 , an operating unit 208 , a display 209 , an LCD drive circuit 210 , and an AF sensor 211 .
  • a z-axis positive direction is defined as the direction parallel to the optical axis 21 toward the image sensor 100
  • an x-axis positive direction is defined as the direction toward the viewer of the sheet of FIG. 1 in the plane orthogonal to the z axis
  • a y-axis positive direction is defined as the upward direction in the sheet of FIG. 1 .
  • their coordinate axes show how the respective drawings are arranged relative to the coordinate axes of FIG. 1 .
  • the image-capturing lens 20 is constituted by a group of optical lenses and configured to form an image from the subject luminous flux from a scene in the vicinity of its focal plane.
  • the image-capturing lens 20 is hypothetically represented by a single lens positioned in the vicinity of the pupil in FIG. 1 .
  • the image sensor 100 is positioned in the vicinity of the focal plane of the image-capturing lens 20 .
  • the image sensor 100 is an image sensor having a two-dimensionally arranged photoelectric converter elements, for example, a CCD or CMOS sensor.
  • the timing of the image sensor 100 is controlled by the drive unit 204 so that the image sensor 100 can convert a subject image formed on the light receiving surface into an image signal and outputs the image signal to the A/D converter circuit 202 .
  • the A/D converter circuit 202 converts the image signal output from the image sensor 100 into a digital image signal and outputs the digital image signal to the memory 203 .
  • the image processor 205 uses the memory 203 as its workspace to perform a various image processing operations and thus generates image data.
  • the image processor 205 additionally performs general image processing operations such as adjusting image data in accordance with a selected image format.
  • the produced image data is converted by the LCD drive circuit 210 into a display signal and displayed on the display 209 .
  • the produced image data is stored in the memory card 220 attached to the memory card IF 207 .
  • the AF sensor 211 is a phase detection sensor having a plurality of ranging points set in a subject space and configured to detect a defocus amount of a subject image for each ranging point.
  • a series of image-capturing sequences is initiated when the operating unit 208 receives a user operation and outputs an operating signal to the controller 201 .
  • the various operations such as AF and AE associated with the image-capturing sequences are performed under the control of the controller 201 .
  • the controller 201 analyzes the detection signal from the AF sensor 211 to perform focus control to move a focus lens that constitutes a part of the image-capturing lens 20 .
  • FIG. 2 schematically illustrates the cross-section of the image sensor 100 relating to the present embodiment.
  • the image sensor 100 is structured in such a manner that microlenses 101 , color filters 102 , interconnection layer 103 , an reflection rate adjusted film 105 and photoelectric converter elements 108 are arranged in the stated order when seen from the side facing a subject.
  • the photoelectric converter elements 108 are formed by photodiodes that may convert incoming light into an electrical signal.
  • the photoelectric converter elements 108 are arranged two-dimensionally on the surface of a substrate 109 .
  • the image signals produced by the conversion performed by the photoelectric converter elements 108 , control signals to control the photoelectric converter elements 108 and the like are transmitted and received via interconnections 104 provided in the interconnection layer 103 .
  • a reflection rate adjusted film 105 is formed on a surface of the substrate 109 including a light receiving surface of the photoelectric converter elements 108 .
  • the reflection rate adjusted film 105 is constituted by a first portion 106 formed on at least a part of the light receiving surface of each photoelectric converter element 108 and a second portion 107 formed on other parts than the first portion 106 .
  • the first portion 106 is provided in a one-to-one correspondence with each photoelectric converter element 108 , and its reflection rate is adjusted to cause incident light to pass instead of reflecting the incident light.
  • the first portion 106 is shifted for each corresponding photoelectric converter element 108 , and the relative position thereof is strictly defined.
  • the reflection rate of the second portion 107 is adjusted to reflect almost all the incident light. In this manner, in the reflection rate adjusted film 105 , the reflection rate of the first portion 106 is adjusted to be smaller than the reflection rate of the second portion 107 .
  • parallax is caused in the subject luminous flux received by the photoelectric converter element 108 .
  • the photoelectric converter element 108 not causing parallax, only the first portion 106 is formed to pass the entire incident luminous flux, and not having the second portion 107 .
  • the color filter 102 is provided on the interconnection layer 103 .
  • Each of the color filters 102 is colored so as to transmit a particular wavelength range to a corresponding one of the photoelectric converter elements 108 , and the color filters 102 are arranged in a one-to-one correspondence with the photoelectric converter elements 108 .
  • To output a color image at least two different types of color filters that are different from each other need to be arranged. However, three or more different types of color filters may need to be arranged to produce a color image with higher quality. For example, red filters (R filters) to transmit the red wavelength range, green filters (G filters) to transmit the green wavelength range, and blue filters (B filters) to transmit the blue wavelength range may be arranged in a lattice pattern. The way how the filters are specifically arranged will be described later.
  • the microlenses 101 are provided on the color filters 102 .
  • the microlenses 101 are each a light collecting lens to guide more of the incident subject luminous flux to the corresponding photoelectric converter element 108 .
  • the microlenses 101 are provided in a one-to-one correspondence with the photoelectric converter elements 108 .
  • the optical axis of each microlens 101 is preferably shifted so that more of the subject luminous flux is guided to the corresponding photoelectric converter element 108 taking into consideration the relative positions between the pupil center of the image-capturing lens 20 and the corresponding photoelectric converter element 108 .
  • each of the microlenses 101 as well as the position of the first portion 106 of the corresponding reflection rate adjusted film 105 may be adjusted to allow more of the particular subject luminous flux to be incident, which will be described later. Note that in the case of the image sensor having a favorable light collecting efficiency and a favorable photoelectric conversion efficiency, no microlens 101 may be provided.
  • a pixel is defined as a single set constituted by one of the reflection rate adjusted films 105 , one of the color filters 102 , and one of the microlenses 101 , which are provided in a one-to-one correspondence with one of the photoelectric converter elements 108 .
  • a pixel provided with a first portion 106 that causes parallax is referred to as a parallax pixel
  • a pixel provided with a first portion 106 that does not cause parallax is referred to as a no-parallax pixel.
  • the image sensor 100 has an effective pixel region of approximately 24 mm ⁇ 16 mm, the number of pixels reaches as many as approximately 12 million.
  • FIG. 3A and FIG. 3B explain a configuration of a reflection rate adjusted film 105 according to the present embodiment.
  • FIG. 3A is a plan view of the reflection rate adjusted film 105 worth of one pixel.
  • the first portion 106 passes a certain luminous flux out of the incident luminous fluxes, to guide the certain luminous flux towards a predetermined certain region on the light receiving surface of the corresponding photoelectric converter element 108 .
  • the second portion 107 prevents a luminous flux from being incident on any regions other than the certain region, on the light receiving surface of the photoelectric converter element 108 . According to this configuration, parallax is caused on the subject luminous flux received by the photoelectric converter element 108 .
  • FIG. 3B shows a cross section of an area in the vicinity of the first portion 106 of the reflection rate adjusted film 105 .
  • the reflection rate adjusted film 105 is a multilayer film made by sequentially stacking a SiO 2 film and a SiN film.
  • the film thickness of each film in the first portion 106 and the film thickness of each film in the second portion 107 are adjusted.
  • the film thickness of each film in the first portion 106 is defined so that the first portion 106 has a reflection rate of less than 10%, i.e., a transmission rate of 90% or more.
  • the film thickness of each film in the second portion 107 is defined so that the second portion 107 has a reflection rate of 99% or more, i.e., a transmission rate of less than 1%.
  • the following explains a method of forming the reflection rate adjusted film 105 .
  • First a SiO 2 film is formed on the surface of the substrate 109 in which the light receiving surface of the photoelectric converter element 108 is exposed. Then, photolithography and etching are performed so that the film thickness of the SiO 2 film on the first portion 106 is a predefined film thickness, and the film thickness of the SiO 2 film on the second portion 107 is a predefined film thickness.
  • the film thickness of the SiO 2 film on the first portion 106 is set to be smaller than the film thickness of the SiO 2 film on the second portion 107 , the SiO 2 film is formed on the surface of the substrate 109 to the film thickness on the second portion 107 , and the portion of the first portion 106 is partially removed by photolithography and etching.
  • a SiN film is formed on the SiO 2 film having been formed. Then, photolithography and etching are performed to have the film thickness of the SiN film on the first portion 106 being a predefined film thickness and the film thickness of the SiN film on the second portion 107 being a predefined film thickness.
  • the reflection rate adjusted film 105 is formed in which SiO 2 films and SiN films are sequentially stacked.
  • the photoelectric converter element 108 by forming the first portion 106 and the second portion 107 of the reflection rate adjusted film 105 , on the light receiving surface of the photoelectric converter element 108 , reception by the photoelectric converter element 108 of unnecessary luminous fluxes different from the luminous fluxes for causing parallax is efficiently prevented.
  • the reflection rate of the first portion 106 by reducing the reflection rate of the first portion 106 as much as possible, the amount of light of the certain luminous flux received by the photoelectric converter element 108 can be larger than when there is no reflection rate adjusted film 105 formed.
  • the entire thickness of the first portion 106 is smaller than the entire thickness of the second portion 107 .
  • the present invention is not limited to this configuration. As long as the reflection rate of the first portion 106 and the reflection rate of the second portion 107 satisfy the defined values, the entire thickness of the first portion 106 may be equal to the entire thickness of the second portion 107 or larger than that.
  • the SiO 2 film and the SiN film ware used as films configuring the reflection rate adjusted film 105 .
  • the present invention is not limited to this configuration, and may alternatively use a film made of another material, such as a SiON film.
  • the material of the film configuring the first portion 106 may differ from the material of the film configuring the second portion 107 .
  • the reflection rate adjusted film 105 is configured by two portions having different refractive indexes from each other.
  • the present invention is not limited to this configuration, and may be configured by three or more portions having refractive indexes from each other.
  • the reflection rate adjusted film 105 may include a connecting portion connecting between the first portion 106 and the second portion 107 , and having consecutively changing reflective indexes from the refractive index of the first portion 106 to the refractive index of the second portion 107 .
  • the lengthwise direction of the first portion 106 i.e., the width in y axis direction, matches the width of the photoelectric converter element 108 .
  • the lengthwise width of the first portion 106 may be larger than the width of the photoelectric converter element 108 .
  • the structure of the reflection rate adjusted film 105 may be constant irrespective of the type of the color filter 102 .
  • the characteristic of the reflection rate adjusted film 105 may be different depending on the type of the color filter 102 .
  • the film thickness of each film constituting the first portion 106 and the second portion 107 is adjusted for each type of color filters, so that each type of color filters 102 has a predefined reflection rate.
  • the film thickness of each film is adjusted so that the transmittivity of light in a green wavelength region is favorable.
  • the film thickness of each film is adjusted so that the reflectivity of light in a green wavelength is favorable.
  • FIG. 4 is an overview showing an enlarged view of a part of an image sensor 100 .
  • the colors of the color filters 102 are not mentioned till later.
  • it can be interpreted as an image sensor resulting from collecting only parallax pixels having color filters 102 of a same color. Therefore, the following-explained repeating pattern may be considered as adjacent pixels in the same-colored color filter 102 .
  • the first portion 106 of the reflection rate adjusted film 105 is relatively shifted with respect to each pixel.
  • the respective first portions 106 of the adjacent pixels are displaced to each other.
  • reflection rate adjusted films 105 in six types of pixel units are prepared in which the first portions 106 shifted in left and right directions each other are formed, and the second portions 107 are formed in portions different from where the first portions 106 are formed.
  • groups of photoelectric converter elements are arranged two dimensionally and periodically, in which in one group, six parallax pixels having reflection rate adjusted films 105 whose first portions 106 gradually shift from the left side towards the right side of the paper.
  • the alignment pattern of photoelectric converter element groups is referred to as a repeating pattern 110 .
  • FIGS. 5A , 5 B, and 5 C are each a conceptual diagram illustrating the relation between parallax pixels and a subject.
  • FIG. 5A illustrates a photoelectric converter element group having a repeating pattern 1101 arranged at the center of the image sensor 100 through which the image-capturing optical axis 21 extends.
  • FIG. 5B schematically illustrates a photoelectric converter element group having a repeating pattern 110 u of the parallax pixels arranged in the peripheral portion of the image sensor 100 .
  • a subject 30 is positioned at a focus position relative to the image-capturing lens 20 .
  • FIG. 5C schematically illustrates the relation between the parallax pixels and the subject when a subject 31 at a non-focus position relative to the image-capturing lens 20 is captured, correspondingly to the relation shown in FIG. 5A .
  • the following first describes the relation between the parallax pixels and the subject when the image-capturing lens 20 captures the subject 30 at the focused state.
  • the subject luminous flux is guided through the pupil of the image-capturing lens 20 to the image sensor 100 .
  • six partial regions Pa to Pf are defined in the entire cross-sectional region through which the subject luminous flux transmits. For example, see the pixel, on the extreme left in the sheet of FIGS. 5A to 5C , of the photoelectric converter element groups having the repeating patterns 110 t and 110 u .
  • the position of the first portion 106 f of the reflection rate adjusted film 105 is defined so that only the subject luminous flux emitted from the partial region Pf reaches the photoelectric converter element 108 as seen from the enlarged view.
  • the position of the first portion 106 e is defined so as to correspond to the partial region Pe
  • the position of the first portion 106 d is defined so as to correspond to the partial region Pd
  • the position of the first portion 106 c is defined so as to correspond to the partial region Pc
  • the position of the first portion 106 b is defined so as to correspond to the partial region Pb
  • the position of the first portion 106 a is defined so as to correspond to the partial region Pa.
  • the gradient of the principal ray Rf of the subject luminous flux (partial luminous flux) emitted from the partial region Pf may determine the position of the first portion 106 f .
  • the photoelectric converter element 108 receives the subject luminous flux through the first portion 106 f from the subject 30 at the focus position, the subject luminous flux forms an image on the photoelectric converter element 108 as indicated by the dotted line.
  • the gradient of the principal ray Re determines the position of the first portion 106 e
  • the gradient of the principal ray Rd determines the position of the first portion 106 d
  • the gradient of the principal ray Rc determines the position of the first portion 106 c
  • the gradient of the principal ray Rb determines the position of the first portion 106 b
  • the gradient of the principal ray Ra determines the position of the first portion 106 a.
  • the luminous flux emitted from a micro region Ot of the subject 30 at the focus position passes through the pupil of the image-capturing lens 20 and reaches the respective pixels of the photoelectric converter element group having the repeating pattern 110 t .
  • the pixels of the photoelectric converter element group having the repeating pattern 110 t respectively receive the luminous flux emitted from the single micro region Ot through the six partial regions Pa to Pf.
  • the micro region Ot has a certain spread that can accommodate the different positions of the respective pixels of the photoelectric converter element group having the repeating pattern 110 t , but can be substantially approximated by substantially the same object point.
  • FIG. 5A the luminous flux emitted from a micro region Ot of the subject 30 at the focus position, which coincides with the optical axis 21 on the subject 30 , passes through the pupil of the image-capturing lens 20 and reaches the respective pixels of the photoelectric converter element group having the repeating pattern 110 t .
  • the pixels of the photoelectric converter element group having the repeating pattern 110 t respectively receive the
  • the luminous flux emitted from a micro region Ou of the subject 30 at the focus position, which is spaced away from the optical axis 21 on the subject 30 passes through the pupil of the image-capturing lens 20 to reach the respective pixels of the photoelectric converter element group having the repeating pattern 110 u .
  • the respective pixels of the photoelectric converter element group having the repeating pattern 110 u respectively receive the luminous flux emitted from the single micro region Ou through the six partial regions Pa to Pf.
  • the micro region Ou has a certain spread that can accommodate the different positions of the respective pixels of the photoelectric converter element group having the repeating pattern 110 u , but can be substantially approximated by substantially the same object point.
  • the photoelectric converter element groups capture different micro regions depending on the positions of the repeating patterns 110 on the image sensor 100 , and the respective pixels of each photoelectric converter element group capture the same micro region through the different partial regions.
  • the corresponding pixels receive subject luminous flux from the same partial region.
  • the leftmost pixels of the repeating patterns 110 t and 110 u receive the partial luminous flux from the same partial region Pf.
  • the position of the first portion 106 f of the leftmost pixel that receives the subject luminous flux from the partial region Pf in the repeating pattern 110 t at the center through which the image-capturing optical axis 21 extends is different from the position of the first portion 106 f of the leftmost pixel that receives the subject luminous flux from the partial region Pf in the repeating pattern 110 u at the peripheral portion.
  • these reflection rate adjusted film can be treated as the same type of reflection rate adjusted films in that they are both reflection rate adjusted films to receive the subject luminous flux from the partial region Pf. Accordingly, in the example shown in FIGS. 5A to 5C , it can be said that each of the parallax pixels arranged on the image sensor 100 has one of the six types of reflection rate adjusted films.
  • the subject luminous flux from the subject 31 at the non-focus position also passes through the six partial regions Pa to Pf of the pupil of the image-capturing lens 20 to reach the image sensor 100 .
  • the subject luminous flux from the subject 31 at the non-focus position forms an image not on the photoelectric converter elements 108 but at a different position.
  • the subject luminous flux forms an image at a position closer to the subject 31 with respect to the photoelectric converter elements 108 .
  • the subject luminous flux forms an image at a position on the opposite side of the subject 31 with respect to the photoelectric converter elements 108 .
  • the subject luminous flux emitted from a micro region Ot′ of the subject 31 at the non-focus position reaches the corresponding pixels of different repeating patterns 110 depending on which of the six partial regions Pa to Pf the subject luminous flux passes through.
  • the subject luminous flux that has passed through the partial region Pd enters the photoelectric converter element 108 having the first portion 106 d included in the repeating pattern 110 t ′ as a principal ray Rd′ as shown in the enlarged view of FIG. 5C .
  • the subject luminous flux that has passed through the other partial regions may be emitted from the micro region Ot′, but does not enter the photoelectric converter elements 108 included in the repeating pattern 110 f and enters the photoelectric converter elements 108 having the corresponding first portions 106 in different repeating patterns.
  • the subject luminous fluxes that reach the respective photoelectric converter elements 108 constituting the repeating pattern 110 t ′ are subject luminous fluxes emitted from different micro regions of the subject 31 .
  • the subject luminous flux having the principal ray Rd′ enters the photoelectric converter element 108 corresponding to the first portion 106 d
  • the subject luminous fluxes having the principal rays Ra + , Rb + , Rc + , Re + , Rf + which are emitted from different micro regions of the subject 31 , enter the photoelectric converter elements 108 corresponding to the other first portions 106 .
  • the same relation is also seen in the repeating pattern 110 u arranged in the peripheral portion shown in FIG. 5B .
  • a subject image A captured by the photoelectric converter element 108 corresponding to the first portion 106 a and a subject image D captured by the photoelectric converter element 108 corresponding to the first portion 106 d match with each other if they are images of the subject at the focus position, and do not match with each other if they are images of the subject at the non-focus position.
  • the direction and amount of the non-match are determined by on which side the subject at the non-focus position is positioned with respect to the focus position, how much the subject at the non-focus position is shifted from the focus position, and the distance between the partial region Pa and the partial region Pd.
  • the subject images A and D are parallax images causing parallax therebetween. This relation also applies to the other first portions 106 , and six parallax images are formed corresponding to the first portions 106 a to 106 f.
  • a collection of outputs from the corresponding pixels in different ones of the repeating patterns 110 configured as described above produces a parallax image.
  • the outputs from the pixels that have received the subject luminous flux emitted from a particular partial region of the six partial regions Pa to Pf form a parallax image.
  • FIG. 6 is a conceptual diagram to illustrate an operation to produce a parallax image.
  • FIG. 6 shows, from left to right, how parallax image data Im_f is produced by collecting the outputs from the parallax pixels corresponding to the first portions 106 f , how parallax image data Im_e is produced from the outputs of the parallax pixels corresponding to the first portions 106 e , how parallax image data Im_d is produced from the outputs of the parallax pixels corresponding to the first portions 106 d , how parallax image data Im_c is produced from the outputs of the parallax pixels corresponding to the first portions 106 c , how parallax image data Im_b is produced from the outputs of the parallax pixels corresponding to the first portions 106 b , and how parallax pixel data Im_a is produced from the outputs from the parallax pixels corresponding to the first portions 106 a .
  • the following first describes how
  • the repeating patterns 110 each of which has a photoelectric converter element group constituted by a group of six parallax pixels are arranged side-by-side. Accordingly, on the hypothetical image sensor 100 excluding no-parallax pixels, the parallax pixels having the first portions 106 f are found every six pixels in the horizontal direction and consecutively arranged in the vertical direction. These pixels receive subject luminous fluxes from different micro regions as described above. Therefore, parallax images can be obtained by collecting and arranging the outputs from theses parallax pixels.
  • the pixels of the image sensor 100 of the present embodiment are square pixels. Therefore, if the outputs are simply collected, the number of pixels in the horizontal direction is reduced to one-sixth and vertically long image data is produced. To address this issue, interpolation is performed to increase the number of pixels in the horizontal direction six times. In this manner, the parallax image data Im_f is produced as an image having the original aspect ratio. Note that, however, the horizontal resolution is lower than the vertical resolution since the parallax image data before the interpolation represents an image whose number of pixels in the horizontal direction is reduced to one-sixth. In other words, the number of pieces of parallax image data produced is inversely related to the improvement of the resolution.
  • the interpolation applied in the present embodiment will be specifically described later.
  • parallax image data Im_e to parallax image data Im_a are obtained.
  • the digital camera 10 can produce parallax images from six different viewpoints with horizontal parallax.
  • FIG. 7 illustrates a Bayer array.
  • G filters are assigned to the two pixels, i.e., the upper-left (Gb) and lower right (Gr) pixels
  • an R filter is assigned to one pixel, i.e., the lower left pixel
  • a B filter is assigned to one pixel, i.e., an upper right pixel in the Bayer array.
  • an enormous number of different repeating patterns 110 can be defined depending on to what colors of pixels the parallax and no-parallax pixels are allocated and the period in which parallax and no-parallax pixels are allocated. Collecting the outputs of the no-parallax pixels can produce no-parallax captured image data like an ordinary captured image. Accordingly, a high-resolution 2D image can be output by increasing the ratio of the no-parallax pixels relative to the parallax pixels. In this case, the ratio of the parallax pixels decreases relative to the no-parallax pixels and a 3D image formed by a plurality of parallax images exhibits lower image quality.
  • the 3D image exhibits improved image quality.
  • the ratio of the no-parallax pixels decreases relative to the parallax pixels, a low-resolution 2D image is output. If the parallax pixels are allocated to all of the R, G and B pixels, the resulting color image data represents a 3D image having excellent color reproducibility and high quality.
  • the output color image data ideally has high resolution and quality.
  • the region of a 3D image for which an observer senses parallax when observing the 3D image is the non-focus region in which the identical subject images do not match, as understood from the cause of the parallax, which is described with reference to FIGS. 5A , 5 B, and 5 C.
  • the image data required to produce a 3D image does not need to have very high resolution in the region in which parallax is generated.
  • the corresponding image data is extracted from 2D image data.
  • the corresponding image data is extracted from 3D image data.
  • parallax image data can be produced by combining these pieces of image data for the focused and non-focused regions.
  • high-resolution 2D image data is used as basic data and multiplied by the relative ratios of the 3D image data on the pixel-by-pixel basis. In this way, high-resolution parallax image data can be produced.
  • the number of the parallax pixels may be allowed to be smaller than the number of the no-parallax pixels in the image sensor 100 . In other words, a 3D image having a relatively high resolution can be produced even if the number of the parallax pixels is relatively small.
  • the parallax pixels have all of the combinations of the different types of first portions 106 and the three types of, i.e., R, G and B color filters.
  • Parallax Lt pixels having an first portion 106 shifted to the left from the center and parallax Rt pixels having an first portion 106 shifted to the right from the center are taken as an example.
  • the parallax Lt pixels include a pixel having an R filter, a pixel having a G filter, and a pixel having a B filter
  • the parallax Rt pixels include a pixel having an R filter, a pixel having a G filter, and a pixel having a B filter.
  • the image sensor 100 has six different types of parallax pixels.
  • Such an image sensor 100 outputs image data, which is used to form clear color parallax image data to realize a stereoscopic view. Note that, when two types of first portions 106 are combined with two types of color filters, the image sensor 100 has four types of parallax pixels.
  • FIG. 8 illustrates the arrangement of pixels in a repeating pattern 110 relating to a first implementation.
  • the repeating pattern 110 relating to the first implementation includes four Bayer arrays, each of which is formed by four pixels, arranged both in the vertical direction, which is the Y-axis direction, and in the horizontal direction, which is the X-axis direction, and is thus constituted by 64 pixels.
  • This repeating pattern 110 has a pixel group of 64 pixels as a single unit, and a plurality of repeating patterns 110 are periodically arranged horizontally and vertically within the effective pixel region of the image sensor 100 .
  • the repeating pattern 110 bounded by the thick bold line in the drawing is the primitive lattice in the image sensor 100 .
  • the pixels within the repeating pattern 110 are represented as P IJ .
  • the leftmost and uppermost pixel is represented as P 11 and the rightmost and uppermost pixel is represented as P 81 .
  • Each of the parallax pixels relating to the first implementation has one of the two types of reflection rate adjusted films 105 , so that the parallax pixels are divided into the parallax Lt pixels having the first portions 106 shifted to the left from the center of the pixels and the parallax Rt pixels having the first portions 106 shifted to the right from the center of the pixels.
  • the parallax pixels are arranged in the following manner.
  • the pixel arrangement preferably includes the parallax pixels having all of the combinations of the different types of first portions 106 and the different types of color filters within the primitive lattice of the pixel arrangement and has the parallax pixels randomly arranged together with the no-parallax pixels that are more than the parallax pixels.
  • the parallax and no-parallax pixels are counted according to each type of color filters, that the no-parallax pixels are still more than the parallax pixels.
  • FIG. 9 illustrates how the pixels are arranged in a repeating pattern 110 relating to a second implementation.
  • the repeating pattern 110 relating to the second implementation includes four Bayer arrays, each of which is formed by four pixels, both in the vertical direction, which is the Y-axis direction, and in the horizontal direction, which is the X-axis direction, and is thus constituted by 64 pixels.
  • the repeating pattern 110 has a pixel group of 64 pixels as a single unit, and a plurality of repeating patterns 110 are periodically arranged horizontally and vertically within the effective pixel region of the image sensor 100 .
  • the repeating pattern 110 bounded by the thick bold line in the drawing is the primitive lattice in the image sensor 100 .
  • each of the parallax pixels has one of the two types of reflection rate adjusted film 105 , so that the parallax pixels are divided into the parallax Lt pixels having the first portions 106 shifted to the left from the center of the pixels and the parallax Rt pixels having the first portions 106 shifted to the right from the center of the pixels.
  • the parallax pixels are arranged in the following manner.
  • the pixel arrangement preferably includes the parallax pixels having all of the combinations of the different types of first portions 106 and the different types of color filters within the primitive lattice of the pixel arrangement and has the parallax pixels randomly arranged together with the no-parallax pixels that are more than the parallax pixels.
  • the parallax and no-parallax pixels are counted according to each type of color filters, that the no-parallax pixels are still more than the parallax pixels.
  • Image data representing an image matching the characteristic can be formed by separating the pixel output of the image sensor 100 into each pixel group characterized to be the same, and then collecting them. For example, as already explained with reference to FIG. 6 , by collecting the output of the parallax pixel according to each type of the first portion 106 would generate a plurality of pieces of parallax image data having parallax to each other. In this way, respective pieces of image data obtained by separating the parallax pixel output into each pixel group characterized to be the same and collecting them is referred to as plane data.
  • the image processor 205 receives RAW original image data in which the output values are arranged in the order of pixel array of the image sensor 100 , and executes plane separation to separate the RAW original image data into a plurality of pieces of plane data.
  • the following explains a generation process of each piece of plane data, with reference to an example of output from the image sensor 100 of the first embodiment example explained with reference to FIG. 8 .
  • FIG. 10 illustrates, as an example, how to produce 2D-RGB plane data, which is 2D image data.
  • the top drawing shows the outputs from the pixels in the single repeating pattern 110 and its surrounding pixels in the image sensor 100 in accordance with the pixel arrangement of the image sensor 100 .
  • the pixels are shown in accordance with the example of FIG. 8 so that the different types of pixels can be understood, but it is actually the output values corresponding to the pixels that are arranged.
  • the image processor 205 first removes the pixel values of the parallax pixels and creates empty pixel positions.
  • the pixel value for each empty pixel position is calculated by interpolation using the pixel values of the surrounding pixels having the color filters of the same type. For example, the pixel value for an empty pixel position P 11 is calculated by averaging the pixel values of the obliquely adjacent G-filter pixels P -1-1 , P 2-1 , P -12 , P 22 .
  • the pixel value for an empty pixel position P 63 is calculated by averaging the pixel values of the R-filter pixels P 43 , P 61 , P 83 , P 65 that are vertically and horizontally adjacent to the empty pixel position P 63 with one pixel position placed therebetween.
  • the pixel value for an empty pixel position P 76 is calculated by averaging the pixel values of the B-filter pixels P 56 , P 74 , P 96 , P 78 that are vertically and horizontally adjacent to the empty pixel position P 76 with one pixel position placed therebetween.
  • the resulting 2D-RGB plane data obtained by the above-described interpolation is the same as the output from a normal image sensor having the Bayer array and can be subsequently subjected to various types of processing as 2D image data.
  • the image processor 205 performs image processing in accordance with predetermined formats, for example, follows the JPEG standard or the like to produce still image data and follows the MPEG standard or the like to produce moving image data.
  • FIG. 11 illustrates, as an example, how to produce two pieces of G plane data, which are parallax image data.
  • GLt plane data which is left parallax image data
  • GRt plane data which is right parallax image data
  • the image processor 205 removes the pixel values, except for the pixel values of the G(Lt) pixels, from all of the output values of the image sensor 100 and creates empty pixel positions. As a result, two pixel values P 11 and P 55 are left in the repeating pattern 110 .
  • the repeating pattern 110 is vertically and horizontally divided into four portions.
  • the pixel values of the 16 pixels in the upper left portion are represented by the output value at P H
  • the pixel values of the 16 pixels in the lower right portion are represented by the output value at P 55 .
  • the pixel value for the 16 pixels in the upper right portion and the pixel value for the 16 pixels in the lower left portion are interpolated by averaging the surrounding or vertically and horizontally adjacent representative values.
  • the GLt plane data has one value per 16 pixels.
  • the image processor 205 removes the pixel values, except for the pixel values of the G(Rt) pixels, from all of the output values of the image sensor 100 and creates empty pixel positions. As a result, two pixel values P 51 and P 15 are left in the repeating pattern 110 .
  • the repeating pattern 110 is vertically and horizontally divided into four portions. The pixel values of the 16 pixels in the upper right portion are represented by the output value at P 51 , and the pixel values of the 16 pixels in the lower left portion are represented by the output value at P 15 .
  • the pixel value for the 16 pixels in the upper left portion and the pixel value for the 16 pixels in the lower right portion are interpolated by averaging the surrounding or vertically and horizontally adjacent representative values.
  • the GRt plane data has one value per 16 pixels.
  • the GLt plane data and GRt plane data which have lower resolution than the 2D-RGB plane data, can be produced.
  • FIG. 12 illustrates, as an example, how to produce two pieces of B plane data, which are parallax image data.
  • B plane data which is left parallax image data
  • BRt plane data which is right parallax image data
  • the image processor 205 removes the pixel values, except for the pixel value of the B(Lt) pixel, from all of the output values of the image sensor 100 and creates empty pixel positions. As a result, a pixel value P 32 is left in the repeating pattern 110 . This pixel value is used as the representative value of the 64 pixels of the repeating pattern 110 .
  • the image processor 205 removes the pixel values, except for the pixel value of the B(Rt) pixel, from all of the output values of the image sensor 100 and creates empty pixel positions. As a result, a pixel value P 76 is left in the repeating pattern 110 . This pixel value is used as the representative value of the 64 pixels of the repeating pattern 110 .
  • the BLt plane data and BRt plane data which have lower resolution than the 2D-RGB plane data, can be produced.
  • the BLt plane data and BRt plane data have lower resolution than the GLt plane data and GRt plane data.
  • FIG. 13 illustrates, as an example, how to produce two pieces of R plane data, which are parallax image data.
  • RLt plane data which is left parallax image data
  • RRt plane data which is right parallax image data
  • the image processor 205 removes the pixel values, except for the pixel value of the R(Lt) pixel, from all of the output values of the image sensor 100 and creates empty pixel positions. As a result, a pixel value P 27 is left in the repeating pattern 110 . This pixel value is used as the representative value of the 64 pixels of the repeating pattern 110 .
  • the image processor 205 removes the pixel values, except for the pixel value of the R(Rt) pixel, from all of the output values of the image sensor 100 and creates empty pixel positions. As a result, a pixel value P 63 is left in the repeating pattern 110 . This pixel value is used as the representative value of the 64 pixels of the repeating pattern 110 .
  • the RLt plane data and RRt plane data which have lower resolution than the 2D-RGB plane data, can be produced.
  • the RLt plane data and RRt plane data have lower resolution than the GLt plane data and GRt plane data and substantially the same resolution as the BLt plane data and BRt plane data.
  • FIG. 14 is a conceptual view illustrating the relation between the resolutions of the respective planes.
  • the 2D-RGB plane data has output values substantially the same as the number of effective pixels of the image sensor 100 since it has undergone interpolation.
  • the GLt plane data and GRt plane data each have output values equal to 1/16 (1 ⁇ 4 ⁇ 1 ⁇ 4) of the number of pixels of the 2D-RGB plane data due to the interpolation.
  • the high-resolution 2D image can be first output.
  • the information of the 2D-RGB plane data is used, and for the non-focused region, parallax image data such as GLt plane data is used to perform synthesis processing or the like. In this way, 3D image which has sufficient resolution can be output.
  • the allocation ratio of the no-parallax pixels with respect to the color filter, the allocation ratio of the parallax Lt pixel, and the allocation ratio of the parallax Rt pixel can be arbitrarily set. Not limited to the allocation ratio in the first embodiment example and the second embodiment example, it is also effective to set, to be the same, the allocation ratio of the no-parallax pixel, the allocation ratio of the parallax Lt pixel, and the allocation ratio of the parallax Rt pixel. For example, all of the respective allocation ratios can be set to 1:1:1, or to 2:1:1 so that the ratio of G is greater. By adjusting the allocation ratios in the stated manner, the correspondence between no-parallax image data and parallax image data can become easier.
  • parallax images corresponding to the two viewpoints can be obtained by using the two different types of parallax pixels as in the first and second implementations
  • various numbers of types of parallax pixels can be used depending on the desired number of parallax images to output.
  • Various repeating patterns 110 can be formed depending on the specifications, purposes or the like, irrespective of whether the number of viewpoints increases.
  • the primitive lattice of the image sensor 100 includes parallax pixels having all of the combinations of the different types of first portions 106 and the different types of color filters and that the no-parallax pixels are more than the parallax pixels.
  • the exemplary case is described in which the Bayer array is employed as the color filter arrangement. It goes without saying, however, other color filter arrangements can be used. Furthermore, in the above-described example, the three primary colors of red, green and blue are used for the color filters. However, four or more primary colors including emerald green may be used. In addition, red, green and blue can be replaced with three complementary colors of yellow, magenta and cyan.
  • the first portion 106 may be formed so that the area of the first portion 106 for the no-parallax pixel can correspond to a summation between the area of the first portion 106 for the parallax Lt pixel and the area of the first portion 106 for the parallax Rt pixel.
  • FIG. 15 explains a shape of a first portion 106 .
  • the first portion 106 n of the no-parallax pixel is formed to have a same size as the photoelectric converter element 108 .
  • the first portion 1061 of the parallax Lt pixel is formed to have a same size as a left half of the photoelectric converter element 108 .
  • the first portion 106 r of the parallax Rt pixel is formed to have a same size as a right half of the photoelectric converter element 108 .
  • the shape of the first portion 1061 of the parallax Lt pixel and the shape of the first portion 106 r of the parallax Rt pixel are respectively the same as the shape of the respective portion resulting from dividing the shape of the first portion 106 n of the no-parallax pixel by the center line 120 .
  • the area of the first portion 106 n of the no-parallax pixel becomes a summation between the area of the first portion 1061 of the parallax Lt pixel and the area of the first portion 106 r of the parallax Rt pixel.
  • each of the first portion 106 n of the no-parallax pixel, the first portion 1061 of the parallax Lt pixel, and the first portion 106 r of the parallax Rt pixel has a function as an aperture diaphragm. Therefore, the amount of out of focus of the no-parallax pixel having the first portion 106 n having an area twice as the first portion 1061 (first portion 106 r ) will be the same level as the summation of the amounts of out of focus of the parallax Lt pixel and the parallax Rt pixel.
  • the output of the AF sensor 211 is used for the determination of the focused region.
  • the determination can also be done by comparing the output values of the parallax image data.
  • the controller 201 determines that it is the focused state when the pixel values of the corresponding pixels of the GLt plane data and the GRt plane data are the same as each other, and determines that the region including the pixel is the focused region.
  • the above-described parallax pixel can be aligned as a phase detection pixel in the plurality of focus detection regions set to the effective pixel region of the image sensor 100 .
  • the parallax Rt pixels may be aligned one dimensionally in the left and right direction in the focus detection region, as the phase detection pixels in the left and right direction.
  • the parallax Lt pixels are aligned one dimensionally in the left and right direction in the focus detection region, as the phase detection pixels in the left and right direction.
  • the controller 201 executes a correlation operation using output of the parallax Rt pixel and the output of the parallax Lt pixel in the focus detection region, to perform focus determination.
  • the parallax pixel and the no-parallax pixel can be mixed and aligned as stated above. Or, only no-parallax pixels may be aligned to generate 2D image data without any parallax.
  • parallax Rt pixel and the parallax Lt pixel may be alternately aligned one dimensionally in the left and right direction in the focus detection region.
  • the upper parallax pixel in which the first portion 106 is shifted in the upper direction from the center and the lower parallax pixel in which the first portion 106 is shifted in the lower direction from the center may be used as a phase detection pixel in the upper and lower direction.
  • the phase detection pixel may not be provided with a color filter 102 .
  • the focus detection region may be constituted by phase detection pixels. It is sufficient that the phase detection pixels sufficient to perform focus determination favorably are aligned in the focus detection region.
  • FIG. 16 schematically shows a cross section of an image sensor 300 according to a first modification example.
  • the image sensor 300 is such that an aperture mask 301 is provided to the above-described image sensor 100 .
  • the members of the image sensor 300 that are the same as those of the image sensor 100 are assigned the same reference numerals, and the explanation of the function thereof is omitted in the following.
  • the aperture mask 301 is provided to contact the interconnection layer 103 .
  • a color filter 102 is provided on the aperture mask 301 .
  • the aperture 302 of the aperture mask 301 is provided in a one-to-one correspondence with each photoelectric converter element 108 .
  • the aperture 302 is shifted for each corresponding photoelectric converter element 108 , and the relative position thereof is strictly defined.
  • the aperture 302 is provided in a one-to-one correspondence with each first portion 106 .
  • the aperture 302 passes a certain luminous flux out of the incident luminous flux, and guides the certain luminous flux to a corresponding first portion 106 .
  • the operation of the first portion 106 and the aperture 302 causes a parallax in the subject luminous flux received by the photoelectric converter element 108 .
  • no aperture mask 301 is provided over the photoelectric converter element 108 that does not cause any parallax.
  • an aperture mask 301 that includes an aperture 302 that does not block the subject luminous flux incident to the corresponding photoelectric converter element 108 , i.e., that permits all the incident luminous flux to pass, is provided.
  • the two members i.e., the reflection rate adjusted film 105 and the aperture mask 301
  • the two members are used as a light-blocking member, which enhances the light-blocking efficiency of unnecessary luminous fluxes.
  • the aperture mask 301 can block unnecessary luminous flux to some extent
  • the reflection rate of the second portion 107 of the reflection rate adjusted film 105 in the first modification example can be smaller than in the above-explained embodiment that is without any aperture mask 301 .
  • the reflection rate of the second portion 107 is defined to about 50%.
  • the aperture mask 301 may be aligned independently and separately corresponding to each photoelectric converter element 108 .
  • the aperture mask 301 may be collectively formed to the plurality of photoelectric converter elements 108 , in the similar manner to the manufacturing process of the color filter 102 .
  • the aperture 302 of the aperture mask 301 can be integrally formed.
  • the aperture mask 301 and the interconnection 104 are provided as different entities.
  • the function of the aperture mask 301 in the parallax pixel can be performed by the interconnection 104 . That is, the interconnection 104 may be used to shape the defined aperture form, and this aperture form may be used to restrict the incident luminous flux thereby guiding only a certain partial luminous flux towards the first portion 106 .
  • the interconnection 104 shaping the aperture form is preferably closest to the photoelectric converter element 108 in the interconnection layer 103 .
  • FIG. 17 schematically shows a cross section of an image sensor 400 according to a second modification example.
  • the image sensor 400 is a backside illumination image sensor in which the interconnection layer 103 is provided on a side of the substrate 109 opposite to a side to which the photoelectric converter element 108 is provided. Note that the members of the image sensor 400 are the same as those of the image sensor 100 , and so the explanation of the function thereof is omitted in the following.
  • the color filter 102 is provided on the reflection rate adjusted film 105 .
  • the interconnection layer 103 is provided on a surface opposite to the surface of the substrate 109 on which the light receiving surface of the photoelectric converter element 108 is exposed. In this way, the reflection rate adjusted film according to the present embodiment described above can also be applied to a backside illumination image sensor.
  • FIG. 18 explains a configuration of a reflection rate adjusted film 105 adjusted to the incident light characteristic.
  • the lateral axis shows a position of an aperture with respect to the x axis direction (the left and right direction on the paper of the drawing) of the photoelectric converter element 108
  • the longitudinal axis shows an optical intensity distribution as an ideal incident light characteristic.
  • the optical intensity distribution of the parallax Lt pixel is shown as a solid line
  • the optical intensity distribution of the parallax Rt pixel is shown by an alternate long and short dash line. So as to assist the realization of such an optical intensity distribution, the region of the photoelectric converter element 108 is divided in a plurality of portions, and each portion is differed in transmission rate.
  • FIG. 18B explains a configuration of a reflection rate adjusted film 105 in a third embodiment example.
  • this is a plan view of the reflection rate adjusted film 105 worth of one pixel.
  • a first portion 501 is a region occupying left 3 ⁇ 4 of the left half of the photoelectric converter element 108 , and its transmission rate is adjusted to 100%.
  • the second portion 502 is a region occupying right 1 ⁇ 4 of the left half of the photoelectric converter element 108 , and its transmission rate is adjusted to 50%.
  • the third portion 503 is a region occupying left 1 ⁇ 4 of the right half of the photoelectric converter element 108 , and its transmission rate is adjusted to 10%
  • the fourth portion 504 is the other region, and its transmission rate is adjusted to 0%, i.e., is adjusted to block the incident light. In this way, the region of one pixel is divided, and the transmission rate of the incident light is differed according to each portion, thereby enabling to obtain an incident light characteristic closer to the ideal state.
  • FIG. 19A and FIG. 19B explain a configuration of a reflection rate adjusted film 105 according to a further different variation.
  • One pixel not only can be divided in x axis direction (left and right direction of the paper of the drawing) of the photoelectric converter element 108 , but also can be divided two dimensionally including y axis direction (upper and lower direction of the paper of the drawing).
  • FIG. 19A explains a configuration of the reflection rate adjusted film 105 according to a fourth modification example.
  • this is a plan view of the reflection rate adjusted film 105 worth of one pixel.
  • a first portion 511 is an ellipsoidal region included in the left 5 ⁇ 8 region of the photoelectric converter element 108 , and its transmission rate is adjusted to 100%.
  • the long axis of the ellipse is the width of the photoelectric converter element 108 in y axis direction.
  • a part of the ellipsoidal region enters the right half region, passing the central axis of the pixel.
  • the second portion 512 is a region corresponding to the left 5 ⁇ 8 region of the photoelectric converter element 108 excluding the first region, and its transmission rate is adjusted to 15%.
  • the third portion 514 is the other region, and its transmission rate is adjusted to 0%, i.e., adjusted to block the incident light. If division is pursued in this way, an incident light characteristic closer to the ideal state can be obtained also in the y axis direction.
  • FIG. 19B explains a configuration of the reflection rate adjusted film 105 according to a fifth modification example.
  • This is a plan view of the reflection rate adjusted film 105 worth of one pixel.
  • the first portion 521 is a region occupying the upper left 1 ⁇ 4 of the photoelectric converter element 108 , and its transmission rate is adjusted to 100%.
  • the second portion 522 is a border region in contact with the two sides of the first portion 521 that are close to the center of the photoelectric converter element 108 , and its transmission rate is adjusted to 30%.
  • the third portion 524 is the other region, and its transmission rate is adjusted to 0%, i.e., adjusted to block the incident light.
  • Such division can also be applied to parallax pixels also giving parallax in the y axis direction.
  • the reflection rate adjusted film 105 was explained to be a multilayer film made by sequentially stacking a SiO 2 film and a SiN film.
  • a SiON film can be used instead of a SiO 2 film
  • a Ta 2 O 5 film, a MgF film, and a SiON film can be used instead of a SiN film.
  • FIG. 20 shows a process flow according to a first manufacturing process. The flow starts from the state in which the substrate on which the photoelectric converter element is formed is fixed.
  • Step S 101 a SiO 2 film is deposited on a substrate.
  • Step S 102 in the deposited SiO 2 film, the film thicknesses of the first portion defined as a transmitting region and the second portion defined as a light-blocking region are adjusted.
  • Step S 103 a SiN film is deposited on the SiO 2 film of which film thickness has been adjusted.
  • Step S 104 in the deposited SiN film, the film thicknesses of the first portion and the second portion are adjusted.
  • Step S 105 a SiO 2 film is deposited on the SiN film of which the film thickness has been adjusted.
  • Step S 106 in the deposited SiO 2 film, the film thicknesses of the first portion and the second portion are adjusted, to end the series of processes. To add more layers, deposition of a SiN film, a SiO 2 film, and their film thickness adjustment can be repeated.
  • FIG. 21 shows a process flow according to the second manufacturing process, regarding the manufacturing process of the film structure having a deposition composition of three layers of a SiO 2 film, a SiN film, and a SiO 2 film.
  • the flow starts from the state in which the substrate on which the photoelectric converter element is formed is fixed.
  • step S 201 a SiO 2 film is deposited on the substrate.
  • Step S 202 masking is performed to the deposited SiO 2 film, to divide a first portion defined as a transmitting region from a second portion defined as a light-blocking region.
  • Step S 203 etching is performed to the SiO 2 film. The region not provided with masking is etched away, to adjust the film thickness.
  • Step S 204 a SiN film is deposited on the SiO 2 film of which the film thickness has been adjusted.
  • Step S 205 masking is performed to the deposited SiN film, to divide the first portion from the second portion.
  • Step S 206 etching is performed to the SiN film. The region not provided with masking is etched away, to adjust the film thickness
  • Step S 207 a SiO 2 film is deposited on the SiN film of which the film thickness has been adjusted.
  • Step S 208 masking is performed to the deposited SiO 2 film, to divide the first portion from the second portion.
  • Step S 209 etching is performed to the SiO 2 film. The region not provided with masking is etched away, to adjust the film thickness, and the series of processes ends.
  • deposition of a SiN film, a SiO 2 film, masking, and etching may be repeated. Note that the masked region of the SiO 2 film and the masked region of the SiN film may be the same region, or they may alternate. If the masked regions are to alternate, the first portion in the SiO 2 film is masked, and the second portion in the SiN film is masked, for example.
  • the region other than the photoelectric converter element 108 may be left remaining. In this region, if the film is left without being etched away, a cross-talk prevention effect may be obtained in some cases.
  • FIG. 22 shows a simulation result of a reflection rate of each film composition with respect to an incident wavelength of a visible light region.
  • the lengthwise axis represents a wavelength (nm) of incident light corresponding to a visible light region
  • the longitudinal axis represents reflection rate (%).
  • the curve 801 represents a reflection rate characteristic of a configuration of a film A deposited under a reflection rate increasing condition.
  • An example of the reflection rate increasing condition is such that, on a Si substrate, four layers of a SiO 2 film having a film thickness of t 1 nm, a SiN film having a film thickness of t 2 nm, a SiO 2 film having a film thickness of t 3 nm, and a SiN film having a film thickness of t 4 nm are stacked.
  • the reflection rate of this stacked film gradually increases from a short wavelength side, and gradually decreases towards a longer wavelength side, with the apex being around W 1 nm.
  • the curve 802 represents a reflection rate characteristic of a configuration of a film B deposited under a reflection rate decreasing condition.
  • An example of the reflection rate decreasing condition is such that, on a Si substrate, four layers of a SiO 2 film having a film thickness of t 5 nm, a SiN film having a film thickness of t 6 nm, a SiO 2 film having a film thickness of t 7 nm, and a SiN film having a film thickness of t 8 nm, having a film-thickness combination different from the film-thickness combination of the film A, are stacked.
  • the reflection rate of the stacked film gradually decreases from a short wavelength side, reaches 0 around W 1 nm, and gradually increases towards a longer wavelength side.
US14/476,367 2012-03-16 2014-09-03 Image sensor and imaging device Abandoned US20150077524A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2012060753 2012-03-16
JP2012-060753 2012-03-16
PCT/JP2013/001812 WO2013136820A1 (ja) 2012-03-16 2013-03-15 撮像素子および撮像装置

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/001812 Continuation WO2013136820A1 (ja) 2012-03-16 2013-03-15 撮像素子および撮像装置

Publications (1)

Publication Number Publication Date
US20150077524A1 true US20150077524A1 (en) 2015-03-19

Family

ID=49160742

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/476,367 Abandoned US20150077524A1 (en) 2012-03-16 2014-09-03 Image sensor and imaging device

Country Status (4)

Country Link
US (1) US20150077524A1 (ja)
JP (1) JPWO2013136820A1 (ja)
CN (1) CN104303302A (ja)
WO (1) WO2013136820A1 (ja)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170347046A1 (en) * 2014-12-18 2017-11-30 Lg Innotek Co., Ltd. Image sensor, image acquiring device comprising same, and portable terminal comprising the device
US11302734B2 (en) 2018-06-29 2022-04-12 Taiwan Semiconductor Manufacturing Company, Ltd. Deep trench isolation structures resistant to cracking

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112335049B (zh) * 2018-08-24 2024-03-22 宁波舜宇光电信息有限公司 成像组件、触摸屏、摄像模组、智能终端、相机和距离测量方法
CN114973943A (zh) * 2019-04-03 2022-08-30 京东方科技集团股份有限公司 显示面板和显示装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5844289A (en) * 1996-05-21 1998-12-01 Nec Corporation Solid-state image sensor with microlens and optical fiber bundle
US6307648B1 (en) * 1998-07-06 2001-10-23 Mitsubishi Denki Kabushiki Kaisha Line source unit and image input apparatus
US7262404B2 (en) * 2003-11-18 2007-08-28 Matsushita Electric Industrial Co., Ltd. Photodetector manufacturable by semiconductor processes
US20070206242A1 (en) * 2006-03-06 2007-09-06 Micron Technology, Inc. Method, apparatus and system providing an integrated hyperspectral imager
US20080259191A1 (en) * 2004-09-17 2008-10-23 Kunihiro Imamura Image Input Apparatus that Resolves Color Difference
US20090256225A1 (en) * 2008-04-11 2009-10-15 Sharp Kabushiki Kaisha Solid-state image capturing device, manufacturing method of the solid-state image capturing device, and electronic information device
US20110290322A1 (en) * 2009-02-03 2011-12-01 Kaneka Corporation Substrate with transparent conductive film and thin film photoelectric conversion device

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04152674A (ja) * 1990-10-17 1992-05-26 Sony Corp 固体撮像素子
JPH0669523A (ja) * 1992-08-18 1994-03-11 Fujitsu Ltd 赤外線検知素子およびその製造方法
JP2000196051A (ja) * 1998-12-25 2000-07-14 Matsushita Electric Ind Co Ltd 固体撮像素子およびその製造方法
JP2005142510A (ja) * 2003-11-10 2005-06-02 Matsushita Electric Ind Co Ltd 固体撮像装置およびその製造方法
JP4807131B2 (ja) * 2006-04-05 2011-11-02 株式会社ニコン 撮像素子および撮像装置
JP2009099817A (ja) * 2007-10-18 2009-05-07 Nikon Corp 固体撮像素子
JP4798270B2 (ja) * 2009-02-13 2011-10-19 株式会社ニコン 撮像素子、撮像装置及び撮像素子の製造方法
JP5503209B2 (ja) * 2009-07-24 2014-05-28 キヤノン株式会社 撮像素子及び撮像装置
JP2012004264A (ja) * 2010-06-16 2012-01-05 Fujifilm Corp 固体撮像素子および撮影装置
JP2012003009A (ja) * 2010-06-16 2012-01-05 Fujifilm Corp 固体撮像素子及びその製造方法並びに撮影装置
JP2012038938A (ja) * 2010-08-06 2012-02-23 Canon Inc 固体撮像素子およびカメラ

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5844289A (en) * 1996-05-21 1998-12-01 Nec Corporation Solid-state image sensor with microlens and optical fiber bundle
US6307648B1 (en) * 1998-07-06 2001-10-23 Mitsubishi Denki Kabushiki Kaisha Line source unit and image input apparatus
US7262404B2 (en) * 2003-11-18 2007-08-28 Matsushita Electric Industrial Co., Ltd. Photodetector manufacturable by semiconductor processes
US20080259191A1 (en) * 2004-09-17 2008-10-23 Kunihiro Imamura Image Input Apparatus that Resolves Color Difference
US20070206242A1 (en) * 2006-03-06 2007-09-06 Micron Technology, Inc. Method, apparatus and system providing an integrated hyperspectral imager
US20090256225A1 (en) * 2008-04-11 2009-10-15 Sharp Kabushiki Kaisha Solid-state image capturing device, manufacturing method of the solid-state image capturing device, and electronic information device
US20110290322A1 (en) * 2009-02-03 2011-12-01 Kaneka Corporation Substrate with transparent conductive film and thin film photoelectric conversion device

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170347046A1 (en) * 2014-12-18 2017-11-30 Lg Innotek Co., Ltd. Image sensor, image acquiring device comprising same, and portable terminal comprising the device
US10057520B2 (en) * 2014-12-18 2018-08-21 Lg Innotek, Co., Ltd. Image sensor, image acquiring device comprising same, and portable terminal comprising the device
US11302734B2 (en) 2018-06-29 2022-04-12 Taiwan Semiconductor Manufacturing Company, Ltd. Deep trench isolation structures resistant to cracking

Also Published As

Publication number Publication date
JPWO2013136820A1 (ja) 2015-08-03
CN104303302A (zh) 2015-01-21
WO2013136820A1 (ja) 2013-09-19

Similar Documents

Publication Publication Date Title
US10412358B2 (en) Image sensor, image-capturing apparatus and image-capturing system
JP5915537B2 (ja) 撮像素子、及び、撮像装置
US9727985B2 (en) Image processing apparatus, image-capturing apparatus, and storage medium having image processing program stored thereon
US9838665B2 (en) Image processing device, imaging device, and image processing program
JP5621059B2 (ja) カラー撮像素子及び撮像装置
US20150077524A1 (en) Image sensor and imaging device
US9854224B2 (en) Image processing apparatus, image capturing apparatus, and image processing program
US20140307060A1 (en) Image sensor
JP6197316B2 (ja) 撮像素子および撮像装置
WO2013038598A1 (ja) 撮像素子、撮像装置および画像処理装置
JP2013175812A (ja) 撮像素子および撮像装置
WO2012153504A1 (ja) 撮像装置および撮像装置の制御プログラム
JP6476630B2 (ja) 撮像装置
JP6205770B2 (ja) 撮像素子および撮像システム
JP6051568B2 (ja) 撮像素子および撮像装置
JP6019611B2 (ja) 撮像装置
JP5978737B2 (ja) 画像処理装置、撮像装置および画像処理プログラム
JP2013219180A (ja) 撮像素子および撮像装置
JP5978735B2 (ja) 画像処理装置、撮像装置および画像処理プログラム
JP2013150055A (ja) 画像処理装置、画像処理方法、及び、プログラム

Legal Events

Date Code Title Description
AS Assignment

Owner name: NIKON CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SUZUKI, SATOSHI;REEL/FRAME:034120/0322

Effective date: 20141014

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION