WO2015093332A1 - 撮像モジュール及び撮像装置 - Google Patents

撮像モジュール及び撮像装置 Download PDF

Info

Publication number
WO2015093332A1
WO2015093332A1 PCT/JP2014/082391 JP2014082391W WO2015093332A1 WO 2015093332 A1 WO2015093332 A1 WO 2015093332A1 JP 2014082391 W JP2014082391 W JP 2014082391W WO 2015093332 A1 WO2015093332 A1 WO 2015093332A1
Authority
WO
WIPO (PCT)
Prior art keywords
optical system
light receiving
image
pupil
annular
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2014/082391
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
小野　修司
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to CN201480067516.6A priority Critical patent/CN105814882B/zh
Priority to EP14871445.4A priority patent/EP3086546B1/en
Publication of WO2015093332A1 publication Critical patent/WO2015093332A1/ja
Priority to US15/163,755 priority patent/US10027918B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G02B13/04—Reversed telephoto objectives
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B17/00—Systems with reflecting surfaces, with or without refracting elements
    • G02B17/08—Catadioptric systems
    • G02B17/0804—Catadioptric systems using two curved mirrors
    • G02B17/0808—Catadioptric systems using two curved mirrors on-axis systems with at least one of the mirrors having a central aperture
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B17/00—Systems with reflecting surfaces, with or without refracting elements
    • G02B17/08—Catadioptric systems
    • G02B17/0856—Catadioptric systems comprising a refractive element with a reflective surface, the reflection taking place inside the element, e.g. Mangin mirrors
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B3/00—Simple or compound lenses
    • G02B3/0006—Arrays
    • G02B3/0037—Arrays characterized by the distribution or form of lenses
    • G02B3/0056—Arrays characterized by the distribution or form of lenses arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/58—Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/843—Demosaicing, e.g. interpolating colour pixel values
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H04N25/447—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by preserving the colour pattern with or without loss of information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/78—Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters

Definitions

  • the present invention relates to an imaging module and an imaging apparatus, and more particularly to an imaging module and an imaging apparatus that can simultaneously capture a plurality of images having different characteristics.
  • An imaging optical system 1 composed of an annular optical system 1b having different characteristics, an image sensor 3, and an array composed of a plurality of microlenses (pupil imaging lenses) disposed on the incident surface side of the image sensor 3.
  • an imaging apparatus that includes a lens 2 and an array lens 2 that forms a pupil image of a photographing optical system on an image sensor 3 using each microlens (Patent Document 1).
  • the image plane of the photographing optical system 1 is on the array lens 2, and the array lens 2 forms a pupil image of the photographing optical system 1 on the image sensor 3.
  • FIG. 21 shows one light receiving cell 3 a on the image sensor 3 and a pupil image of the photographing optical system 1 that is imaged on the image sensor 3 by one microlens of the array lens 2.
  • This pupil image has a central pupil image (wide-angle lens component) corresponding to the central optical system 1a and an annular pupil image (telephoto lens component) corresponding to the annular optical system 1b.
  • the center pupil image (wide-angle lens component) is received by the light receiving cell at the center for every 5 ⁇ 5 25 light receiving cell groups, and the light receiving cells at the periphery thereof.
  • An annular pupil image (telephoto lens component) is received.
  • an image signal for one pixel of the wide-angle image is generated from the light-receiving cell that receives the wide-angle lens component, and similarly, an image for one pixel of the telephoto image from the light-receiving cell that receives the telephoto lens component.
  • a wide-angle image corresponding to the wide-angle lens and a telephoto image corresponding to the telephoto lens are obtained as shown in FIGS. 22 (b) and 22 (c).
  • the simplest method for suppressing the decrease in the number of pixels of images with different characteristics obtained from the image sensor 3 is to reduce the number of light receiving cells (number of allocation) allocated to one microlens.
  • the number of pixels of images with different characteristics that can be extracted can be increased by the amount corresponding to the reduced number of allocation.
  • 23A and 23B show an example in which 5 ⁇ 5 light receiving cells 3a of the image sensor 3 are assigned to one microlens, and an example in which 3 ⁇ 3 light receiving cells 3a are assigned to each microlens. Is shown.
  • the number of light receiving cells that can be assigned per microlens of the array lens is limited to 3 ⁇ 3.
  • the light receiving cells of the image sensor 3 are limited.
  • Patent Document 1 there is a description that color filters are arranged in a light receiving element in a predetermined pattern in order to capture a color image, but there is no description regarding a specific color filter arrangement.
  • Patent Documents 2 and 3 use a general imaging lens and an array lens (microlens array) arranged on the incident surface side of the image sensor, and pass the passing light of the imaging lens by the lens array.
  • an imaging device that obtains a pixel signal based on the amount of received light by being incident on each light receiving cell of an image sensor while being separated into light rays from a plurality of viewpoints.
  • Patent Documents 2 and 3 describe that a light beam that has passed through one microlens is received by 3 ⁇ 3 light receiving cells, and further, a Bayer array color filter is provided on the image sensor to receive light. There is a description that any one of red (R), green (G), and blue (B) color filters is provided for each cell.
  • the generated viewpoint image is also a Bayer array color image (mosaic image) (see FIG. 11 of Patent Document 2).
  • the pupil dividing means there is one in which a light beam that has passed through each region having different characteristics of various lenses is incident on different light receiving cells by a micro lens and a light shielding mask provided for each light receiving cell (Patent Document 4). ).
  • an image sensor having a general Bayer arrangement as described in Patent Documents 2 and 3 as shown in FIG. Consider an imaging apparatus having a configuration in which the light receiving cells are assigned to one microlens. Note that an imaging apparatus having this configuration is not a known one.
  • the image sensor and array lens shown in part (a) of FIG. 24 have a grid-like 6 ⁇ 6 light receiving cell (2 ⁇ 2 array lens) as a basic block, and this basic block is repeated in the horizontal and vertical directions. Arranged and configured.
  • FIG. 24 (b) shows a basic block.
  • the basic block is composed of four unit blocks, with one microlens and 3 ⁇ 3 light receiving cells per one microlens as unit blocks. ing.
  • the (c1) part and (c2) part of FIG. 24 are respectively a group of light receiving cells in the center of the unit block (3 ⁇ 3) (light receiving cells into which a light beam having passed through the central optical system 1a shown in FIG. 20 is incident). And a group of eight surrounding light receiving cells (light receiving cells into which a light beam that has passed through the annular optical system 1b shown in FIG. 20 is incident).
  • the image of the central light receiving cell group is a mosaic image of the Bayer array.
  • a color image can be obtained without any problem by demosaicing (also referred to as synchronization processing) the Bayer array mosaic image.
  • a group of eight surrounding light receiving cells includes eight light receiving cells including all RGB light receiving cells and eight light receiving cells having no R light receiving cells. And 8 light receiving cells having no B light receiving cells are mixed, and the arrangement of RGB light receiving cells is not balanced.
  • the eight light receiving cells around the 3 ⁇ 3 light receiving cell whose central light receiving cell is a G light receiving cell are two R light receiving cells, four G light receiving cells, and two G light receiving cells. B light receiving cells, and there is all RGB color information.
  • the 8 light receiving cells around the 3 ⁇ 3 light receiving cell whose center light receiving cell is R have 4 G light receiving cells, 4 B light receiving cells, and the R light receiving cells are Similarly, the eight light receiving cells around the 3 ⁇ 3 light receiving cell whose central light receiving cell is B have four R light receiving cells, four G light receiving cells, and B light receiving cells. There is no cell.
  • the 8 light receiving cells around the 3 ⁇ 3 light receiving cell without the R light receiving cell or the B light receiving cell use the R light receiving cell or the B light receiving cell acquired in the adjacent unit block. Therefore, there is a problem that the processing for complementation is required, which is troublesome and the resolution performance of an image generated by a group of eight surrounding light receiving cells deteriorates.
  • the number of light receiving cells that can be assigned per microlens of the array lens is limited to 3 ⁇ 3.
  • the number of pixels of different images is reduced.
  • the present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging module and an imaging apparatus capable of improving the image quality and resolution of a plurality of images with different characteristics that are simultaneously captured.
  • an imaging module simultaneously captures X (X is an integer of 2 or more) subject images having different characteristics, and at least one subject of the X subject images.
  • An imaging module that outputs pixel signals of a plurality of wavelength ranges to an image, and has X regions on which subject light is incident, and subject images having different characteristics for each of the X regions are superimposed on the same image plane
  • pupil dividing means for dividing the pupil image of the various lenses into X areas
  • X light receiving areas for receiving X pupil images divided by the pupil dividing means, respectively.
  • Y (an integer greater than or equal to X + 1) photoelectric converters, and a plurality of photoelectric converters are provided in at least one of the X light receiving areas. Multiple photoelectric converters arranged in one light receiving area And it outputs a pixel signal of the wavelength range.
  • the pupil images of various lenses having different characteristics for each of the X regions, and the X pupil images divided by the pupil dividing unit correspond to the X light receiving regions. It is made to inject into Y (an integer greater than or equal to X + 1) photoelectric converters.
  • a plurality of photoelectric converters are arranged in at least one of the X light receiving areas, and the plurality of photoelectric converters output pixel signals in a plurality of wavelength ranges, so that the characteristics of being simultaneously imaged
  • pixel signals in a plurality of wavelength ranges can be obtained from a plurality of photoelectric converters corresponding to one pupil image, and the resolution per pixel is improved. Can be made.
  • the plurality of photoelectric converters arranged in one light receiving region include all the pixels necessary for generating pixels that form an image including information of a plurality of wavelength regions. It is preferable to output a pixel signal in the wavelength range.
  • the plurality of photoelectric converters arranged in one light receiving region outputs pixel signals in all the wavelength regions necessary for generating pixels constituting an image composed of information in a plurality of wavelength regions.
  • a demosaic image signal of one pixel can be obtained using only pixel signals from a plurality of photoelectric converters arranged in one light receiving region. .
  • all of the pixel signals output from a plurality of photoelectric converters arranged in one light receiving region are all necessary for generating pixels that constitute an image composed of information of a plurality of wavelength ranges.
  • the pixel signals (pixels in the specific wavelength range) output from the surrounding Y photoelectric converters Signal) must be interpolated to generate a pixel signal in a specific wavelength range.
  • the resolution (substantial number of pixels) of the output image is reduced. According to the aspect, such a problem can be solved.
  • the pupil dividing means is an array lens composed of two-dimensionally arranged microlenses, and is arranged on the incident surface side of the photoelectric converter. It is preferable that the lens is an array lens in which pupil images of various lenses are incident on Y photoelectric converters by a microlens.
  • the plurality of photoelectric converters arranged in one light receiving region are pixel signals in the same wavelength region. It is preferable that the two or more photoelectric converters are arranged symmetrically with respect to the center of the unit block.
  • the plurality of photoelectric converters disposed in one light receiving region include pixels for each of the red (R), green (G), and blue (B) wavelength regions. Output a signal. Thereby, all color information can be acquired from a plurality of photoelectric converters arranged in one light receiving region, and a high-definition color image can be generated.
  • the various lenses are provided in the central first optical system and in the peripheral portion of the first optical system, and the second has different characteristics from the first optical system. Those consisting of these optical systems are preferred.
  • the various lenses formed by the central first optical system and the second optical system provided at the periphery of the first optical system are, for example, in comparison with a photographing optical system composed of an optical system divided vertically. Lens performance will be excellent.
  • one of the first optical system and the second optical system is a wide-angle optical system and the other is a telephoto optical system.
  • the first optical system of the various lenses is a circular central optical system
  • the second optical system is a ring arranged concentrically with respect to the central optical system.
  • An optical system is preferable.
  • the first optical system of the various lenses is a circular central optical system
  • the second optical system is a ring arranged concentrically with respect to the central optical system.
  • the unit block has 3 ⁇ 3 photoelectric converters arranged in a lattice shape, and corresponds to the central optical system.
  • the pupil image is preferably incident on the photoelectric converter in the center of the unit block, and the annular pupil image corresponding to the annular optical system is preferably incident on eight photoelectric converters around the unit block.
  • the photoelectric converter at the center of the unit block may be any one of pixel signals in the wavelength range of red (R), green (G), and blue (B). It is preferable that the photoelectric converters that output pixel signals and that output pixel signals in the R, G, and B wavelength regions are periodically arranged in the central photoelectric converter of the plurality of unit blocks. Accordingly, the pixel signal output from the central photoelectric converter of the 3 ⁇ 3 photoelectric converters includes pixel signals in the R, G, and B wavelength regions, and is all necessary to generate one image. Can be obtained.
  • the eight photoelectric converters around the unit block include four photoelectric converters that output pixel signals in the G wavelength region, and pixels in the R wavelength region. It is preferable to include two photoelectric converters that output signals and two photoelectric converters that output pixel signals in the B wavelength region. As a result, the ratio of the number of RGB pixel signals is 1: 2: 1, and a large number of G photoelectric converters that contribute most to obtaining a luminance signal can be arranged.
  • an image sensor is configured by using 6 ⁇ 6 photoelectric converters arranged in a grid as basic blocks, and the basic blocks are repeatedly arranged in the horizontal and vertical directions. It is preferable. Since the basic blocks of 6 ⁇ 6 photoelectric converters are repeated in the horizontal direction and the vertical direction, when performing image processing such as demosaic processing in the subsequent stage, processing can be performed according to a repetition pattern.
  • the array lens overlaps a part of adjacent pupil images on the photoelectric converter among the pupil images incident on the photoelectric converter by each microlens. It is preferable to make it.
  • the array lens a part of the pupil images adjacent to each other are overlapped on the photoelectric converter, so that the number of photoelectric converters allocated substantially per microlens of the array lens is 3 ⁇ 3. As a result, it is possible to increase the number of pixels of images with different characteristics captured simultaneously.
  • the various lenses are provided in the central first optical system and in the peripheral portion of the first optical system, and the second has different characteristics from the first optical system. If the pupil dividing means and the Y photoelectric converters are unit blocks, the unit block has 3 ⁇ 3 photoelectric converters arranged in a lattice pattern, and is the first of various lenses.
  • the optical system is a circular central optical system
  • the second optical system is an annular optical system arranged concentrically with respect to the central optical system
  • the central pupil image corresponding to the first optical system is The annular pupil image corresponding to the second optical system that is incident on the photoelectric converter in the center of the unit block is incident on the eight photoelectric converters around the unit block, and is arranged in a lattice shape.
  • the basic block is repeatedly arranged in the horizontal and vertical directions. It is preferable that the image sensor is configured.
  • the various lenses are provided in the central first optical system and in the peripheral portion of the first optical system, and the second has different characteristics from the first optical system.
  • the first optical system of the various lenses is a circular central optical system
  • the second optical system is an annular optical system arranged concentrically with respect to the central optical system, and an array
  • the lens overlaps a part of the annular pupil images corresponding to the annular optical systems adjacent to each other on the Y photoelectric converters among the pupil images formed on the Y photoelectric converters by the microlenses.
  • the central pupil image corresponding to the adjacent central optical system overlaps with a part of the annular pupil image corresponding to the second optical system, and the annular optical system overlaps with the central pupil image corresponding to the central optical system.
  • the part corresponding to a part of the annular pupil image is shielded or centered It is preferred that the portion corresponding to a portion of the annular pupil image that overlaps with the central pupil image corresponding to the academic system is formed missing.
  • the annular optical system is formed such that a part thereof is shielded from light or a part thereof is omitted so that the central pupil image and the annular pupil image do not overlap on the photoelectric converter. ing.
  • the substantial number of photoelectric converters allocated to one microlens of the array lens can be further reduced.
  • the various lenses are provided in the central first optical system and in the peripheral portion of the first optical system, and the second has different characteristics from the first optical system.
  • the first optical system of the various lenses is a circular central optical system
  • the second optical system is an annular optical system disposed concentrically with respect to the central optical system
  • An annular optical system in which third and fourth optical systems having different characteristics are alternately arranged, and the array lens is a pupil image formed on each of Y photoelectric converters by each microlens.
  • the first annular pupil images corresponding to the third optical system of the annular optical systems adjacent to each other are overlapped on the Y photoelectric converters
  • the fourth optical system of the annular optical system adjacent to each other is used.
  • the corresponding second annular pupil images are overlapped on the Y photoelectric converters.
  • Door is preferable. Thereby, three types of images having different characteristics can be simultaneously acquired by one imaging.
  • the unit block is a single central photoelectric module.
  • a central pupil image corresponding to the central optical system is incident on the central photoelectric converter and includes a first optical system corresponding to the third optical system of the annular optical system.
  • the annular pupil image is incident on three photoelectric converters in 120 degrees and three directions from the central photoelectric converter among the six surrounding photoelectric converters, and corresponds to the fourth optical system of the annular optical system. It is preferable that the second annular pupil image is incident on the other three photoelectric converters at 120 degrees and three directions from the central photoelectric converter among the six surrounding photoelectric converters.
  • the central optical system of the various lenses is a wide-angle optical system
  • the third optical system and the fourth optical system of the annular optical system have different focal lengths. It is preferable that Thereby, it is possible to simultaneously acquire a wide-angle image and two telephoto images having different shooting magnifications by one imaging.
  • the central optical system of the various lenses is a wide-angle optical system
  • the third optical system and the fourth optical system of the annular optical system are telephoto optical systems having different shooting distances. It is preferable that Accordingly, it is possible to simultaneously acquire a wide-angle image and two telephoto images focused on subjects having different shooting distances by one imaging.
  • the annular optical system has a reflection optical system that reflects the light beam twice or more.
  • the dimension of the annular optical system in the optical axis direction can be shortened, and the apparatus can be made compact.
  • An imaging apparatus outputs any one of the imaging modules described above, the pupil dividing unit, and Y photoelectric converters as unit blocks, and is output from the photoelectric converters in one unit block.
  • an image generation unit that generates an image signal for one pixel constituting at least one image composed of information of a plurality of wavelength regions based on the pixel signal.
  • a demosaiced image signal for one pixel is generated using a pixel signal in a unit block for at least one of X images that are simultaneously captured.
  • the image quality and resolution of the image can be improved.
  • the present invention it is possible to improve the image quality and resolution of at least one of a plurality of images that are simultaneously imaged and have different characteristics.
  • the array lens overlaps a part of pupil images of various adjacent lenses on the photoelectric converter, the number of photoelectric converters allocated substantially per microlens of the array lens is reduced. As a result, the number of pixels of images having different characteristics that can be simultaneously imaged can be increased.
  • 1 is an external perspective view of an imaging apparatus including an imaging module according to the present invention.
  • 1 is a block diagram showing an embodiment of an internal configuration of the imaging apparatus shown in FIG. Sectional drawing which shows 1st Embodiment of the imaging optical system applied to the imaging device shown in FIG. The principal part enlarged view of the array lens used in order to demonstrate 1st Embodiment of the imaging device which concerns on this invention, and an image sensor
  • the figure used for explaining the suitable color filter arrangement in the image sensor Other figures used to describe a preferred color filter array in an image sensor
  • the block diagram which shows the principal part constitution of the smart phone The figure which shows the conventional imaging device provided with the imaging
  • photography optical system which has a center optical system and a ring optical system, an array lens, and an image sensor.
  • the figure which shows the relationship between one light receiving cell and a pupil image The figure which shows an example of each pupil image imaged on the conventional image sensor
  • the figure which shows the other example of each pupil image imaged on the conventional image sensor The figure used for explaining the problem to be solved by the invention
  • FIG. 1 is an external perspective view of an imaging apparatus including an imaging module according to the present invention.
  • a variety of lenses (shooting optical system) 12 a flash light emitting unit 19 and the like are disposed on the front surface of the imaging apparatus 10, and a shutter button 38-1 is provided on the upper surface.
  • L1 represents the optical axis of the various lenses 12.
  • FIG. 2 is a block diagram showing an embodiment of the internal configuration of the imaging apparatus 10.
  • the image pickup apparatus 10 records the picked-up image on the memory card 54, and is mainly characterized by the image pickup module 11 including the various lenses 12, the array lens 16, the image sensor 18, and the like.
  • FIG. 3 is a cross-sectional view showing a first embodiment of a variety of lenses applied to the imaging apparatus 10 (imaging module 11).
  • the various lenses 12 each have a central optical system (first optical system) 13 and an annular optical system (second optical system) at the periphery thereof arranged on the same optical axis. 14.
  • the central optical system 13 is a wide-angle optical system (wide-angle lens) including a first lens 13 a, a second lens 13 b, a third lens 13 c, a fourth lens 13 d, and a common lens 15. An image is formed.
  • the annular optical system 14 includes a first lens 14a, a second lens 14b, a first reflection mirror 14c (reflection optical system), a second reflection mirror 14d (reflection optical system), and a telephoto optical system (common lens 15).
  • a telephoto lens), and a telephoto image is formed on the array lens 16.
  • the light beam incident through the first lens 14a and the second lens 14b is reflected twice by the first reflection mirror 14c and the second reflection mirror 14d, and then passes through the common lens 15.
  • the light beam is folded back by the first reflecting mirror 14c and the second reflecting mirror 14d, so that the length in the optical axis direction of the telephoto optical system (telephoto lens) having a long focal length is shortened.
  • FIG. 4 is an enlarged view of a main part of the array lens 16 and the image sensor 18 shown in FIGS.
  • the array lens 16 is configured by two-dimensionally arranging a plurality of microlenses (pupil imaging lenses) 16a.
  • the horizontal and vertical intervals between the microlenses are the light receiving cells (photoelectric cells) of the image sensor 18. This corresponds to three intervals of the converter 18a. That is, the microlenses of the array lens 16 are formed corresponding to the positions of every two light receiving cells in the horizontal and vertical directions.
  • Each micro lens 16a of the array lens 16 has a circular central pupil image (first pupil image) 17a and an annular pupil image (second pupil image) corresponding to the central optical system 13 and the annular optical system 14 of the various lenses 12.
  • a pupil image) 17b is formed on the light receiving cell 18a in the corresponding light receiving region of the image sensor 18.
  • the array lens 16 and the image sensor 18 of the first embodiment shown in FIG. 4 there are 3 ⁇ 3 light receiving cells 18 a having a lattice shape (square lattice shape) per 1 micro lens 16 a of the array lens 16. Assigned.
  • one microlens 16a and a light receiving cell group (3 ⁇ 3 light receiving cells 18a) corresponding to one microlens 16a are referred to as a unit block.
  • the central pupil image 17a is formed only on the light receiving cell 18a at the center of the unit block, and the annular pupil image 17b is formed on eight light receiving cells 18a around the unit block.
  • the imaging apparatus 10 (imaging module 11) according to the present invention can simultaneously capture a wide-angle image corresponding to the central optical system 13 and a telephoto image corresponding to the annular optical system 14, as will be described later.
  • FIG. 5 is a diagram showing an image sensor 18 applied to the image pickup apparatus 10 (image pickup module 11) according to the present invention.
  • the array lens 16 is omitted, but the area indicated by a circle is a unit including 3 ⁇ 3 light receiving cells on which pupil images are formed by the microlenses 16 a of the array lens 16. Indicates a block.
  • a color filter array composed of color filters arranged on each light receiving cell is provided.
  • This color filter array is constituted by three primary color filters (hereinafter referred to as R filter, G filter, and B filter) that transmit light in each wavelength band of red (R), green (G), and blue (B).
  • R filter three primary color filters
  • G filter three primary color filters
  • B filter blue filter
  • One of the RGB filters is disposed on each light receiving cell.
  • the light receiving cell in which the R filter is disposed is referred to as “R light receiving cell”
  • the light receiving cell in which the G filter is disposed is referred to as “G light receiving cell”
  • B light receiving cell the light receiving cell in which the B filter is disposed
  • the color filter array shown in part (a) of FIG. 5 includes 6 ⁇ 6 light receiving cells as basic blocks B (blocks indicated by a thick frame in part (a) of FIG. 5 and part (b) of FIG. 5). ), And the basic block B is repeatedly arranged in the horizontal direction and the vertical direction.
  • the basic block B is composed of four unit blocks B1 to B4.
  • (C1) and (c2) in FIG. 5 are groups of light receiving cells (light receiving cells into which light beams having passed through the central optical system 13 shown in FIG. 3 are incident) at the center of each of the four unit blocks B1 to B4. And a group of eight surrounding light receiving cells (light receiving cells into which the speed of light that has passed through the annular optical system 14 shown in FIG. 3 is incident).
  • the image of the central light receiving cell group is a Bayer array mosaic image.
  • a color image can be obtained without any problem by demosaicing the Bayer array mosaic image.
  • a group of eight light receiving cells around each central light receiving cell of the unit blocks B1 to B4 includes all the light receiving cells of RGB in the eight light receiving cells. (R light receiving cell, G light receiving cell, B light receiving cell) and the RGB light receiving cells are arranged in the same pattern regardless of the unit blocks B1 to B4.
  • the G light receiving cells are arranged in the four light receiving cells at the four corners of each of the unit blocks B1 to B4, and the R light receiving cells are arranged in the two upper and lower light receiving cells across the center light receiving cell.
  • the B light receiving cells are arranged in the two left and right light receiving cells with the central light receiving cell interposed therebetween.
  • the R light receiving cell, the G light receiving cell, and the B light receiving cell are respectively arranged at symmetrical positions with respect to the center light receiving cell (center) of the unit block.
  • the pixel value of the G pixel at the center position of the unit block (1 microlens) can be obtained.
  • Pixel values can be acquired.
  • the pixel values of the RGB light receiving cells in the unit block are used. Therefore, it is not necessary to generate pixel values of pixels in a specific wavelength range by interpolating the pixel values of the light receiving cells of the surrounding unit blocks, and the resolution of the output image (substantial number of pixels) ) Is not reduced.
  • the eight light receiving cells around the unit blocks B1 to B4 have two R light receiving cells, four G light receiving cells, and two B light receiving cells.
  • the ratio of the RGB light receiving cells is 1 : 2: 1, and many G light receiving cells that contribute most to obtaining a luminance signal are arranged.
  • the imaging apparatus 10 (imaging module 11) includes the various lenses 12 having the central optical system 13 and the annular optical system 14 described with reference to FIG. 3, and the first embodiment described with reference to FIGS. 4 and 5.
  • the array lens 16 and the image sensor 18 are provided.
  • the operation of the entire apparatus is centrally controlled by a central processing unit (CPU) 40 based on a camera control program stored in an EEPROM (ElectricallyrErasable Programmable Read-Only Memory) 56.
  • the EEPROM 56 stores various types of parameters and tables used for pixel defect information, image processing, and the like of the image sensor 14.
  • the imaging device 10 is provided with an operation unit 38 such as a shutter button 38-1, a mode dial (mode switching device), a playback button, a MENU / OK key, a cross key, and a BACK key.
  • a signal from the operation unit 38 is input to the CPU 40, and the CPU 40 controls each circuit of the imaging device 10 based on the input signal. For example, photographing operation control, image processing control, image data recording / reproduction control, liquid crystal monitor The display control of the (LCD) 30 is performed.
  • the shutter button 38-1 (FIG. 1) is an operation button for inputting an instruction to start shooting, and is composed of a two-stroke switch having an S1 switch that is turned ON when half-pressed and an S2 switch that is turned ON when fully pressed. Has been.
  • the mode dial is a selection means for switching between an auto shooting mode for shooting a still image, a manual shooting mode, a scene position such as a person, a landscape, a night view, and a moving image mode for shooting a moving image.
  • the mode dial is a first imaging mode for obtaining a wide-angle image (first image) formed through the central optical system 13 and a telephoto image formed through the annular optical system 14 in the imaging mode. It functions as a selection unit that switches between a second shooting mode for acquiring an image (second image), a hybrid shooting mode for simultaneously acquiring a wide-angle image and a telephoto image, and the like.
  • the playback button is a button for switching to a playback mode in which a captured still image or moving image is displayed on the liquid crystal monitor 30.
  • the MENU / OK key is an operation key having both a function as a menu button for instructing to display a menu on the screen of the liquid crystal monitor 30 and a function as an OK button for instructing confirmation and execution of the selection contents. It is.
  • the cross key is an operation unit for inputting instructions in four directions, up, down, left, and right, and functions as a button (cursor moving operation means) for selecting an item from the menu screen or instructing selection of various setting items from each menu. To do.
  • the up / down key of the cross key functions as a zoom switch for shooting or a playback zoom switch in playback mode
  • the left / right key functions as a frame advance (forward / reverse feed) button in playback mode.
  • the BACK key is used to delete a desired object such as a selection item, cancel an instruction content, or return to the previous operation state.
  • the subject light is imaged on the light receiving surface of the image sensor 18 through the various lenses 12 and the array lens 16.
  • each light receiving cell (photoelectric converter) of the image sensor 18 is converted into a signal voltage (or electric charge) of an amount corresponding to the amount of incident light.
  • the signal voltage (or charge) accumulated in the image sensor 18 is accumulated in the light receiving cell itself or an attached capacitor.
  • the stored signal voltage (or charge) is read by the sensor control unit 32 together with the selection of the light receiving cell position by using a method of a MOS type image pickup device (so-called CMOS sensor) using the XY address method.
  • CMOS sensor MOS type image pickup device
  • the pixel signal indicating the wide-angle image of the group of the light receiving cells in the center corresponding to the central optical system 13 from the image sensor 18 and the pixel indicating the telephoto image of the group of eight light receiving cells corresponding to the annular optical system 14. Signal can be read out.
  • the pixel signal (voltage signal) read from the image sensor 18 is converted into an output signal for each light receiving cell for the purpose of reducing correlated double sampling processing (noise (particularly thermal noise) included in the sensor output signal).
  • the pixel signal for each light-receiving cell is sampled and held by a process of obtaining accurate pixel data by taking the difference between the included feedthrough component level and the signal component level, and is amplified and applied to the A / D converter 20 .
  • the A / D converter 20 converts sequentially input pixel signals into digital signals and outputs them to the image input controller 22. Note that some MOS type sensors include an A / D converter. In this case, a digital signal is directly output from the image sensor 18.
  • the pixel signal indicating the wide-angle image and the pixel signal indicating the telephoto image can be selectively read by selecting the light receiving cell position of the image sensor 18 and reading the pixel signal.
  • a pixel signal indicating an image (a pixel signal indicating a Bayer array mosaic image) can be acquired, and on the other hand, the pixel signals of the eight light receiving cells on which the annular pupil image 17b of the image sensor 18 enters are selectively read out.
  • eight pixel signals can be acquired per microlens.
  • pixel signals are read from the image sensor 18 and temporarily stored in the memory (SDRAM) 48, and the same as described above based on the pixel signals stored in the memory 48 by the digital signal processing unit (image generation unit) 24.
  • image generation unit image generation unit
  • pixel signals of two images, a wide-angle image and a telephoto image may be grouped.
  • the digital signal processing unit 24 performs offset processing, gamma correction processing, and RGB processing on digital pixel signals (RGB dot-sequential R, G, and B signals) input via the image input controller 22.
  • Predetermined signal processing such as demosaic processing is performed on the mosaic image signal.
  • the demosaic process is a process of calculating all color information for each pixel from the RGB mosaic image corresponding to the color filter array of the single-plate image sensor 18, and is also referred to as a synchronization process.
  • this is a process of calculating color information of all RGB for each pixel from a mosaic image composed of RGB.
  • the demosaic processing unit included in the digital signal processing unit 24 has no R light receiving cell and B light receiving cell at the position of the G light receiving cell in the wide-angle image (Bayer array mosaic image).
  • R signal and B signal of the R light receiving cell and B light receiving cell are respectively interpolated to generate R signal and B signal at the position of the G light receiving cell.
  • the G light receiving cell around the R light receiving cell, the G signal of the B light receiving cell, and the B signal are respectively interpolated.
  • G signal and B signal are generated at the position of the R light receiving cell, and since there are no G light receiving cell and R light receiving cell at the position of the B light receiving cell of the mosaic image, the G light receiving cells around the B light receiving cell, The G signal and R signal at the position of the B light receiving cell are generated by interpolating the G signal and R signal of the R light receiving cell, respectively.
  • the telephoto image is composed of 8 mosaic images per microlens 16a (8 around the 3 ⁇ 3 unit block), and all the RGB color information (R light reception) is contained in the 8 light reception cells.
  • the demosaic processing unit constructs an image that is demosaic processed for each unit block using the output signals of the eight light receiving cells in the unit block.
  • One pixel RGB pixel value
  • a demosaic processing unit (image generation unit) that performs demosaic processing on a mosaic image of a telephoto image obtains an average value of pixel values of four G light receiving cells in the unit block, thereby obtaining a unit block (1 micron).
  • the G pixel value of the pixel at the center position of the lens is calculated, and similarly the average value of the pixel values of the two R light receiving cells and the average value of the pixel values of the two B light receiving cells in the unit block are obtained.
  • the R pixel value and the B pixel value of the pixel at the center position of each unit block are calculated.
  • the demosaic image of the telephoto image of the two demosaic images of the wide-angle image and the telephoto image generated by the demosaic processing unit is demosaiced using the output signals of the eight light receiving cells in the unit block.
  • the resolution is substantially higher than the demosaic image of the wide-angle image in which the demosaic process is performed using (interpolating) the output signals of the light receiving cells of the surrounding unit blocks.
  • the digital signal processing unit 24 performs RGB / YC conversion for generating the luminance signal Y and the color difference signals Cb and Cr from the RGB color information (R signal, G signal, and B signal) demosaiced by the demosaic processing unit. Do.
  • the image signal processed by the digital signal processing unit 24 is input to a VRAM (Video Random Access Memory) 50.
  • VRAM Video Random Access Memory
  • the image signal read from the VRAM 50 is encoded by the video encoder 28 and output to the liquid crystal monitor 30 provided on the back of the camera, whereby the subject image is displayed on the display screen of the liquid crystal monitor 30.
  • the CPU 40 starts the AE operation, and the image data output from the A / D converter 20 is the AE detection unit 44. Is taken in.
  • the AE detection unit 44 integrates the image signals of the entire screen, or integrates the image signals with different weights at the center and the periphery of the screen, and outputs the integrated value to the CPU 40.
  • the CPU 40 calculates the brightness of the subject (shooting Ev value) from the integrated value input from the AE detection unit 44, and based on this shooting Ev value, the aperture value of an aperture (not shown) and the electronic shutter ( (Shutter speed) is determined according to a predetermined program diagram, the aperture is controlled based on the determined aperture value, and the charge accumulation time in the image sensor 18 is determined via the sensor control unit 32 based on the determined shutter speed. Control.
  • image data output from the A / D converter 20 in response to the depression is stored from the image input controller 22 into the memory.
  • SDRAM “Synchronous Dynamic” RAM
  • the image signal temporarily stored in the memory 48 is appropriately read out by the digital signal processing unit 24, where predetermined signal processing is performed and stored in the memory 48 again.
  • the image signals stored in the memory 48 are respectively output to the compression / decompression processing unit 26, and after predetermined compression processing such as JPEG (Joint Photographic Experts Group) is performed, the image signal is sent to the memory card 54 via the media controller 52. To be recorded.
  • predetermined compression processing such as JPEG (Joint Photographic Experts Group)
  • a wide-angle image or a telephoto image can be selectively acquired.
  • the hybrid shooting mode is selected with the mode dial, the wide-angle image is selected. Images and telephoto images can be acquired simultaneously. Accordingly, it is possible to acquire a wide-angle image and a telephoto image without mechanical switching between the wide-angle optical system and the telephoto optical system and without a zoom operation of the zoom lens.
  • FIG. 6 is a diagram showing another embodiment of the image sensor 18 applied to the imaging device 10 (imaging module 11) according to the present invention, and in particular, a second embodiment of the color filter disposed in the image sensor 18. It is a figure which shows the color filter arrangement
  • the color filter array of another embodiment of the image sensor 18 includes 6 ⁇ 6 light receiving cells in the same manner as the color filter array shown in part (a) of FIG. 5.
  • a basic block B (refer to a block indicated by a thick frame in part (a) of FIG. 6 and a part (b) of FIG. 6) is formed by repeatedly arranging the basic block B in the horizontal direction and the vertical direction.
  • the basic block B is composed of four unit blocks B1 to B4.
  • the image of the center light receiving cell group is a mosaic image of a Bayer array.
  • a color image can be obtained without any problem by demosaicing the Bayer array mosaic image.
  • a group of eight light receiving cells around each central light receiving cell of the unit blocks B1 to B4 includes all the light receiving cells of RGB in the eight light receiving cells. (R light receiving cell, G light receiving cell, B light receiving cell) and the RGB light receiving cells are arranged in the same pattern regardless of the unit blocks B1 to B4.
  • the G light receiving cells are arranged in the four light receiving cells in the upper, lower, left, and right sides of the center light receiving cell of each unit block B1 to B4, and the two light receiving cells in the upper left and lower right of the unit block are , R light receiving cells are arranged, and B light receiving cells are arranged in the two light receiving cells at the upper right and lower left of the unit block.
  • the R light receiving cell, the G light receiving cell, and the B light receiving cell are respectively arranged at symmetrical positions with respect to the center light receiving cell (center) of the unit block. Accordingly, it is possible to generate one pixel (RGB pixel value) constituting an image after demosaic processing for each unit block using the output signal of the RGB light receiving cells in the unit block.
  • the pixel value of the G light receiving cell at the center position of the unit block (1 microlens) can be obtained.
  • the R pixel value and B of the pixel at the center position of the unit block respectively. Pixel values can be acquired.
  • the eight light receiving cells around the unit blocks B1 to B4 have two R light receiving cells, four G light receiving cells, and two B light receiving cells.
  • the ratio of the RGB light receiving cells is 1 : 2: 1, and many G light receiving cells that contribute most to obtaining a luminance signal are arranged.
  • color filters are periodically arranged in a fixed pattern in each central light-receiving cell so that the image of the group of light-receiving cells in the center of the unit blocks B1 to B4 becomes a Bayer array mosaic image.
  • the color filter arranged in each center light receiving cell may be a color filter such as a G stripe R / G complete checkered pattern, an X-Trans (registered trademark) arrangement, In short, it is only necessary that the color filters are arranged so that pixel signals in all wavelength regions can be obtained.
  • the color filter arrangement for the eight light receiving cells around the unit block having 3 ⁇ 3 light receiving cells may be various in addition to the first and second embodiments shown in FIGS. .
  • FIG. 7 (a) and 7 (b) show the first embodiment of the color filter shown in FIG. 5 (a), respectively, and transmit the annular optical system 14 of the various lenses 12, in particular.
  • the incident light beams are incident on the eight light receiving cells around the 3 ⁇ 3 unit blocks, which are incident due to the directivity of the microlens 16a.
  • the incident area of the light beam transmitted through the annular optical system 14 is a clean circular shape. There is no difference in RGB filter arrangement in the eight light receiving cells.
  • RGB light receiving cells that output pixel signals in each wavelength region are uniformly assigned to eight light receiving cells.
  • the crescent-shaped luminous flux is evenly distributed in the surrounding eight RGB light receiving cells.
  • the area ratio of each of the RGB light receiving cells that are incident and the crescent-shaped light beam is incident is substantially equal (G ⁇ R ⁇ B).
  • the eight light receiving cells in the surrounding RGB are not uniformly arranged in the surrounding direction, and each light receiving cell in RGB in which a crescent-shaped light beam enters.
  • the area ratio becomes non-uniform (G >> R> B).
  • the eight light receiving cells in the surrounding RGB are not uniformly arranged in the surrounding direction, and each light receiving in RGB in which a crescent-shaped light beam enters.
  • the filter arrangement of the eight light receiving cells around the unit block having 3 ⁇ 3 light receiving cells is the first, shown in the part (a) of FIG. 5 and the part (b) of FIG.
  • the RGB light receiving cells are arranged evenly with respect to the surrounding orientation (so that the center of gravity is at the center of the unit block).
  • 8 (a) and 8 (b) show the color filter arrangements of the first and second embodiments shown in FIG. 5 (a) and FIG. 6 (b), respectively.
  • wavelength filters color filters
  • the filter parts can be connected, so the manufacturing process for forming the wavelength filter is simplified.
  • the wavelength filters of pixels in contact with adjacent unit blocks are arranged to be the same type.
  • the G light receiving cell among the eight light receiving cells when the unit blocks having 3 ⁇ 3 light receiving cells are arranged is 2 ⁇ 2 4 G light receiving cells are adjacent to each other, and among the surrounding 8 light receiving cells, 2 R light receiving cells are adjacent in the vertical direction, and 2 B light receiving cells are horizontally disposed. Adjacent.
  • the color filter manufacturing process of the color filter array of the first embodiment is the same. Since it becomes simple and interference can be reduced, it can be said that it is a more suitable color filter arrangement.
  • FIG. 9 is an enlarged view of a main part of the array lens 116 and the image sensor 118 used for explaining the second embodiment of the imaging apparatus (imaging module) according to the present invention. Note that the imaging device of the second embodiment is different from the imaging device 10 of the first embodiment shown in FIGS. 1 to 3 mainly in the array lens 116 and the image sensor 118. Will be described.
  • the array lens 116 is configured by arranging a plurality of microlenses 116 a in a two-dimensional manner, and the distance between the microlenses in the horizontal direction and the vertical direction is two of the light receiving cells 118 a of the image sensor 118. Corresponds to the minute interval. That is, each microlens of the array lens 116 is formed corresponding to the position of every other light receiving cell in each of the horizontal direction and the vertical direction.
  • each micro lens 116 a of the array lens 116 forms a circular central pupil image 117 a and an annular pupil image 117 b corresponding to the central optical system 13 and the annular optical system 14 of the various lenses 12 on the image sensor 118.
  • the annular pupil images 117b adjacent to each other partially overlap on the image sensor 118. That is, the array lens 116 is disposed at an appropriate position on the incident surface side of the image sensor 118, and among the central pupil image 117a and the annular pupil image 117b formed on the image sensor 118 by each micro lens 116a, respectively. A part of the annular pupil images 117b adjacent to each other is configured to overlap on the image sensor 118.
  • the central pupil image 117a is only in one light receiving cell 118a (3 ⁇ 3 central light receiving cells) on the image sensor 118.
  • the annular pupil image 117b is formed on the eight light receiving cells 118a around the light receiving cell 118a on which the central pupil image 117a is formed.
  • the central pupil image 117a formed on the eight light receiving cells 118a overlaps with the central pupil image 117a adjacent in the horizontal direction and the vertical direction (up and down, left and right directions) in the range of one light receiving cell.
  • the imaging apparatus only needs to be able to capture a wide-angle image corresponding to the central optical system 13 and a telephoto image corresponding to the annular optical system 14, so that the central pupil image 117a and the annular pupil image 117b overlap. If not, it ’s good. That is, even if a part of the annular pupil images 117b adjacent to each other overlaps on the image sensor 118, the image does not fail.
  • FIG. 10 (a) and 10 (b) are diagrams showing a color filter array and the like disposed in the image sensor 118.
  • FIG. 10 (a) and 10 (b) are diagrams showing a color filter array and the like disposed in the image sensor 118.
  • FIG. 10 (a) and 10 (b) are diagrams showing a color filter array and the like disposed in the image sensor 118.
  • the color filter array arranged in the image sensor 118 includes 4 ⁇ 4 light receiving cells in the basic block B (the block indicated by the thick frame in the part (a) of FIG. ), And the basic block B is repeatedly arranged in the horizontal direction and the vertical direction.
  • the image sensor 118 has unit blocks (four types of unit blocks B1 to B4) in which 3 ⁇ 3 light receiving cells are allocated per microlens 16a. Adjacent unit blocks overlap each other within the range of one light receiving cell.
  • the four unit blocks B1 to B4 shown in part (b) of FIG. 10 have the same color filter array as the four unit blocks B1 to B4 shown in part (b) of FIG.
  • the image of the center light receiving cell group is a Bayer array mosaic image.
  • each central light receiving cell of the unit blocks B1 to B4 four light receiving cells at the four corners of each unit block B1 to B4 are arranged with the G light receiving cells.
  • R light receiving cells are arranged in the two upper and lower light receiving cells with the light receiving cell in between, and B light receiving cells are arranged in the two left and right light receiving cells with the center light receiving cell in between.
  • the RGB light receiving cells of the light receiving cells are arranged uniformly in the surrounding direction.
  • the second embodiment of the imaging apparatus of the present invention when the number of light receiving cells of the image sensor 118 is M and the number of pixels of the wide-angle image and the telephoto image obtained from the image sensor is N,
  • FIG. 11A and FIG. 11B are diagrams showing a first modification of the second embodiment, respectively.
  • the first modification of the second embodiment is a color of an image sensor.
  • the filter arrangement is different from that of the second embodiment.
  • the image sensor 218 shown in part (a) of FIG. 11 uses 4 ⁇ 4 light receiving cells as basic blocks B (blocks indicated by thick frames in part (a) of FIG. 11). It is configured to be repeatedly arranged in the vertical direction.
  • the image sensor 218 includes unit blocks (four types of unit blocks B1 to B4) in which 3 ⁇ 3 light receiving cells are allocated per microlens 16a. Adjacent unit blocks overlap each other within the range of one light receiving cell.
  • the eight light receiving cells around the four unit blocks B1 to B4 are all two R light receiving cells, four G light receiving cells, and two B light receiving cells. have.
  • the unit blocks B1 and B4 whose center is the G light receiving cell have two patterns in which the arrangement of the R light receiving cell and the B light receiving cell is different.
  • FIGS. 12A and 12B are diagrams showing a second modification of the second embodiment, respectively.
  • the second modification of the second embodiment is a color of an image sensor.
  • the filter arrangement is different from the previous embodiment.
  • the image sensor 318 shown in part (a) of FIG. 12 uses 4 ⁇ 4 light receiving cells as basic blocks B (blocks indicated by thick frames in part (a) of FIG. 12). It is configured to be repeatedly arranged in the vertical direction.
  • one set of 2 ⁇ 2 R light receiving cells is included in the basic block B, and 2 ⁇ 2 G light receiving cells are included.
  • the image sensor 318 has unit blocks (4 types of unit blocks B1 to B4) in which 3 ⁇ 3 light receiving cells are allocated per microlens 16a. Adjacent unit blocks overlap each other within the range of one light receiving cell.
  • the eight light receiving cells around the unit blocks B1 and B4 whose center is the G light receiving cell among the four unit blocks B1 to B4 are two R light receiving cells and four.
  • G light receiving cells and two B light receiving cells, and the eight light receiving cells around the unit block B2 having the R light receiving cell at the center are three R light receiving cells, four G light receiving cells, and 1
  • the eight light receiving cells around the unit block B3 having the B light receiving cell at the center have one R light receiving cell, four G light receiving cells, and three B light receiving cells.
  • the color balance of the RGB light receiving cells is lower than that of the first modification of the second embodiment.
  • Part (a) and part (b) of FIG. 13 are views showing a third modification of the second embodiment, and in particular, the third modification of the second embodiment is the color of the image sensor.
  • the filter arrangement is different from the previous embodiment.
  • the image sensor 418 shown in the part (a) of FIG. 13 uses the square lattice-like 4 ⁇ 4 light receiving cells as the basic block B (the block indicated by the thick frame in the part (a) of FIG. 13). Are repeatedly arranged in the horizontal direction and the vertical direction.
  • a set of 2 ⁇ 2 G light receiving cells exists in the basic block B as shown in FIG.
  • the manufacturing process is simplified and interference can be reduced.
  • the image sensor 418 has unit blocks (four types of unit blocks B1 to B4) in which 3 ⁇ 3 light receiving cells are allocated per microlens 16a. Adjacent unit blocks overlap each other within the range of one light receiving cell.
  • the eight light receiving cells around the unit blocks B1 and B4 having the G light receiving cell at the center among the four unit blocks B1 to B4 are composed of two R light receiving cells and four R light receiving cells.
  • the G light receiving cells and the two B light receiving cells, the eight light receiving cells around the unit block B2 having the R light receiving cell at the center are one R light receiving cell, four G light receiving cells, and 3
  • the eight light receiving cells around the unit block B3 having the B light receiving cell at the center have three R light receiving cells, four G light receiving cells, and one B light receiving cell.
  • the color balance of the RGB light receiving cells is lower than that of the first modification of the second embodiment.
  • the imaging apparatus according to the third embodiment is different from the first and second embodiments mainly in a variety of lenses, an array lens, and an image sensor, and the differences will be described below.
  • the various lenses instead of the above-described various lenses 12 from which the central pupil image 17a and the annular pupil image 17b shown in the portion (a) of FIG. 14 are obtained, the central pupil image 517a shown in the portion (b) of FIG. What uses the annular pupil image 517b is used.
  • the various lenses from which the central pupil image 517a and the annular pupil image 517b shown in part (b) of FIG. 14 are obtained are configured by shielding a part of the annular optical system 14 of the various lenses 12 shown in FIG. can do.
  • the annular optical system corresponding to the annular pupil image 517b can be configured by forming partial openings only in the upper, lower, left and right directions and shielding the other portions. As a result, a partially missing annular pupil image 517b is obtained.
  • the ring is formed only in the upper, lower, left and right peripheral portions (positions corresponding to the partial opening of the ring optical system)
  • Four optical systems having the same characteristics as the optical system may be arranged.
  • the microlenses of the array lens are staggered with respect to the light receiving cells 518a arranged in a grid pattern of the image sensor 518 as shown in FIG. 14 (c).
  • the pupil image formed on the image sensor by each microlens of the array lens is made incident on 3 ⁇ 3 light receiving cells.
  • the annular pupil image 517b lacks a portion overlapping the adjacent central pupil image 517a, the central pupil image 517a and the annular pupil image 517b do not overlap on the image sensor 518.
  • FIG. 14 part of FIG. 14 is a figure which shows the color filter arrangement
  • the color filter array arranged in the image sensor 518 includes 4 ⁇ 4 light receiving cells in the basic block B (the block indicated by the thick frame in FIG. 14D). ), And the basic block B is repeatedly arranged in the horizontal direction and the vertical direction.
  • the light receiving cell indicated by a thick frame corresponds to the light receiving cell at the center of 3 ⁇ 3 light receiving cells corresponding to one microlens.
  • the image of the group of light receiving cells in the center of the 3 ⁇ 3 light receiving cells corresponding to each microlens of the array lens arranged in a staggered manner is a mosaic image in which RGB light receiving cells are arranged in a staggered manner.
  • the mosaic image in which the RGB light receiving cells are arranged in a staggered manner has alternating G lines in which G light receiving cells are continuous in the horizontal direction and BR lines in which B light receiving cells and R light receiving cells are alternately arranged. Arranged.
  • four light receiving cells on the top, bottom, left, and right sides sandwiching the light receiving cell in the center of the 3 ⁇ 3 light receiving cells include RGB light receiving cells. That is, the upper and lower light receiving cells sandwiching the central G light receiving cell are G light receiving cells, and the left and right light receiving cells sandwiching the central G light receiving cell are the R light receiving cell and the B light receiving cell.
  • the upper and lower light receiving cells sandwiching the R or B center light receiving cell are the R light receiving cell and the B light receiving cell, and the left and right light receiving cells sandwiching the center R or B light receiving cell are the G light receiving cells.
  • the four light receiving cells corresponding to the annular pupil image 517b (portion (b) in FIG. 14) in which partial openings are formed only in the four directions, up, down, left, and right, are uniformly assigned RGB light receiving cells.
  • pixel signals in all wavelength ranges are obtained from the four light receiving cells on the top, bottom, left, and right across the center light receiving cell among the five light receiving cells corresponding to one microlens. be able to.
  • FIGS. 15 and 16 a fourth embodiment of the imaging apparatus according to the present invention will be described with reference to FIGS. 15 and 16.
  • the imaging apparatus of the fourth embodiment is different from the first, second, and third embodiments mainly in a variety of lenses, an array lens, and an image sensor, and the differences will be described below.
  • the various lenses instead of the various lenses 12 for obtaining the central pupil image 17a and the annular pupil image 17b shown in the portion (a) of FIG. 15, the central pupil image 617a and the annular pupil shown in the portion (b) of FIG. An image from which images 617b and 617c are obtained is used.
  • the annular optical system among the various lenses including the central optical system and the annular optical system divided concentrically is only six directions whose directions are different by 60 degrees around the central optical system corresponding to the central pupil image 617a.
  • two sets of optical systems (a third optical system and a fourth optical system) disposed in the openings corresponding to the 120 ° three-way annular pupil image 617b and the annular pupil image 617c. ).
  • the central optical system corresponding to the central pupil image 617a is a wide-angle optical system
  • the third optical system and the fourth optical system corresponding to the annular pupil image 617b and the annular pupil image 617c are respectively These are two types of telephoto optical systems having different focal lengths.
  • the light receiving cells 618a are arranged in a hexagonal lattice pattern as shown in a part (c) of FIG.
  • the microlenses of the array lens are arranged in a staggered manner with respect to the respective light receiving cells 618a arranged in a hexagonal lattice pattern of the image sensor 618 as shown in part (c) of FIG. It is assumed that every other one is arranged, and every other two are arranged in the vertical direction.
  • the central pupil image 617a formed on the image sensor 618 by each microlens of the array lens is incident on one light receiving cell corresponding to the center position of each microlens
  • the annular pupil image) and the annular pupil image 617c (second annular pupil image) are each composed of six light receiving cells (one located in three directions of 120 degrees) around one light receiving cell corresponding to the center position of each microlens. X2 light receiving cell).
  • the annular pupil image 617b and the annular pupil image 617c overlap with the adjacent annular pupil image 617b and annular pupil image 617c on the image sensor 618, respectively.
  • the image 617c does not overlap.
  • the third optical system and the fourth optical system corresponding to the annular pupil image 617b and the annular pupil image 617c are two types of telephoto optical systems having different focal lengths.
  • two telephoto optical systems having different shooting distances (focus positions) may be used.
  • 16 (a) to 16 (f) are diagrams each showing a color filter array disposed in the image sensor 618.
  • FIG. 16 (a) to 16 (f) are diagrams each showing a color filter array disposed in the image sensor 618.
  • the color filter array shown in part (a) of FIG. 16 is configured by using the nine light receiving cells shown in the upper left as basic blocks, and the basic blocks are repeatedly arranged in the horizontal and vertical directions.
  • the light receiving cell indicated by a thick frame is the central light receiving cell among the seven light receiving cells (one central light receiving cell and six surrounding light receiving cells) corresponding to one microlens. Correspond.
  • the image of the group of light receiving cells in the center of the seven light receiving cells corresponding to each microlens of the array lens arranged in a staggered manner is a mosaic image in which RGB light receiving cells are arranged in a staggered manner, and RGB images are arranged on each horizontal line. Light receiving cells.
  • three light receiving cells located in three directions at 120 degrees corresponding to the 120 degree three-way annular pupil image 617b each have one RGB light receiving cell.
  • three light receiving cells located at 120 degrees in three directions corresponding to the 120 degree three-way annular pupil image 617c are also assigned one RGB light receiving cell.
  • pixel signals in all wavelength ranges can be obtained from the surrounding six light receiving cells among the seven light receiving cells corresponding to one microlens.
  • color filter array shown in the (b) to (f) portions of FIG. 16 is also similar to the color filter array shown in the (a) portion of FIG. Two sets of pixel signals in all wavelength regions can be obtained from the six light receiving cells around the cell.
  • FIG. 17 is a cross-sectional view showing a second embodiment of a variety of lenses applicable to the imaging device 10 (imaging module 11).
  • the various lenses 112 are each composed of a central optical system 113 at the center and an annular optical system 114 at the periphery thereof arranged on the same optical axis.
  • the central optical system 113 is a telephoto optical system including a first lens 113a, a second lens 113b, and a common lens 115, and has an angle of view ⁇ .
  • the annular optical system 114 is a wide-angle optical system including a lens 114 a and a common lens 115, has an angle of view ⁇ ( ⁇ > ⁇ ), and is wider than the central optical system 113.
  • the various lenses 112 do not use a reflecting mirror
  • the central optical system 113 is a telephoto optical system
  • the annular optical system 114 is a wide-angle optical system. It is different in point.
  • imaging device 10 includes, for example, a mobile phone or smartphone having a camera function, a PDA (Personal Digital Assistant), and a portable game machine.
  • a smartphone will be described as an example, and will be described in detail with reference to the drawings.
  • FIG. 18 shows an appearance of a smartphone 500 that is another embodiment of the imaging apparatus 10.
  • a smartphone 500 illustrated in FIG. 18 includes a flat housing 502, and a display input in which a display panel 521 as a display unit and an operation panel 522 as an input unit are integrated on one surface of the housing 502. Part 520.
  • the housing 502 includes a speaker 531, a microphone 532, an operation unit 540, and a camera unit 541.
  • the configuration of the housing 502 is not limited to this, and for example, a configuration in which the display unit and the input unit are independent, or a configuration having a folding structure and a slide mechanism can be employed.
  • FIG. 19 is a block diagram showing a configuration of the smartphone 500 shown in FIG.
  • the main components of the smartphone 500 include a wireless communication unit 510, a display input unit 520, a call unit 530, an operation unit 540, a camera unit 541, a storage unit 550, and an external input / output.
  • Unit 560 GPS (Global Positioning System) reception unit 570, motion sensor unit 580, power supply unit 590, and main control unit 501.
  • a wireless communication function for performing mobile wireless communication via the base station device BS and the mobile communication network NW is provided as a main function of the smartphone 500.
  • the wireless communication unit 510 performs wireless communication with the base station apparatus BS accommodated in the mobile communication network NW according to an instruction from the main control unit 501. Using this wireless communication, transmission and reception of various file data such as audio data and image data, e-mail data, and reception of Web data and streaming data are performed.
  • the display input unit 520 displays images (still images and moving images), character information, and the like visually by the control of the main control unit 501, and visually transmits information to the user, and detects user operations on the displayed information.
  • This is a so-called touch panel, and includes a display panel 521 and an operation panel 522.
  • the display panel 521 is preferably a 3D display panel.
  • the display panel 521 uses an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like as a display device.
  • LCD Liquid Crystal Display
  • OELD Organic Electro-Luminescence Display
  • the operation panel 522 is a device that is placed so that an image displayed on the display surface of the display panel 521 is visible and detects one or a plurality of coordinates operated by a user's finger or stylus.
  • a detection signal generated due to the operation is output to the main control unit 501.
  • the main control unit 501 detects an operation position (coordinates) on the display panel 521 based on the received detection signal.
  • the display panel 521 and the operation panel 522 of the smartphone 500 integrally form the display input unit 520, but the operation panel 522 is disposed so as to completely cover the display panel 521. ing.
  • the operation panel 522 may have a function of detecting a user operation even in an area outside the display panel 521.
  • the operation panel 522 includes a detection area (hereinafter referred to as a display area) for an overlapping portion that overlaps the display panel 521 and a detection area (hereinafter, a non-display area) for an outer edge portion that does not overlap the other display panel 521. May be included).
  • the size of the display area and the size of the display panel 521 may be completely matched, but it is not always necessary to match the two.
  • the operation panel 522 may include two sensitive regions of the outer edge portion and the other inner portion. Further, the width of the outer edge portion is appropriately designed according to the size of the housing 502 and the like.
  • examples of the position detection method employed in the operation panel 522 include a matrix switch method, a resistance film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, and a capacitance method. You can also
  • the call unit 530 includes a speaker 531 and a microphone 532, and converts a user's voice input through the microphone 532 into voice data that can be processed by the main control unit 501, and outputs the voice data to the main control unit 501, or a wireless communication unit 510 or the audio data received by the external input / output unit 560 is decoded and output from the speaker 531.
  • the speaker 531 and the microphone 532 can be mounted on the same surface as the surface on which the display input unit 520 is provided.
  • the operation unit 540 is a hardware key using a key switch or the like, and receives an instruction from the user.
  • the operation unit 540 is mounted on a lower portion and a lower side of the display unit of the housing 502 of the smartphone 500 and is turned on when pressed with a finger or the like, and is turned off when a finger is released with a restoring force such as a spring. It is a button type switch.
  • the storage unit 550 includes control programs and control data of the main control unit 501, address data in which names and telephone numbers of communication partners are associated, transmitted and received e-mail data, Web data downloaded by Web browsing, and downloaded contents Data is stored, and streaming data and the like are temporarily stored.
  • the storage unit 550 includes an internal storage unit 551 with a built-in smartphone and an external storage unit 552 having a removable external memory slot.
  • Each of the internal storage unit 551 and the external storage unit 552 constituting the storage unit 550 includes a flash memory type (flash memory type), a hard disk type (hard disk type), a multimedia card micro type (multimedia card micro type), It is realized using a storage medium such as a card type memory (for example, Micro SD (registered trademark) memory), RAM (Random Access Memory), ROM (Read Only Memory), or the like.
  • flash memory type flash memory type
  • hard disk type hard disk type
  • multimedia card micro type multimedia card micro type
  • a storage medium such as a card type memory (for example, Micro SD (registered trademark) memory), RAM (Random Access Memory), ROM (Read Only Memory), or the like.
  • the external input / output unit 560 serves as an interface with all external devices connected to the smartphone 500, and communicates with other external devices (for example, universal serial bus (USB), IEEE1394, etc.) or a network.
  • external devices for example, universal serial bus (USB), IEEE1394, etc.
  • a network for example, the Internet, wireless LAN, Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee ( ZigBee) (registered trademark, etc.) for direct or indirect connection.
  • Examples of the external device connected to the smartphone 500 include a memory card connected via a wired / wireless headset, wired / wireless external charger, wired / wireless data port, card socket, and SIM (Subscriber).
  • Identity Module Card / UIM User Identity Module Card
  • external audio / video equipment connected via audio / video I / O (Input / Output) terminal
  • external audio / video equipment connected wirelessly, yes / no
  • the external input / output unit may transmit data received from such an external device to each component inside the smartphone 500, or may allow data inside the smartphone 500 to be transmitted to the external device. it can.
  • the GPS receiving unit 570 receives GPS signals transmitted from the GPS satellites ST1 to STn in accordance with instructions from the main control unit 501, performs positioning calculation processing based on the received GPS signals, and calculates the latitude of the smartphone 500, A position consisting of longitude and altitude is detected.
  • the GPS receiving unit 570 can acquire position information from the wireless communication unit 510 or the external input / output unit 560 (for example, a wireless LAN), the GPS receiving unit 570 can also detect the position using the position information.
  • the motion sensor unit 580 includes, for example, a three-axis acceleration sensor, and detects the physical movement of the smartphone 500 in accordance with an instruction from the main control unit 501. By detecting the physical movement of the smartphone 500, the moving direction and acceleration of the smartphone 500 are detected. This detection result is output to the main control unit 501.
  • the power supply unit 590 supplies power stored in a battery (not shown) to each unit of the smartphone 500 in accordance with an instruction from the main control unit 501.
  • the main control unit 501 includes a microprocessor, operates according to a control program and control data stored in the storage unit 550, and controls each unit of the smartphone 500 in an integrated manner. Further, the main control unit 501 includes a mobile communication control function for controlling each unit of the communication system and an application processing function in order to perform voice communication and data communication through the wireless communication unit 510.
  • the application processing function is realized by the main control unit 501 operating in accordance with application software stored in the storage unit 550.
  • Application processing functions include, for example, an infrared communication function that controls the external input / output unit 560 to perform data communication with the opposite device, an e-mail function that transmits and receives e-mails, and a web browsing function that browses web pages. .
  • the main control unit 501 has an image processing function such as displaying video on the display input unit 520 based on image data (still image data or moving image data) such as received data or downloaded streaming data.
  • the image processing function is a function in which the main control unit 501 decodes the image data, performs image processing on the decoding result, and displays an image on the display input unit 520.
  • the main control unit 501 executes display control for the display panel 521 and operation detection control for detecting a user operation through the operation unit 540 and the operation panel 522.
  • the main control unit 501 displays an icon for starting application software, a software key such as a scroll bar, or a window for creating an e-mail.
  • a software key such as a scroll bar, or a window for creating an e-mail.
  • the scroll bar refers to a software key for accepting an instruction to move the display portion of a large image that does not fit in the display area of the display panel 521.
  • the main control unit 501 detects a user operation through the operation unit 540, or accepts an operation on the icon or an input of a character string in the input field of the window through the operation panel 522. Or a display image scroll request through a scroll bar.
  • the main control unit 501 causes the operation position with respect to the operation panel 522 to overlap with the display panel 521 (display area) or other outer edge part (non-display area) that does not overlap with the display panel 521.
  • a touch panel control function for controlling the sensitive area of the operation panel 522 and the display position of the software key.
  • the main control unit 501 can also detect a gesture operation on the operation panel 522 and execute a preset function according to the detected gesture operation.
  • Gesture operation is not a conventional simple touch operation, but an operation that draws a trajectory with a finger or the like, designates a plurality of positions at the same time, or combines these to draw a trajectory for at least one of a plurality of positions. means.
  • the camera unit 541 is a digital camera that performs electronic photography using an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device).
  • the imaging device 10 or the imaging module 11 described above can be applied to the camera unit 541.
  • a wide-angle image and a telephoto image can be taken without requiring a mechanical switching mechanism or the like, which is suitable as a camera unit incorporated in a thin portable terminal like the smartphone 500.
  • the camera unit 541 converts image data obtained by shooting into compressed image data such as JPEG (JointoPhotographic coding Experts Group) under the control of the main control unit 501, and records the data in the storage unit 550.
  • the data can be output through the external input / output unit 560 and the wireless communication unit 510.
  • the camera unit 541 is mounted on the same surface as the display input unit 520, but the mounting position of the camera unit 541 is not limited to this and may be mounted on the back surface of the display input unit 520. Alternatively, a plurality of camera units 541 may be mounted. Note that when a plurality of camera units 541 are mounted, the camera unit 541 used for shooting can be switched to shoot alone, or a plurality of camera units 541 can be used simultaneously for shooting.
  • the camera unit 541 can be used for various functions of the smartphone 500.
  • an image acquired by the camera unit 541 can be displayed on the display panel 521, or the image of the camera unit 541 can be used as one of operation inputs of the operation panel 522.
  • the GPS receiving unit 570 detects the position, the position can also be detected with reference to an image from the camera unit 541.
  • the optical axis direction of the camera unit 541 of the smartphone 500 is determined without using the triaxial acceleration sensor or in combination with the triaxial acceleration sensor. It is also possible to determine the current usage environment.
  • the image from the camera unit 541 can be used in the application software.
  • the light receiving cells of three colors of RGB are assigned to each microlens as pixels for each wavelength region for generating at least one image among a plurality of images that are simultaneously imaged.
  • color image sensors that obtain one output from a plurality of wavelengths include two color image sensors in addition to three color (RGB, etc.) color image sensors, so two light receiving cells are provided for each microlens. You may make it allocate.
  • Different light receiving cells are assigned to detect two types of wavelengths (two-color sensing), and one light receiving cell is assigned to the second group to detect one type of wavelength.
  • the effective data sampling number of the first group two-color sensing
  • the sampling number of the second group can also be made equal to the number of microlenses. it can.
  • a light receiving cell that outputs a pixel signal for each wavelength region for generating one image instead of any light receiving cell of RGB light receiving cell or in addition to RGB light receiving cell, transparent (white)
  • a light receiving cell having a color filter corresponding to another color such as an emerald color may be allocated.
  • a light receiving cell having a filter that cuts visible light and transmits only infrared light may be allocated, and according to this, an infrared image can be acquired.
  • one of the central optical system and the annular optical system is a wide-angle optical system, and the other is a telephoto optical system.
  • the present invention is not limited to this.
  • Various optical systems such as an optical system and two types of optical systems having different spatial frequency characteristics (blur) can be considered.
  • an optical system in which a plurality of optical systems having different characteristics are concentrically divided is used. Good.
  • the reflecting mirror in the reflecting mirror type lens configuration of the various lenses 12 shown in FIG. 3 is not limited to a concave mirror or a convex mirror, and may be a plane mirror. Also, the number of reflecting mirrors is not limited to two, but three You may make it provide above.
  • a common lens for the central optical system and the annular optical system, or a moving mechanism for moving the image sensor in the optical axis direction may be provided, and thereby focus adjustment may be performed.
  • the array lens 16 is used as the pupil dividing means.
  • the present invention is not limited to this, and a pinhole is provided at the position of each microlens 16a of the array lens 16, and pupil images of various lenses are obtained by each pinhole.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Optics & Photonics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Studio Devices (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Color Television Image Signal Generators (AREA)
  • Solid State Image Pick-Up Elements (AREA)
PCT/JP2014/082391 2013-12-20 2014-12-08 撮像モジュール及び撮像装置 Ceased WO2015093332A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201480067516.6A CN105814882B (zh) 2013-12-20 2014-12-08 摄像模块及摄像装置
EP14871445.4A EP3086546B1 (en) 2013-12-20 2014-12-08 Image capturing module and image capturing device
US15/163,755 US10027918B2 (en) 2013-12-20 2016-05-25 Imaging module that captures subject images having different characteristics and imaging device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013-263660 2013-12-20
JP2013263660A JP6151632B2 (ja) 2013-12-20 2013-12-20 撮像モジュール及び撮像装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/163,755 Continuation US10027918B2 (en) 2013-12-20 2016-05-25 Imaging module that captures subject images having different characteristics and imaging device

Publications (1)

Publication Number Publication Date
WO2015093332A1 true WO2015093332A1 (ja) 2015-06-25

Family

ID=53402681

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/082391 Ceased WO2015093332A1 (ja) 2013-12-20 2014-12-08 撮像モジュール及び撮像装置

Country Status (5)

Country Link
US (1) US10027918B2 (https=)
EP (1) EP3086546B1 (https=)
JP (1) JP6151632B2 (https=)
CN (1) CN105814882B (https=)
WO (1) WO2015093332A1 (https=)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3131292A1 (en) * 2015-08-14 2017-02-15 Thomson Licensing Plenoptic camera comprising a shuffled color filter array
WO2017029827A1 (ja) * 2015-08-19 2017-02-23 富士フイルム株式会社 撮像装置、撮像方法、プログラム、及び非一時的記録媒体
WO2017126242A1 (ja) * 2016-01-18 2017-07-27 富士フイルム株式会社 撮像装置、及び、画像データ生成方法
CN108139559A (zh) * 2015-08-19 2018-06-08 富士胶片株式会社 透镜装置
WO2020050289A1 (ja) * 2018-09-07 2020-03-12 ソニーセミコンダクタソリューションズ株式会社 画像処理装置、画像処理方法および画像処理プログラム

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9876946B2 (en) * 2015-08-03 2018-01-23 Semiconductor Energy Laboratory Co., Ltd. Imaging device and electronic device
CN107924107B (zh) 2015-08-19 2020-09-18 富士胶片株式会社 摄像装置
JP2018197765A (ja) * 2015-10-08 2018-12-13 富士フイルム株式会社 撮像装置
JP2018198344A (ja) * 2015-10-08 2018-12-13 富士フイルム株式会社 レンズ装置、撮像ユニット及び撮像装置
USD861211S1 (en) * 2015-12-08 2019-09-24 Coast Cutlery Co. Headlamp
USD812278S1 (en) * 2016-01-03 2018-03-06 Industrial Revolution, Inc. Headlamp assembly with adjustable head strap
USD843035S1 (en) * 2016-01-03 2019-03-12 Industrial Revolutions, Inc. Headlamp
CN109564928B (zh) * 2016-08-09 2022-12-16 索尼公司 固态摄像元件、固态摄像元件用光瞳校正方法、摄像装置和信息处理装置
EP3686642A1 (en) * 2019-01-24 2020-07-29 Depixus Multichannel close-up imaging device
TWI710793B (zh) 2019-08-23 2020-11-21 大立光電股份有限公司 光學攝影系統及電子裝置
JP7694020B2 (ja) * 2020-11-27 2025-06-18 セイコーエプソン株式会社 画像処理装置
CN112911150A (zh) * 2021-01-29 2021-06-04 广州合易威视信息科技有限公司 一种目标区域高清人脸自动抓拍方法
KR102902680B1 (ko) 2021-04-19 2025-12-19 삼성전자주식회사 이미지 센서 및 이미지 센싱 시스템
KR20230006343A (ko) 2021-07-02 2023-01-10 삼성전자주식회사 전자 장치 및 그 제어 방법
KR102635554B1 (ko) * 2021-11-26 2024-02-13 국방과학연구소 이미지를 생성하기 위한 장치 및 방법
CN115629076B (zh) * 2022-09-27 2025-02-18 威海华菱光电股份有限公司 一种阵列式图像检测装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0583746U (ja) * 1992-04-14 1993-11-12 コニカ株式会社 フィルム一体型カメラの多焦点レンズ
JP2010239337A (ja) * 2009-03-31 2010-10-21 Sony Corp 固体撮像装置、固体撮像装置の信号処理方法および撮像装置
JP2012247645A (ja) * 2011-05-27 2012-12-13 Fujifilm Corp 撮像装置
JP2012253670A (ja) 2011-06-06 2012-12-20 Fujifilm Corp 撮像装置
JP2013090059A (ja) 2011-10-14 2013-05-13 Sony Corp 撮像装置、画像生成システム、サーバおよび電子機器
JP2013115532A (ja) 2011-11-28 2013-06-10 Sony Corp 画像処理装置および方法、記録媒体、並びに、プログラム

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0583746A (ja) * 1991-09-19 1993-04-02 Nippon Telegr & Teleph Corp <Ntt> ３次元表示装置
US7764440B2 (en) * 2007-11-27 2010-07-27 Eastman Kodak Company Dual focal length lens system
JP5302663B2 (ja) * 2008-12-24 2013-10-02 キヤノン株式会社 焦点検出装置及び方法、及び撮像装置
JP5246424B2 (ja) * 2009-05-11 2013-07-24 ソニー株式会社 撮像装置
CN103155543B (zh) * 2010-10-01 2014-04-09 富士胶片株式会社 成像设备
GB2488519A (en) * 2011-02-16 2012-09-05 St Microelectronics Res & Dev Multi-channel image sensor incorporating lenslet array and overlapping fields of view.
EP2536153B1 (en) * 2011-04-22 2016-04-13 Panasonic Intellectual Property Management Co., Ltd. Image capture device, image capture system, and image capture method
EP2750374B1 (en) * 2011-08-24 2018-03-28 FUJIFILM Corporation Imaging device
CN103237162A (zh) * 2013-04-15 2013-08-07 华中科技大学 光学数据采集装置及产生输出与场景对应图像数据的方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0583746U (ja) * 1992-04-14 1993-11-12 コニカ株式会社 フィルム一体型カメラの多焦点レンズ
JP2010239337A (ja) * 2009-03-31 2010-10-21 Sony Corp 固体撮像装置、固体撮像装置の信号処理方法および撮像装置
JP2012247645A (ja) * 2011-05-27 2012-12-13 Fujifilm Corp 撮像装置
JP2012253670A (ja) 2011-06-06 2012-12-20 Fujifilm Corp 撮像装置
JP2013090059A (ja) 2011-10-14 2013-05-13 Sony Corp 撮像装置、画像生成システム、サーバおよび電子機器
JP2013115532A (ja) 2011-11-28 2013-06-10 Sony Corp 画像処理装置および方法、記録媒体、並びに、プログラム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3086546A4

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9807368B2 (en) 2015-08-14 2017-10-31 Thomson Licensing Plenoptic camera comprising a shuffled color filter array
EP3131292A1 (en) * 2015-08-14 2017-02-15 Thomson Licensing Plenoptic camera comprising a shuffled color filter array
EP3339931A4 (en) * 2015-08-19 2018-08-08 Fujifilm Corporation Lens device
JPWO2017029827A1 (ja) * 2015-08-19 2018-01-18 富士フイルム株式会社 撮像装置、撮像方法、プログラム、及び非一時的記録媒体
CN108139559A (zh) * 2015-08-19 2018-06-08 富士胶片株式会社 透镜装置
WO2017029827A1 (ja) * 2015-08-19 2017-02-23 富士フイルム株式会社 撮像装置、撮像方法、プログラム、及び非一時的記録媒体
US10393993B2 (en) 2015-08-19 2019-08-27 Fujifilm Corporation Lens device
US10552963B2 (en) 2015-08-19 2020-02-04 Fujifilm Corporation Imaging device, imaging method, and non-transitory recording medium
WO2017126242A1 (ja) * 2016-01-18 2017-07-27 富士フイルム株式会社 撮像装置、及び、画像データ生成方法
CN108432233A (zh) * 2016-01-18 2018-08-21 富士胶片株式会社 摄像装置及图像数据生成方法
JPWO2017126242A1 (ja) * 2016-01-18 2018-10-25 富士フイルム株式会社 撮像装置、及び、画像データ生成方法
US10691002B2 (en) 2016-01-18 2020-06-23 Fujifilm Corporation Imaging device and image data generation method
CN108432233B (zh) * 2016-01-18 2020-08-18 富士胶片株式会社 摄像装置及图像数据生成方法
WO2020050289A1 (ja) * 2018-09-07 2020-03-12 ソニーセミコンダクタソリューションズ株式会社 画像処理装置、画像処理方法および画像処理プログラム
US11847759B2 (en) 2018-09-07 2023-12-19 Sony Semiconductor Solutions Corporation Image processing device, image processing method, and image processing program

Also Published As

Publication number Publication date
EP3086546A4 (en) 2016-12-14
US20160269667A1 (en) 2016-09-15
CN105814882B (zh) 2018-10-02
US10027918B2 (en) 2018-07-17
JP2015119456A (ja) 2015-06-25
CN105814882A (zh) 2016-07-27
EP3086546A1 (en) 2016-10-26
JP6151632B2 (ja) 2017-06-21
EP3086546B1 (en) 2018-08-01

Similar Documents

Publication Publication Date Title
JP6151632B2 (ja) 撮像モジュール及び撮像装置
JP6077967B2 (ja) 撮像装置
JP5690977B2 (ja) 撮像素子及び撮像装置
US9674461B2 (en) Imaging device and imaging method
JP6165680B2 (ja) 撮像装置
JP5697801B2 (ja) 撮像装置及び自動焦点調節方法
JP5802858B2 (ja) 撮像装置、画像処理装置、画像処理方法及びプログラム
US10003731B2 (en) Image element, and imaging device and imaging method using the same for achieving improved image quality regardless of an incident angle of light
CN104995904B (zh) 摄像装置
JP6427720B2 (ja) 撮像素子及び撮像装置
JP5749409B2 (ja) 撮像装置、画像処理方法及びプログラム
JP5542248B2 (ja) 撮像素子及び撮像装置
US9344646B2 (en) Imaging device and imaging method
JPWO2014006783A1 (ja) 撮像装置及び画像処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14871445

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2014871445

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2014871445

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE