WO2022213316A1 - Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande - Google Patents

Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande Download PDF

Info

Publication number
WO2022213316A1
WO2022213316A1 PCT/CN2021/085906 CN2021085906W WO2022213316A1 WO 2022213316 A1 WO2022213316 A1 WO 2022213316A1 CN 2021085906 W CN2021085906 W CN 2021085906W WO 2022213316 A1 WO2022213316 A1 WO 2022213316A1
Authority
WO
WIPO (PCT)
Prior art keywords
lenses
image sensors
incident
camera system
incident lights
Prior art date
Application number
PCT/CN2021/085906
Other languages
English (en)
Inventor
Takahiro Okubo
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp., Ltd. filed Critical Guangdong Oppo Mobile Telecommunications Corp., Ltd.
Priority to PCT/CN2021/085906 priority Critical patent/WO2022213316A1/fr
Publication of WO2022213316A1 publication Critical patent/WO2022213316A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/13—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with multiple sensors

Definitions

  • Various example embodiments relate to a camera system including multiple lenses and multiple image sensors to perform multiple exposures, which results in improved color reproducibility, sensitivity and resolution at low cost.
  • Cameras including a single lens 101 and a single image sensor 102 of a Bayer array shown in Fig. 1, and cameras including a single lens 111, a dichroic prism 112 and three RGB image sensors 113, 114 and 115, one for each of the three colors, are widely used as RGB digital cameras, as shown in Fig. 1 and Fig. 2.
  • the advantage of a camera including a single image sensor of a Bayer array is that it requires only one image sensor; thus it is relatively low cost, and the optical system circuit is small and easy to mount.
  • a multi-lens camera installed in a smartphone is a single-panel camera with an independent Bayer array or with an independent similar array, even though it is equipped with a plurality of image sensors. Therefore, the camera configuration has low image quality despite the high cost.
  • the present disclosure has been made in consideration of the above situation, and thus the present disclosure provides a camera system including multiple lenses and image sensors to perform multiple exposures and a method for controlling the same, which results in improved color reproducibility, sensitivity and resolution.
  • a camera system comprising:
  • an incident optical path selection circuit provided between the lenses and the image sensors
  • the incident optical path selection circuit selects one of incident lights from one of the plurality of lenses and transmits the selected incident light to the plurality of image sensors so as to perform a plurality of exposures.
  • the incident optical path selection circuit may select one of a plurality of optical paths between the plurality of lenses and the plurality of image sensors.
  • the image sensors may include an image sensor with a red (R) color filter, an image sensor with a green (G) color filter and an image sensor with a blue (B) color filter.
  • R red
  • G green
  • B blue
  • the image sensors may further include an image sensor with a near-infrared transmission filter.
  • the image sensors may further include an image sensor with a white filter.
  • the plurality of lenses may include telescope lenses, wide-angle lenses and selfie lenses.
  • the incident optical path selection circuit may include a plurality of shutters provided for each of the lenses, each of the shutters opening and closing in time division for each of lenses, and one of the shutters opening and the other shutters closing in a certain exposure period,
  • each of the demultiplexers separating an incident light from a corresponding lens
  • each of the multiplexers adds the incident lights from the demultiplexers to transmit the incident lights to each of the image sensors.
  • each of the demultiplexers may separate each of the incident lights, based on an amount of incident lights using a certain weight, and distributes the separated incident lights to each of the image sensors.
  • each of the demultiplexers may separate each of the incident lights, based on wavelengths of the incident lights, and distributes the separated incident lights to each of the image sensors.
  • each of the demultiplexers may separate an incident light into at least three primary colors of blue (B) , green (G) and red (R) .
  • each of the shutters may open and close in each exposure period in accordance with a time division for each lens.
  • each of the shutters may open and close in synchronization with a timing of a vertical blanking period of the image sensors.
  • a method for controlling an apparatus including a plurality of lenses and a plurality of image sensors provided for each of the lenses comprising:
  • the method may comprise, when selecting one of incident lights from one of the plurality of lenses, selecting one of incident lights in a certain exposure period, and separating the selected one of incident lights, and
  • the method may comprise, when separating the selected one of incident lights, separating the selected one of incident light based on an amount of incident lights using a certain weight.
  • the method may comprise, when separating the selected one of incident lights, separating the selected one of incident light based on wavelengths of the incident lights.
  • the method may comprise, when separating the selected one of incident lights, separating the selected one of incident light into at least three primary colors of blue (B) , green (G) and red (R) .
  • the method may comprise, when selecting one of incident lights, selecting one of incident lights in each exposure period in accordance with a time division for each lens.
  • the method may comprise, when selecting one of incident lights, selecting one of incident lights in synchronization with a timing of a vertical blanking period of the image sensors.
  • a device comprising:
  • At least one memory including program codes
  • the at least one memory and the program codes configured to, with the at least one processor, cause the apparatus to perform the above method.
  • Fig. 1 is an explanatory plan view of a basic configuration of a camera including a single image sensor of Bayer array;
  • Fig. 2 is an explanatory plan view of a basic configuration of a camera including three RGB image sensors;
  • Fig. 3 is an explanatory plan view of a basic configuration of a camera according to a comparative example
  • Fig. 4A is a block diagram schematically showing an overall arrangement of a camera system according to a first embodiment of the present disclosure
  • Fig. 4B is a block diagram schematically showing an overall arrangement of a camera system according to a second embodiment of the present disclosure
  • Fig. 5 is an explanatory plan view of a configuration including a plurality of lenses, image sensors and an incident optical path selection circuit of the camera system according to the first or second embodiment of the present disclosure
  • Figs. 6A, 6B and 6C are explanatory plan views illustrating an operating principle of the camera system according to the first or second embodiment of the present disclosure, specifically a time division shutter;
  • Fig. 7 is an explanatory plan view illustrating an operating principle of the camera system according to the first or second embodiment of the present disclosure, specifically a shutter control based on a vertical blanking period;
  • Fig. 8 is an explanatory plan view illustrating an operating principle of the camera system according to the first or second embodiment of the present disclosure, specifically a weighted demultiplexer;
  • Fig. 9 is an explanatory plan view illustrating an operating principle of the camera system according to the first or second embodiment of the present disclosure, specifically a wavelength separator;
  • Fig. 10 is an explanatory plan view illustrating an operating principle of the camera system according to the first or second embodiment of the present disclosure, specifically a multiplexer.
  • the camera system according to the comparative example includes a plurality of lenses 121A, 121B and 121C and a plurality of image sensors 122A, 122B and 122C, which constitute a camera module installed inside a camera for selfies, and other camera modules equipped with a plurality of single focus lenses with different magnifications which are installed outside the camera.
  • a plurality of independent camera modules each consisting of one lens and one image sensor which are connected, are mounted in parallel inside the camera.
  • one camera is selected from a plurality of cameras and activated according to the shooting mode set by the user, such as selfie of an inside camera or zoom magnification of an outside camera.
  • the other cameras are inactive and not functioning. That is, although a plurality of expensive image sensors are mounted, only one image sensor is used at the time of shooting.
  • a plurality of lenses such as an outer telescope lens and an outer wide-angle lens, are directed in the same direction. Although a selfie lens is frequently used, it is directed in a different direction.
  • Image composition artifacts occur due to the difference in subject magnification due to the difference in lens distortion and curvature.
  • a camera system 1A according to a first embodiment of the present disclosure comprises a camera module 10 including an optical system 1, an imaging device 2 and an image sensor driver 3, a processor 20, a memory 30 and a display unit 40. Images captured by the camera system 1A may include still images and/or moving images.
  • the optical system 1 includes a plurality of sets of lenses, preferably a plurality of lens groups having a plurality of optical lenses, shutters, demultiplexers, multiplexers, diaphragm adjustment mechanisms, zoom mechanisms, and auto focusing mechanisms.
  • the imaging device 2 includes three RGB image sensors, one for each of the three colors, such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) image sensor, having a plurality of photoelectric conversion elements arrayed along a horizontal direction and along a vertical direction, and a color filter arranged on the image sensor.
  • the imaging device 2 driven by the image sensor driver 3, receives light from the optical system 1 and outputs color signals.
  • the processor 20 receives the color signals from the imaging device 2 and controls the optical path selection circuit described later in detail.
  • the memory 30 receives and outputs data, from/to the camera module 10, the processor 20 and the display unit 40.
  • the display unit 40 receives data from the processor 20, and displays an image acquired by the camera module 10 and processed by the processor 20.
  • the camera system shown in Fig. 4A comprises one processor 20, the processor 20 performing not only image processes but also system management processes.
  • the camera system shown in Fig. 4B comprises an image signal processor 21 and a main processor 22.
  • the image signal processor 21 performs image processes
  • the main processor 22 performs mostly system management processes and may partially perform image processes.
  • an incident optical path selection circuit is inserted between a plurality of lenses, such as an outer telescope lens 11A, an outer wide-angle lens 11B and an inner selfie lens 11C, and a plurality of image sensors 15A, 15B and 15C, such as an image sensor with a red (R) color filter, an image sensor with a green (G) color filter and an image sensor with a blue (B) color filter as shown in Fig. 5.
  • Lenses are not limited to the above telescope lens, wide-angle lens and selfie lens. It is possible to include, such as a zoom lens (a standard zoom lens, a wide angle zoom lens or a telescope zoom lens) , and/or a macro lens.
  • an image sensor with a near-infrared transmission filter so as to realize multi-spectral shooting, and/or with a white (transparent) filter.
  • the incident optical path selection circuit includes a plurality of shutters 12A, 12B and 12C, a plurality of demultiplexers 13A, 13B and 13C, and a plurality of multiplexers 14A, 14B and 14C.
  • Each of the shutters 12A, 12B and 12C is provided behind the lenses 11A, 11B and 11C, respectively, and has a function of transmitting or blocking an incident light from each of the lenses 11A, 11B and 11C.
  • the shutters 12A, 12B and 12C may be, for example, mechanical shutters, LCD shutters, MEMS shutters, etc.
  • the opening and closing of the shutters 12A, 12B and 12C are performed by time-division and exclusive control.
  • Figs. 6A, 6B and 6C only one of the shutters 12A, 12B and 12C is transparent, and the others are light shielded in each exposure period.
  • an opening and closing timing of the shutters 12A, 12B and 12C are synchronized with an exposure stop period, that is, a vertical blanking period, of the image sensors 15A, 15B and 15C.
  • Each of the demultiplexers 13A, 13B and 13C is provided behind the shutters 12A, 12B and 12C, respectively, and has a function of separating the incident light from the lenses 11A, 11B and 11C.
  • the demultiplexers 13A, 13B and 13C are composed by combining, for example, transparent mirrors and reflective mirrors.
  • a demultiplexer weighing an intensity of its output light using weight parameters, such as x, y and z, and separating the light into output lights of In * [x/ (x+y+z) ] , In * [y/ (x+y+z) ] and In * [z/ (x+y+z) ] , as shown in Fig. 8.
  • Another type of a demultiplexer controls its refractive index and separates each wavelength of light into output lights of blue (B) having a wavelength A to B, green (G) having a wavelength B to C and red (R) having a wavelength C to D, as shown in Fig. 9.
  • HDR high dynamic range
  • the separated lights are not limited to lights of blue (B) , green (G) and red (R) .
  • B blue
  • R red
  • Each of the multiplexers 14A, 14B and 14C is provided between the demultiplexers 13A, 13B and 13C and the image sensors 15A, 15B and 15C, respectively, and has a function of adding incident lights of the above plurality of incident optical paths.
  • the multiplexers 14A, 14B and 14C may be composed of, for example, reflective mirrors.
  • the outputs of the multiplexers 14A, 14B and 14C are directly connected to the image sensors 15A, 15B and 15C.
  • the multiplexers 14A, 14B and 14C function as optical selectors by being combined with the opening and closing functions of the shutters 12A, 12B and 12C, as shown in Fig. 10.
  • the incident light passing through the lens 13A enters into the demultiplexer 13A and it is distributed to the multiplexers 14A, 14B and 14C.
  • Each of the multiplexers 14A, 14B and 14C receives and transmits each incident light to the image sensors 15A, 15B and 15C.
  • the incident light passing through the lens 13B enters the demultiplexer 13B and is distributed to the multiplexers 14A, 14B and 14C.
  • the multiplexers 14A, 14B and 14C receive and transmit the incident lights to the image sensors 15A, 15B and 15C.
  • the incident light passing through the lens 13C enters the demultiplexer 13C and is distributed to the multiplexers 14A, 14B and 14C.
  • the multiplexers 14A, 14B and 14C receive and transmit the incident lights to the image sensors 15A, 15B and 15C.
  • the camera systems according to the first and second embodiments of the present disclosure make it possible to allow the incident light from each lens to reach a plurality of image sensors by the incident optical path selection circuit provided between the lenses 11A, 11B and 11C, and the image sensors 15A, 15B and 15C.
  • Color mixing is a phenomenon in which colors are mixed, as a result of light separated by a color filter in a Bayer array being incident on an adjacent photodiode.
  • color mixing occurs because the material applied to the color filter cannot completely separate the wavelengths of red (R) , green (G) , and blue (B) .
  • RGB matrix calculation correction is performed so that it approaches the ideal RGB, it cannot be completely corrected, and adverse effects such as lightening of colors and increased noise occur.
  • the camera system of the first or second embodiment of the present disclosure operates as a camera system including three RGB image sensors by using optical components to separate incident lights based on an amount of incident lights, or to separate wavelengths, and by independently exposing the three RGB image sensors with the respective RGB color, so that color mixing is prevented.
  • the color signals of red (R) , green (G) and blue (B) are restored by interpolating pixels of different colors in the Bayer array by signal processing.
  • the recoverable resolution is theoretically inferior to that of the camera system including three RGB image sensors.
  • the camera system including three RGB image sensors does not require color grid interpolation, signal processing can be simplified.
  • the amounts of collected light of each color, red (R) , green (G) and blue (B) differ from each other depending on the difference in the number of pixels of photodiodes for photoelectric conversion.
  • a sensitivity of green (G) is two times that of red (R) and that of blue (B) .
  • first and second are used herein for purposes of description and are not intended to indicate or imply relative importance or significance or to imply the number of indicated technical features.
  • the feature defined with “first” and “second” may comprise one or more of this feature.
  • a plurality of means two or greater than two, unless specified otherwise.
  • the terms “mounted” , “connected” , “coupled” and the like are used broadly, and may be, for example, fixed connections, detachable connections, or integral connections; may also be mechanical or electrical connections; may also be direct connections or indirect connections via intervening structures; may also be inner communications of two elements, which can be understood by those skilled in the art according to specific situations.
  • a structure in which a first feature is "on" or “below” a second feature may include an embodiment in which the first feature is in direct contact with the second feature, and may also include an embodiment in which the first feature and the second feature are not in direct contact with each other, but are contacted via an additional feature formed therebetween.
  • a first feature "on” , “above” or “on top of” a second feature may include an embodiment in which the first feature is right or obliquely “on” , “above” or “on top of” the second feature, or just means that the first feature is at a height higher than that of the second feature; while a first feature “below” , “under” or “on bottom of” a second feature may include an embodiment in which the first feature is right or obliquely “below” , "under” or “on bottom of” the second feature, or just means that the first feature is at a height lower than that of the second feature.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

L'invention concerne un système de caméra pour obtenir une reproductibilité de couleur, une sensibilité et une résolution améliorées à faible coût. Le système de caméra comprend une pluralité de lentilles, une pluralité de capteurs d'image prévus pour chacune des lentilles, et un circuit de sélection de trajets optiques incidents disposé entre les lentilles et les capteurs d'image, le circuit de sélection de trajets optiques incidents sélectionnant l'une des lumières incidentes provenant d'une lentille de la pluralité de lentilles et transmettant la lumière incidente sélectionnée à la pluralité de capteurs d'image de façon à effectuer une pluralité d'expositions.
PCT/CN2021/085906 2021-04-08 2021-04-08 Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande WO2022213316A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/085906 WO2022213316A1 (fr) 2021-04-08 2021-04-08 Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/085906 WO2022213316A1 (fr) 2021-04-08 2021-04-08 Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande

Publications (1)

Publication Number Publication Date
WO2022213316A1 true WO2022213316A1 (fr) 2022-10-13

Family

ID=83545934

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/085906 WO2022213316A1 (fr) 2021-04-08 2021-04-08 Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande

Country Status (1)

Country Link
WO (1) WO2022213316A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6975358B1 (en) * 2000-08-31 2005-12-13 Intel Corporation Dual channel imaging device
US20130070140A1 (en) * 2011-09-21 2013-03-21 Robert Gove Image sensors with multiple lenses of varying polarizations
EP2592837A1 (fr) * 2011-11-10 2013-05-15 Research In Motion Limited Appareil et procédé associé pour la formation d'une image couleur de caméra
US20150285743A1 (en) * 2014-04-07 2015-10-08 Seagate Technology Llc Features maps of articles with polarized light
US20170126986A1 (en) * 2015-10-30 2017-05-04 Raytheon Company Dual-field-of-view optical system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6975358B1 (en) * 2000-08-31 2005-12-13 Intel Corporation Dual channel imaging device
US20130070140A1 (en) * 2011-09-21 2013-03-21 Robert Gove Image sensors with multiple lenses of varying polarizations
EP2592837A1 (fr) * 2011-11-10 2013-05-15 Research In Motion Limited Appareil et procédé associé pour la formation d'une image couleur de caméra
US20150285743A1 (en) * 2014-04-07 2015-10-08 Seagate Technology Llc Features maps of articles with polarized light
US20170126986A1 (en) * 2015-10-30 2017-05-04 Raytheon Company Dual-field-of-view optical system

Similar Documents

Publication Publication Date Title
CN102292974B (zh) 摄像设备
CN102487432B (zh) 摄像设备
EP2835965B1 (fr) Dispositif d'imagerie et capteur d'image
CN102197328B (zh) 图像拾取元件和图像拾取装置
US8558915B2 (en) Photographing apparatus and method
US8890994B2 (en) Image capturing apparatus
US8810678B2 (en) Imaging apparatus and method of driving solid-state imaging device
US9332199B2 (en) Imaging device, image processing device, and image processing method
JP2003189183A (ja) 撮像装置
WO2017199557A1 (fr) Dispositif d'imagerie, procédé d'imagerie, programme et support d'enregistrement non transitoire
CN103843318A (zh) 摄像设备及其控制方法
CN114650377A (zh) 摄像模组、摄像模组的控制方法以及电子设备
CN113132597B (zh) 图像采集系统及终端
JP2015087401A (ja) 撮像装置
EP3649772B1 (fr) Appareil d'imagerie avec un deuxième élément d'imagerie utilisé pour la correction du vignettage dans des images capturées par un premier élément d'imagerie
CN103843320B (zh) 图像传感器和成像装置
WO2019155757A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image, et système de traitement d'image
WO2022213316A1 (fr) Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, et son procédé de commande
CN114666469B (zh) 图像处理装置、方法及具有该图像处理装置的镜头模组
US8848077B2 (en) Imaging apparatus and imaging method
JP4530149B2 (ja) ハイダイナミックレンジカメラシステム
EP4246959A1 (fr) Capteur d'image et appareil d'imagerie
WO2023123110A1 (fr) Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, son procédé de commande et dispositif électronique
WO2023123095A1 (fr) Système de caméra comprenant de multiples lentilles et de multiples capteurs d'image, son procédé de commande et dispositif électronique
JPH07123418A (ja) 撮像装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21935546

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21935546

Country of ref document: EP

Kind code of ref document: A1