CN109932852B - Method for maximizing utilization rate of light field camera sensor - Google Patents

Method for maximizing utilization rate of light field camera sensor Download PDF

Info

Publication number
CN109932852B
CN109932852B CN201910350700.6A CN201910350700A CN109932852B CN 109932852 B CN109932852 B CN 109932852B CN 201910350700 A CN201910350700 A CN 201910350700A CN 109932852 B CN109932852 B CN 109932852B
Authority
CN
China
Prior art keywords
area
main
sensor
light field
field camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910350700.6A
Other languages
Chinese (zh)
Other versions
CN109932852A (en
Inventor
金欣
李钏溥
戴琼海
陈磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Graduate School Tsinghua University
Original Assignee
Shenzhen Graduate School Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Graduate School Tsinghua University filed Critical Shenzhen Graduate School Tsinghua University
Priority to CN201910350700.6A priority Critical patent/CN109932852B/en
Publication of CN109932852A publication Critical patent/CN109932852A/en
Application granted granted Critical
Publication of CN109932852B publication Critical patent/CN109932852B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

Hair brushA method for maximizing the utilization rate of light field camera sensor is disclosed, which is based on the aperture value F of the main lens of the sensormainWith the diameter D of the macropixel unit, and by determining the sensor utilization SU and the total area S of each macropixel unittolArea S of the overlap region in each celloveAnd the area S of the ineffective regionwasThe relationship between the sensor utilization rate SU and the main lens aperture value F is obtainedmainThe relationship between the two is determined by an optimization solution method to maximize the utilization rate of the sensor SUmainDetermining a macro-pixel mask according to the corresponding parameters, and finally extracting the aperture value F of the light field camera in the main lens by using the maskmainAnd effective pixels in the macro pixels determined below. The method can effectively improve the utilization rate of the light field camera sensor, improve the angular resolution of the light field camera, and enable the light field camera to decode to obtain more sub-viewing angles.

Description

Method for maximizing utilization rate of light field camera sensor
Technical Field
The invention relates to the field of light field cameras, in particular to a method for maximizing the utilization rate of a light field camera sensor.
Background
In recent years, light field images captured by a light field camera have attracted much attention in the fields of computer vision, such as stereoscopic display, three-dimensional reconstruction, virtual reality, and the like. The improvement and enhancement of the light field camera have also become important research, and according to the imaging mechanism of the light field camera, after the main lens of the light field camera is imaged, the direction information and the angle information of each ray are recorded by using a Micro Lens Array (MLA), and a series of circular macro pixels are formed on the sensor. However, circular macro-pixels cannot be densely laid on a plane, so that a plurality of areas on a sensor of the light field camera cannot be used because effective light field data is not recorded, and the angular resolution of the light field camera and the number of sub-views that can be decoded are affected.
Most of existing solutions are based on matching the aperture value of the main lens of the light field camera with the aperture value of the micro lens array by adjusting the aperture value of the main lens of the light field camera, so that circular macro pixels are tangent to each other, but even then, many areas which do not receive light field information exist on the sensor, and the utilization rate of the sensor does not reach the theoretical maximum value.
Disclosure of Invention
The invention mainly aims to overcome the defects of the prior art, provides a method for maximizing the utilization rate of a light field camera sensor, improves the utilization rate of the light field camera sensor and improves the angular resolution.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method of maximizing light field camera sensor utilization, comprising the steps of:
a1: acquiring the distance b between a microlens array and a sensor of the light field camera and the diameter of the microlens, taking the diameter of the microlens as the diameter D of the macropixel, and determining the aperture value F of a main lensmainThe relation to the macropixel diameter D;
a2: obtaining the side length A of the macro-pixel unit and determining the total area S of each macro-pixel unittolAnd the area S of the overlapping area in each macro-pixel unitoveAnd the area S of the ineffective areawas；
A3: determining sensor utilization SU and total area StolArea S of the overlapping regionoveAnd the area S of the ineffective regionwasUsing the results obtained in step A1 and step A2 to obtain the sensor utilization rate SU and the main lens aperture value FmainDetermining a main lens aperture value capable of maximizing the utilization rate of the sensor by an optimization solution method;
a4: for the special-shaped macro-pixel determined by the main lens aperture value obtained in the step A3, the side length A, the distance b and the main lens aperture value F are corresponding to the parametersmainAnd determining a corresponding processing mask, and processing the special-shaped macro-pixel by using the mask to obtain an effective part of the special-shaped macro-pixel.
Further:
in step A1, a main lens aperture F is determined according to the distance b between the microlens array and the sensor and the diameter D of the microlensmainThe relationship with the macropixel diameter D is as follows:
Figure BDA0002043840490000021
in step A2, for a square microlens array, the total area S of each macro-pixel unittolComprises the following steps:
Stol＝A2
area S of the overlapping region in each celloveComprises the following steps:
Figure BDA0002043840490000022
area S of the ineffective regionwasComprises the following steps:
Figure BDA0002043840490000023
wherein
Figure BDA0002043840490000024
In step A2, for a hexagonal arrangement of microlens arrays, the total area S of each macro-pixel celltolComprises the following steps:
Figure BDA0002043840490000025
area S of the overlapping region in each celloveComprises the following steps:
Figure BDA0002043840490000026
area S of the ineffective regionwasComprises the following steps:
Figure BDA0002043840490000027
wherein
Figure BDA0002043840490000031
In step A3, sensor utilization SU is compared to total area S of each macropixel unittolArea S of the overlapping regionoveAnd the area S of the ineffective regionwasThe relationship of (c) is determined as follows:
Figure BDA0002043840490000032
in step A3, for a light field camera with a square microlens array, the results obtained in steps a1 and a2 are substituted into the relational expression
Figure BDA0002043840490000033
Obtaining a functional relation:
Figure BDA0002043840490000034
wherein, FmainHas a value range of
Figure BDA0002043840490000035
And obtaining the corresponding main lens aperture value when the utilization rate of the sensor is maximum by a method of solving the maximum value through the concave function.
In step A3, for the light field cameras with the microlens array in the hexagonal arrangement, the results obtained in steps a1 and a2 are substituted into the relational expression
Figure BDA0002043840490000036
Obtaining a functional relation:
Figure BDA0002043840490000037
wherein, FmainHas a value range of
Figure BDA0002043840490000038
And obtaining the corresponding main lens aperture value when the utilization rate of the sensor is maximum by a method of solving the maximum value through the concave function.
In step A4, for the light field cameras with the microlens array arranged in a square shape, according to the corresponding parameters b, A and FmainThe corresponding process mask is determined as follows:
Figure BDA0002043840490000041
and performing dot multiplication processing on the special-shaped macro-pixels by using the mask, and screening to obtain the effective part of the special-shaped macro-pixels.
In step A4, for the light field cameras with the microlens array arranged in a hexagon, according to the corresponding parameters b, A and FmainThe corresponding process mask is determined as follows:
Figure BDA0002043840490000042
and performing dot multiplication processing on the special-shaped macro-pixels by using the mask, and screening to obtain the effective part of the special-shaped macro-pixels.
The invention has the following beneficial effects:
the invention provides a method for maximizing the utilization rate of a light field camera sensor based on the self-adaption of the aperture value of a main lens, which is used for maximizing the utilization rate of the light field camera sensor according to the aperture value F of the main lens of the sensormainWith the diameter D of the macropixel unit, and by determining the sensor utilization SU and the total area S of each macropixel unittolArea S of the overlap region in each celloveAnd the area S of the ineffective regionwasThe relationship between the sensor utilization rate SU and the main lens aperture value F is obtainedmainThe relation between the two, the method for solving by optimizationMethod for determining main lens aperture value F capable of maximizing sensor utilization rate SUmainDetermining a macro-pixel mask according to the corresponding parameters, and extracting the aperture value F of the light field camera in the main lens by using the maskmainLower determined macropixel (Change F)mainLater, the shape changes). The method can effectively improve the utilization rate of the light field camera sensor, improve the angular resolution of the light field camera, and enable the light field camera to decode to obtain more sub-viewing angles.
Drawings
FIG. 1 is a flow chart of an embodiment of a method of maximizing light field camera sensor utilization according to the present invention.
Fig. 2 and 3 are schematic diagrams of the total area Stol of each cell and the area love of the overlapping region and the area Swas of the ineffective region in each cell, wherein the case of the square arrangement corresponds to fig. 2, and the case of the hexagonal arrangement corresponds to fig. 3.
FIG. 4 is a diagram illustrating a mask corresponding to a square microlens array according to an embodiment of the present invention.
FIG. 5 is a mask corresponding to a hexagonal microlens array according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary in nature and is not intended to limit the scope of the invention or its application.
Referring to fig. 1-5, in one embodiment, a method for maximizing light field camera sensor utilization includes the steps of:
a1: acquiring the distance b between a microlens array and a sensor of the light field camera and the diameter of the microlens, taking the diameter of the microlens as the diameter D of the macropixel, and determining the aperture value F of a main lensmainThe relation to the macropixel diameter D;
a2: acquiring the side length A of the macro-pixel unit, and determining the total area S of each macro-pixel unit aiming at the micro-lens arrays with different arrangement formstolAnd the area S of the overlapping area in each macro-pixel unitoveAnd the area S of the ineffective areawas；
A3: determining sensor utilization SU and total area StolArea S of the overlapping regionoveAnd the area S of the ineffective regionwasUsing the results obtained in step A1 and step A2 to obtain the sensor utilization rate SU and the main lens aperture value FmainDetermining a main lens aperture value capable of maximizing the utilization rate of the sensor by an optimization solution method;
a4: for the special-shaped macro-pixel determined by the main lens aperture value obtained in the step A3, the side length A, the distance b and the main lens aperture value F are corresponding to the parametersmainAnd determining a corresponding processing mask, and processing the special-shaped macro-pixel by using the mask to obtain an effective part of the special-shaped macro-pixel.
In a preferred embodiment, in step a1, the main lens aperture F is determined based on the distance b between the microlens array and the sensor and the diameter D of the microlensesmainThe relationship with the macropixel diameter D is as follows:
Figure BDA0002043840490000051
referring to FIG. 2, in a preferred embodiment, in step A2, the total area S of each macro-pixel unit is a square microlens arraytolComprises the following steps:
Stol＝A2
area S of the overlapping region in each celloveComprises the following steps:
Figure BDA0002043840490000061
area S of the ineffective regionwasComprises the following steps:
Figure BDA0002043840490000062
wherein
Figure BDA0002043840490000063
Referring to FIG. 3, in another preferred embodiment, in step A2, the total area S of each macro-pixel cell for a hexagonal microlens arraytolComprises the following steps:
Figure BDA0002043840490000064
area S of the overlapping region in each celloveComprises the following steps:
Figure BDA0002043840490000065
area S of the ineffective regionwasComprises the following steps:
Figure BDA0002043840490000066
wherein
Figure BDA0002043840490000067
In a preferred embodiment, in step A3, the sensor utilization SU is related to the total area S of each macropixel unittolArea S of the overlapping regionoveAnd the area S of the ineffective regionwasThe relationship of (c) is determined as follows:
Figure BDA0002043840490000068
in a preferred embodiment, in step A3, for a light field camera with a square microlens array, the results obtained in steps a1 and a2 are used to substitute the relational expression
Figure BDA0002043840490000069
Obtaining a functional relation:
Figure BDA0002043840490000071
wherein, FmainHas a value range of
Figure BDA0002043840490000072
And obtaining the corresponding main lens aperture value when the utilization rate of the sensor is maximum by a method of solving the maximum value through the concave function.
In a further preferred embodiment, in step a4, for a light field camera with a square arrangement of microlens array, according to the corresponding parameters b, a and FmainThe corresponding process mask is determined as follows:
Figure BDA0002043840490000073
referring to fig. 4, the mask is used to perform dot multiplication on the irregular macro-pixels, and the effective part of the irregular macro-pixels is obtained by screening.
In another preferred embodiment, in step A3, for a light field camera with a hexagonal microlens array, the results obtained in steps a1 and a2 are used to substitute the relational expression
Figure BDA0002043840490000074
Figure BDA0002043840490000075
Obtaining a functional relation:
Figure BDA0002043840490000076
wherein, FmainHas a value range of
Figure BDA0002043840490000077
And obtaining the corresponding main lens aperture value when the utilization rate of the sensor is maximum by a method of solving the maximum value through the concave function.
In a further preferred embodiment, in step a4, for a light field camera with a hexagonal arrangement of microlens arrays, according to the corresponding parameters b, a and FmainThe corresponding process mask is determined as follows:
Figure BDA0002043840490000081
referring to fig. 5, the mask is used to perform dot multiplication on the irregular macro-pixels, and the effective part of the irregular macro-pixels is obtained by screening.
The foregoing is a more detailed description of the invention in connection with specific/preferred embodiments and is not intended to limit the practice of the invention to those descriptions. It will be apparent to those skilled in the art that various substitutions and modifications can be made to the described embodiments without departing from the spirit of the invention, and these substitutions and modifications should be considered to fall within the scope of the invention.

Claims (9)

1. A method of maximizing light field camera sensor utilization, comprising the steps of:
a1: acquiring the distance b between a microlens array and a sensor of the light field camera and the diameter of the microlens, taking the diameter of the microlens as the diameter D of the macropixel, and determining the aperture value F of a main lensmainThe relation to the macropixel diameter D;
a2: obtaining the side length A of the macro-pixel unit and determining the total area S of each macro-pixel unittolAnd the area S of the overlapping area in each macro-pixel unitoveAnd the area S of the ineffective areawas；
A3: determining sensor utilization SU and total area StolArea S of the overlapping regionoveAnd the area S of the ineffective regionwasUsing the relationship between step A1 and step A2Obtaining the sensor utilization rate SU and the main lens aperture value FmainDetermining a main lens aperture value capable of maximizing the utilization rate of the sensor by an optimization solution method;
a4: for the special-shaped macro-pixel determined by the main lens aperture value obtained in the step A3, the side length A, the distance b and the main lens aperture value F are corresponding to the parametersmainAnd determining a corresponding processing mask, and processing the special-shaped macro-pixel by using the mask to obtain an effective part of the special-shaped macro-pixel.
2. The method as claimed in claim 1, wherein in step a1, the main lens aperture F is determined based on the distance b between the microlens array and the sensor and the diameter D of the microlensesmainThe relationship with the macropixel diameter D is as follows:
Figure FDA0002768474650000011
3. the method of claim 1, wherein in step a2, for a square arrangement of microlens arrays, the total area S of each macropixel celltolComprises the following steps:
Stol＝A2
area S of the overlapping region in each celloveComprises the following steps:
Figure FDA0002768474650000012
area S of the ineffective regionwasComprises the following steps:
Figure FDA0002768474650000013
wherein
Figure FDA0002768474650000021
4. The method of claim 1, wherein in step a2, for a hexagonally arranged microlens array, the total area S of each macropixel celltolComprises the following steps:
Figure FDA0002768474650000022
area S of the overlapping region in each celloveComprises the following steps:
Figure FDA0002768474650000023
area S of the ineffective regionwasComprises the following steps:
Figure FDA0002768474650000024
wherein
Figure FDA0002768474650000025
5. A method according to claim 3 or 4, characterized in that in step A3, the sensor utilization SU is related to the total area S of each macropixel unittolArea S of the overlapping regionoveAnd the area S of the ineffective regionwasThe relationship of (c) is determined as follows:
Figure FDA0002768474650000026
6. the method of claim 5The method is characterized in that in the step A3, for the light field camera with the microlens array arranged in a square shape, the results obtained in the step A1 and the step A2 are used for substituting the relational expression
Figure FDA0002768474650000027
Obtaining a functional relation:
Figure FDA0002768474650000028
wherein, FmainHas a value range of
Figure FDA0002768474650000029
And obtaining the corresponding main lens aperture value when the utilization rate of the sensor is maximum by a method of solving the maximum value through the concave function.
7. The method of claim 5, wherein in step A3, for a light field camera with a hexagonal microlens array, the results obtained in steps A1 and A2 are substituted into the relation
Figure FDA0002768474650000031
Obtaining a functional relation:
Figure FDA0002768474650000032
wherein, FmainHas a value range of
Figure FDA0002768474650000033
And obtaining the corresponding main lens aperture value when the utilization rate of the sensor is maximum by a method of solving the maximum value through the concave function.
8. The method of claim 6, wherein in step A4, for microThe light field cameras with the lens arrays arranged in a square shape are used for imaging according to corresponding parameters b, A and FmainThe corresponding process mask is determined as follows:
Figure FDA0002768474650000034
and performing dot multiplication processing on the special-shaped macro-pixels by using the mask, and screening to obtain the effective part of the special-shaped macro-pixels.
9. The method of claim 7, wherein in step A4, for a light field camera with a hexagonal arrangement of microlens arrays, the parameters b, A and F are determined according tomainThe corresponding process mask is determined as follows:
Figure FDA0002768474650000041
and performing dot multiplication processing on the special-shaped macro-pixels by using the mask, and screening to obtain the effective part of the special-shaped macro-pixels.
CN201910350700.6A 2019-04-28 2019-04-28 Method for maximizing utilization rate of light field camera sensor Active CN109932852B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910350700.6A CN109932852B (en) 2019-04-28 2019-04-28 Method for maximizing utilization rate of light field camera sensor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910350700.6A CN109932852B (en) 2019-04-28 2019-04-28 Method for maximizing utilization rate of light field camera sensor

Publications (2)

Publication Number Publication Date
CN109932852A CN109932852A (en) 2019-06-25
CN109932852B true CN109932852B (en) 2021-02-05

Family

ID=66991237

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910350700.6A Active CN109932852B (en) 2019-04-28 2019-04-28 Method for maximizing utilization rate of light field camera sensor

Country Status (1)

Country Link
CN (1) CN109932852B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101430426A (en) * 2007-02-06 2009-05-13 三菱电机株式会社 Apparatus and method for acquiring 4D light field of scene
JP2017069644A (en) * 2015-09-28 2017-04-06 キヤノン株式会社 Imaging apparatus and imaging method
CN107027025A (en) * 2017-03-28 2017-08-08 清华大学深圳研究生院 A kind of light field image compression method based on grand block of pixels adaptive prediction
CN109246340A (en) * 2018-09-18 2019-01-18 杭州行开科技有限公司 A kind of light field image processing display system and method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7780364B2 (en) * 2008-05-01 2010-08-24 Mitsubishi Electric Research Laboratories, Inc. Apparatus and method for reducing glare in images
WO2010077625A1 (en) * 2008-12-08 2010-07-08 Refocus Imaging, Inc. Light field data acquisition devices, and methods of using and manufacturing same
US9184199B2 (en) * 2011-08-01 2015-11-10 Lytro, Inc. Optical assembly including plenoptic microlens array
JP6019568B2 (en) * 2011-11-28 2016-11-02 ソニー株式会社 Image processing apparatus and method, recording medium, and program
CN104735351A (en) * 2015-03-06 2015-06-24 中国科学院计算技术研究所 High resolution light field image recreation method and imaging device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101430426A (en) * 2007-02-06 2009-05-13 三菱电机株式会社 Apparatus and method for acquiring 4D light field of scene
JP2017069644A (en) * 2015-09-28 2017-04-06 キヤノン株式会社 Imaging apparatus and imaging method
CN107027025A (en) * 2017-03-28 2017-08-08 清华大学深圳研究生院 A kind of light field image compression method based on grand block of pixels adaptive prediction
CN109246340A (en) * 2018-09-18 2019-01-18 杭州行开科技有限公司 A kind of light field image processing display system and method

Also Published As

Publication number Publication date
CN109932852A (en) 2019-06-25

Similar Documents

Publication Publication Date Title
CN110197466B (en) Wide-angle fisheye image correction method
WO2020077866A1 (en) Moire-based image recognition method and apparatus, and device and storage medium
CN108122191B (en) Method and device for splicing fisheye images into panoramic image and panoramic video
CN108426585B (en) A kind of geometric calibration method of light-field camera
CN106485753B (en) The method and apparatus of camera calibration for pilotless automobile
WO2020007363A1 (en) Method and apparatus for identifying number of targets, and computer-readable storage medium
CN104013417B (en) A kind of X-ray optical field imaging based on pinhole array and scaling method
CN110662014B (en) Light field camera four-dimensional data large depth-of-field three-dimensional display method
CN101825840A (en) Multi-camera real-time omnidirectional imaging system
CN201725141U (en) Real-time panoramic imaging system with multi lens
CN106226327A (en) A kind of cambered surface external surface of objects defect inspection method and system
CN104809739A (en) Real-time ultra wide-angle lens camera video correcting method
CN108364309B (en) Space light field recovery method based on handheld light field camera
CN106599806A (en) Local curved-surface geometric feature-based human body action recognition method
Li et al. Rectification of images distorted by microlens array errors in plenoptic cameras
CN109932852B (en) Method for maximizing utilization rate of light field camera sensor
CN115131346A (en) Fermentation tank processing procedure detection method and system based on artificial intelligence
CN109544492A (en) A kind of multi-focus image fusion data set production method based on convolutional neural networks
CN108734054B (en) Non-shielding citrus fruit image identification method
CN105241450A (en) Sky polarization mode detection method and system based on four-quadrant polaroid
CN112633113A (en) Cross-camera human face living body detection method and system
CN110533577B (en) Fisheye image correction method and device
CN115170498B (en) Underwater polarization imaging method based on multi-index optimization
CN208079229U (en) A kind of camera array device for underwater photograph technical
CN107644442A (en) Double space position calibration methods for taking the photograph module

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant