CN108093237A - High spatial resolution optical field acquisition device and image generating method - Google Patents

High spatial resolution optical field acquisition device and image generating method Download PDF

Info

Publication number
CN108093237A
CN108093237A CN201711263820.XA CN201711263820A CN108093237A CN 108093237 A CN108093237 A CN 108093237A CN 201711263820 A CN201711263820 A CN 201711263820A CN 108093237 A CN108093237 A CN 108093237A
Authority
CN
China
Prior art keywords
sensor
lenticule
light field
imaging
compressed encoding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711263820.XA
Other languages
Chinese (zh)
Inventor
周果清
瞿婉新
肖照林
其他发明人请求不公开姓名
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN201711263820.XA priority Critical patent/CN108093237A/en
Publication of CN108093237A publication Critical patent/CN108093237A/en
Priority to CN201811439379.0A priority patent/CN109413407B/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70SSIS architectures; Circuits associated therewith
    • H04N25/76Addressed sensors, e.g. MOS or CMOS sensors

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

The present invention provides a kind of high spatial resolution optical field acquisition device and image generating methods, microlens array is positioned between main lens and compressed encoding sensor, microlens array is parallel to compressed encoding sensor, perpendicular to main lens optical axis, the photosurface of compressed encoding sensor is completely covered;Wherein each lenticule can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies;Target scene is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.The present invention obtains high spatial resolution light field data while light field angular resolution and thang-kng amount is not reduced, it is possible to reduce light field data amount alleviates storage pressure, promotes the network storage of light field video capture and light field data and propagates application development.

Description

High spatial resolution optical field acquisition device and image generating method
Technical field
The present invention relates to computer vision, calculate camera shooting and optical engineering field, and in particular to a kind of to be compiled using compression The optical field acquisition device and high-resolution light field image generation method that code sensor is combined with microlens array.
Background technology
The it is proposed of optical field imaging theory brings revolutionary advancement for digital imaging arts.It is sensed with tradition imaging only record The Theory of Projections model of device plane light distribution is different, and optical field imaging model is by the light in scene by novel optical system Line carries out corresponding according to certain given relation with sensor pixel.Optical field acquisition device can collect the position of light simultaneously And angle information, and novel image is generated by digitized processing on this basis.But compared with conventional imaging techniques, light field There is also certain limitations for imaging technique.Certain in sensor pixel, there are resolution tradeoffs for optical field imaging technology Problem.It is very huge additionally, due to light field data amount, it is unfavorable for storage and network transmission on a large scale.This has become to limit light field Camera and calculating photography are by further widely applied bottleneck problem.
In order to obtain high-resolution light field image, existing method mainly includes multiplexing method and computational methods.Stamford 128 camera arrays of university's design are the embodiments of method for spacial multiplex.The Massachusetts Institute of Technology design annular aperture camera be Time-multiplexed method is realized.The existing method of frequency domain multiplexing there are light field image signal-to-noise ratio it is low the shortcomings that, and less focus on light field The raising of angular resolution.Multiplexing method be by the compromise between light field position and angular resolution be converted to the time, space or Compromise between thang-kng amount, but corresponding acquisition method existence time resolution ratio reduces, collecting device scale increases or light field figure As the shortcomings such as signal-to-noise ratio is low, it is difficult to implement in the application.Computational methods need accurately to estimate inside and outside parameter or needs mostly For the scene geometry of sub-pixel as priori, this is also relatively difficult to achieve in practical applications.It is existing to be modulated using encoding board Light, which carries high-resolution device, there are problems that reducing luminous flux.This causes the signal-to-noise ratio for gathering signal low, the light field of reconstruction The quality of data is poor.
Therefore, on the premise of angular resolution and luminous flux is not reduced, obtaining high spatial resolution light field becomes light field Imaging field urgent problem to be solved.The high spatial resolution optical field acquisition device proposed in the present invention can efficiently solve this and ask Topic can be widely used in the fields such as light field video capture, light-field data compression transmission.
The content of the invention
For overcome the deficiencies in the prior art, the present invention provides a kind of based on random convolution cmos sensor and lenticule battle array The optical field acquisition device combined is arranged, compressed sensing technology can be utilized on the premise of angular resolution and luminous flux is not reduced Acquisition includes the light field signal of more location informations, and passes through the algorithm for reconstructing structure high spatial resolution light field number of global optimization According to.
The technical solution adopted by the present invention to solve the technical problems is:A kind of microlens array and compressed encoding sensor With reference to light field data harvester, including main lens, microlens array and compressed encoding sensor.
The microlens array is positioned between main lens and compressed encoding sensor, and microlens array is parallel to compression Perpendicular to main lens optical axis, the photosurface of compressed encoding sensor can be completely covered for code sensor;It is wherein each micro- Mirror can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies;Target scene It is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.
The arrangement of lenticule includes but are not limited to adjacent lines lenticule using linescan method in the microlens array The square of optical center alignment and the arranged in regular hexagon shape of interlacing lenticule optical center alignment.
It is most short to be no more than twice of imaging sensor pixel for the Airy spot diameter of single lenticule in the microlens array The length on side, the point range figure scope of single lenticule are no more than its Airy spot diameter.
The compressed encoding sensor uses random convolution cmos image sensor, and the physical resolution of sensor is P* Q;Using random convolutional calculation is performed pixel-by-pixel on the pseudo-random sequence control CMOS stored in shift register, then utilize Row, column selection logic unit realizes random site sampling, obtains compression sampling data;In the pseudo-random sequence cycle period Element number be more than P*Q.
Plane where the lenticule optical center is to the vector distance z of main lens focal plane>0, compressed encoding sensor light The distance g of quick face to plane where lenticule optical center is more than the focal length f, 1/g+1/z=1/f of single lenticule;The master of the present invention Lens focus F, main lens clear aperature D and lenslet diameter d meet D/ (F+z+g)=d/g.
The present invention also provides a kind of method based on compressed sensing reconstruction high spatial resolution light field, including following step Suddenly:
1) main lens, the optical dimensions of microlens array and compressed encoding sensor, location information are being unified into frame Parametric description is carried out under frame, is built based on the cost function for minimizing re-projection error, and is carried out by global optimization method It solves;
2) the high resolution 2 d light field signal that angle is coupled with location information recovers, and step is as follows:
2.1) according to the parameterized model of imaging device internal reference, the centre coordinate of each lenticule imaging unit is asked for;So Extraction carries out rotation processing to compression image, makes every row imaging single by the compressed observation signal of compressed encoding sensor afterwards The centre coordinate of member is located at in one-row pixels;Geometry rectification finally is done to the twist distortion of image, makes observation signal and measurement The correspondence of matrix is consistent;
2.2) extraction acts on the pseudo-random sequence of random coded, with reference to the measurement in Preudo-Random Sequences Generation compressed sensing Matrix;The calculation matrix includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix;
2.3) the super-resolution light field two coupled using BP methods, BPDN methods or TV methods reconstruction angle with location information Dimensional signal;
3) angular samples number is calculated according to camera parameter model;Then according to angle information and position in two-dimension light field signal The coupled relation of confidence breath, calculates the spatial resolution size that each lenticule pixel of focal plane provides, and from it is each into As unit same position extract same size block of pixels spliced, synthesize a visual angle under focal plane imaging result; The pixel decimation center of last converter unit imaging, you can synthesize the two dimensional image under multiple visual angles, realize four-dimensional light field data Structure;
4) digital refocusing method is used using light field data, generates the refocusing image of high spatial resolution.
The beneficial effects of the invention are as follows:While light field angular resolution and thang-kng amount is not reduced, high spatial point is obtained Resolution light field data, it is possible to reduce light field data amount alleviates storage pressure, promotes the net of light field video capture and light field data Network stores and propagates application development.
Description of the drawings
Fig. 1 is the light path schematic diagram of optical field acquisition device;
Fig. 2 is optical field acquisition structure drawing of device;
Fig. 3 is microlens array physical parameter schematic diagram;
Fig. 4 is the calculating schematic diagram of angular samples number, wherein, (a) is angular samples number Computing Principle, and (b) is positive four side The microlens array imaging recording interval of shape arrangement, (c) are the microlens array imaging recording interval of arranged in regular hexagon shape;
Fig. 5 is extracted for imaging unit and connecting method schematic diagram, wherein, (a) is the joining method of imaging unit, and (b) is The extraction mode of square arrangement, (c) are the extraction mode of arranged in regular hexagon shape;
Fig. 6 rebuilds route for high spatial resolution light field data.
Specific embodiment
The present invention is further described with reference to the accompanying drawings and examples, and the present invention includes but are not limited to following implementations Example.
The present invention provides the light field data acquisition dress that a kind of microlens array and random convolution cmos image sensor combine It puts, including:Main lens, for object space light beam to be made to be refracted to camera internal;One microlens array passes through main lens for separating The angle and location information of image plane light;One random convolution cmos image detecting sensor, after gathering coding compression Light field signal.
Harvester of the present invention by one piece of microlens array be positioned over main lens and random convolution cmos image sensor it Between, the photosurface of imaging sensor can be completely covered parallel to imaging sensor for microlens array.
Microlens array is positioned over main lens rear (close to image space), and parallel with primary mirror head plane, makes target scene It is recorded after the real image that main lens are in can again be converged by microlens array by imaging sensor.Wherein each lenticule Can focal imaging, and the imaging region of different lenticule is in sensor plane non-overlapping copies.
In harvester of the present invention, in microlens array the arrangement of lenticule use linescan method, include but are not limited to The square of adjacent lines lenticule optical center alignment and the arranged in regular hexagon shape of interlacing lenticule optical center alignment.Microlens array Physical parameter meets following constraint:The Airy spot diameter of single lenticule is no more than twice of imaging sensor pixel most short side Length, the point range figure scope of single lenticule be no more than its Airy spot diameter.
In acquisition system of the present invention, compressed encoding sensor generates pseudo-random sequence and it is multiplied with sensor pixel and asks With the compressed sensing for realizing signal.Specifically, a kind of random convolved image sensor is used in the present embodiment.The compression of the present invention Code sensor includes but are not limited to a kind of this compression sensor.(often row is every for sensor for P*Q for the physical resolution of sensor Row number of pixels is respectively P and Q).Initializing pseudo random sequence first, the element number in cycle period are more than sensor picture Prime P * Q.Then will pseudo-random sequence be pressed into shift register in, using shift register control CMOS on perform pixel-by-pixel with Then machine convolutional calculation realizes that random site samples using row, column selection logic unit.Compression sampling data are finally obtained, Resolution ratio is M (0<M<P*Q).It needs to meet RIP matrix propertieses with reference to the calculation matrix of random convolution and down-sampled processing construction. The arrangement of the angles and positions information of light on the image sensor has coupled relation, and the compression based on random convolution is down-sampled Processing does not destroy the coupling of light field angle information and location information.
The arrangement model (Fig. 1) of each device of acquisition system of the present invention is:Microlens array and random convolution cmos image pass Sensor is located at after the imaging plane of main lens, i.e., microlens array, random convolution cmos image sensor plane are each perpendicular to Main lens optical axis, and be distributed successively along optical axis direction.It is main lens imaging plane, micro- due to the equal focal imaging of each lenticule Lens array, the distance of random convolution cmos image sensor three meet Gaussian imaging equation, and plane where lenticule optical center To vector distance (object distance) the z > 0 of main lens focal plane, the distance of plane where imaging sensor photosurface to lenticule optical center Meet g > f between (image distance) g and the focal length f of single lenticule.The image device parameter selection of the present invention should be according to formula (1-1) Requirement.
D/ (F+z+g)=d/g (1-1)
Wherein, F is main lens focus, and D is main lens clear aperture diameter, and d is lenslet diameter.Lenticule battle array of the present invention The parameter selection of row should be according to the requirement of formula (1-2).
1/g+1/z=1/f (1-2)
The present invention also provides a kind of methods based on compressed sensing reconstruction high spatial resolution light field.Key link bag It includes:Harvester intrinsic parameter is demarcated, and the high resolution 2 d light field signal that angle is coupled with location information recovers, high-space resolution Rate four-dimension light field data is built, and high spatial resolution light field image is rebuild.The method comprises the steps of:
S1, the calibration of harvester intrinsic parameter.The location information of optical dimensions of each component in imaging device, each component is existed It is unified under picture frame and carries out parametric description, build the cost function based on minimum re-projection error, and pass through global excellent Change method is solved.
The high resolution 2 d light field signal that S2, angle are coupled with location information recovers.With reference to compressive sensing theory, to pressure It reduces the staff the low resolution two-dimension light field signal after code and carries out super-resolution rebuilding.Locate in advance including the original signal based on camera parameter Reason, the calculation matrix construction based on compressive sensing theory, the super-resolution light field 2D signal based on global optimum's method are rebuild.
S2.1, the original signal pretreatment based on camera parameter.First according to the parameterized model of imaging device internal reference, ask Take the centre coordinate of each lenticule imaging unit;Then extraction is right by the compressed observation signal of compressed encoding sensor It compresses image and carries out rotation processing, the centre coordinate of every row imaging unit is made to be located at in one-row pixels;Finally to the torsion of image Geometry rectification is done in curved change, is consistent the correspondence of observation signal and calculation matrix.
S2.2, the calculation matrix construction based on compressive sensing theory.Extraction acts on the pseudo-random sequence of random coded, root According to the operation principle of imaging sensor compressed encoding, with reference to the calculation matrix in Preudo-Random Sequences Generation compressed sensing.The present invention The calculation matrix of middle construction, which includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix etc. is suitable for firmly The compressed sensing calculation matrix that part is realized.
The super-resolution light field 2D signal that S2.3, angle are coupled with location information is rebuild.Super-resolution light field two dimension letter Number reconstruction BP (Basis Pursuit) method, BPDN (Basis Pursuit De-Noising) method, TV may be employed The compressed sensing reconstruction algorithms such as (Total Variation) method construct optimal compressed sensing according to the design of calculation matrix Method for reconstructing utilizes observation signal and calculation matrix rebuilding super resolution light field 2D signal.
S3, high spatial resolution four-dimension light field data structure.It is first depending on camera parameter model and calculates angular samples number; Then according to the coupled relation of angle information and location information in two-dimension light field signal, each lenticule picture of focal plane is calculated The spatial resolution size that member provides, and the block of pixels of the same position extraction same size from each imaging unit is spelled It connects, synthesizes the focal plane imaging result under a visual angle;The pixel decimation center of last converter unit imaging, you can synthesis is multiple Two dimensional image under visual angle realizes four-dimensional light field data structure.
S4, high spatial resolution light field image are rebuild.Using light field data using digital refocusing method, high spatial is generated The refocusing image of resolution ratio.
High spatial resolution optical field acquisition device provided in an embodiment of the present invention based on compressed encoding sensor includes three A part, as shown in Figure 1, a main lens 101, one piece of microlens array 103, compressed encoding sensor uses in the present embodiment One random convolution cmos sensor 104 (Fig. 2).Light path schematic diagram is as shown in Figure 1, scene light is reflected by main lens 101 After focus on main lens image plane 102, the light through image plane is initially separated, by the lenslet in microlens array 103 Refraction, converges on imaging sensor 104.
Specifically, microlens array 102 is as shown in figure 3, using arranged in regular hexagon shape, the diameter d of single lenticule 201 is 0.3mm, focal length 2.726mm are planoconvex spotlight.Microlens array 103 is complete by random 104 photosurface of convolved image sensor Covering, by horizontal direction is no less than 120, vertical direction is no less than 92 lenticules and formed with arranged in regular hexagon shape.Random volume Product cmos image sensor 104 and the distance g of microlens array meet formula (1-1).The random passive pixel sensings of convolution CMOS Device is the P*Q binary arrays of a standard, and the effective area of sensor single pixel is 30 μm * 30 μm.What each pixel included Photodiode maximum current is 200 μ A.
High spatial resolution light filed acquisition method proposed by the present invention with reference to shown in Fig. 6, comprises the following steps:
S1, the calibration of harvester intrinsic parameter.The present invention is by the optical dimensions of each component, location information unified into frame Parametric description is carried out under frame, with reference to light field Two plane model, using meeting light field discrete sampling and continuous expression mapping relations Intrinsic parameter scaling method demarcated.Harvester converts N number of posture in calibration process, and n is can extract on scaling boardcA spy Sign.Internal reference matrix H, camera posture T and distortion parameter d are solved using error formula (1-3) is minimized.
Wherein | | | |pt-rayFor the re-projection error of light, that is, the error between light and scaling board priori point is decoded,For Re-projection light, TnFor the Camera extrinsic under each posture, PcFor the priori point of scaling board, c values are from 1 to nc, n values from 1 to N, s value are from 1 to P, and t values are from 1 to Q.
S2. the high resolution 2 d light field signal that angle is coupled with location information recovers.Using random volume in the present embodiment Product sensor does compressed encoding to original signal;Then compressed signal is pre-processed using camera parameter;It is last according to The operation principle construction calculation matrix Φ of machine convolution sensor, utilizes observation signal y and calculation matrix Φ rebuilding super resolution light Field 2D signal.
Original signal pretreatments of the S2.1 based on camera parameter.According to the parameterized model of imaging device internal reference, ask for every The centre coordinate of a lenticule imaging unit, Geometry rectification is done to original signal, makes observation signal is corresponding with calculation matrix to close System is consistent.
Further, for asking for the centre coordinate of lenticule imaging unit.An image exposed in vain is shot first;So Afterwards using inter-class variance maximum algorithm, white exposure diagram is subjected to binary conversion treatment;Finally using algorithm of region growing, picture is divided into Unit asks for the mean center coordinate (formula 1-4) of each unit.
Wherein ciFor the centre coordinate of an imaging unit, (xi,j,yi,j) it is that an imaging unit is in binary picture In point, mcFor pixel number in an imaging unit, the value of j is from 1 to mc
Further, for the Geometry rectification of original signal.Extraction is by the compressed sight of compressed encoding sensor first Survey signal y;Then the centre coordinate of each imaging unit is extracted, the slope of often row unit center is calculated using least square method, Take average;Then rotation process is carried out to initial data y, makes the centre coordinate of every row cell imaging generally within same one-row pixels On;Finally the deformation such as the distortion shear of image are done with Geometry rectification, and removal vignetting processing is done to edge imaging unit.
Calculation matrix constructions of the S2.2 based on compressive sensing theory.The present embodiment believes light field using random volume machine sensor Number do compressed encoding processing.Therefore the signal x ∈ R before compressingPQ*1, by random convolution filter a treated expression formulas such as Shown in formula 1-5.
Wherein r (i) ∈ { 1 ..., M }, i values are from 1 to M, and j values from 1 to P*Q, select at random in section by r (i) values It selects, Φ is the calculation matrix being made of random convolution filter a.In the present embodiment, wave filter a values are chosen for ± 1, measurement The form of matrix Φ is as follows.Therefore calculation matrix is Bernoulli Jacob's matrix that value is ± 1, meets RIP matrix propertieses, can use In the reconstruction of compressed sensing.
S2.3, the super-resolution light field 2D signal method for reconstructing of global optimum.It is combined in the present embodiment using L1 norms The algorithm for reconstructing of Total Variation.Using observation signal y and calculation matrix Φ as input, using shown in formula (1-7) Method rebuilds the light field signal of two-dimensional super-resolution rate.Wherein | | | |TVFor gradient operator, ε is noise error.
S3, high spatial resolution four-dimension light field data structure.The present embodiment is according to angle in 2D signal and location information Coupled relation, by extracting, the block of pixels in joining image-forming unit, obtain two dimension of the light field on focusing surface under a visual angle Imaging.Change the pixel decimation center of cell imaging, the multi-view image on focusing surface can be obtained, so as to decode to obtain four-dimensional light Field data.And the size of block of pixels is related to angular samples number.
Further, for calculating angular samples number m.Angular samples number m and imaging sensor photosurface to lenticule battle array The distance g at row center, the distance z at microlens array center to main lens focal plane are related (Fig. 4), and to | z |/g is directly proportional, then Angular samples number m is as shown in formula 1-7.
M=k × | z |/g (1-8)
The arrangement of wherein k and lenticule, interval, lenticule rear imaging region size are related.Lenticule battle array in the present invention Row use arranged in regular hexagon shape, thereforeFig. 4 (a) is angular samples number Computing Principle, and Fig. 4 (b) arranges for square The microlens array imaging recording interval of cloth, Fig. 4 (c) are the microlens array imaging recording interval of arranged in regular hexagon shape.Wherein r For the radius of a lenticule rear imaging.
Further, for the block of pixels in extraction, joining image-forming unit.To every width imaging unit, distance center is intercepted The block of pixels of same distance d/m sizes is spliced according to the arrangement mode of imaging unit.The splicing of image and extraction principle such as Fig. 5 Shown, (a) is the joining method of imaging unit, and (b) is the extraction mode of square arrangement, and (c) is the pumping of arranged in regular hexagon shape Take mode.
S4, high spatial resolution light field image are rebuild.Digital refocusing method is combined using light field data, generates high spatial The refocusing image of resolution ratio.Then edge effect caused by mitigating splicing using Gaussian filter.
Further, for digital refocusing method.According to the depth of focus of scene, focus point and microlens array are calculated The distance between z.The size for extracting block of pixels is calculated with reference to formula (1-7).Using in extraction, joining image-forming unit in S3 The method of block of pixels synthesizes the imaging results at rendering plane.
Further, for gaussian filtering process.To mitigate the edge effect that splicing generates, [d/m] × [d/ is utilized M] size, average be 6 Gaussian filter to entire image carry out two-dimensional convolution processing, inhibit splicing tape unsmooth effect Fruit.

Claims (6)

1. a kind of high spatial resolution optical field acquisition device, special including main lens, microlens array and compressed encoding sensor Sign is:The microlens array is positioned between main lens and compressed encoding sensor, and microlens array is parallel to compression Perpendicular to main lens optical axis, the photosurface of compressed encoding sensor can be completely covered for code sensor;It is wherein each micro- Mirror can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies;Target scene It is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.
2. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The microlens array The arrangement of middle lenticule includes but are not limited to square and the interlacing of adjacent lines lenticule optical center alignment using linescan method The arranged in regular hexagon shape of lenticule optical center alignment.
3. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The microlens array In the Airy spot diameter of single lenticule be no more than the length of twice imaging sensor pixel most short side, the point range of single lenticule Figure scope is no more than its Airy spot diameter.
4. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The compressed encoding passes Sensor uses random convolution cmos image sensor, and the physical resolution of sensor is P*Q;Utilize what is stored in shift register Random convolutional calculation is performed pixel-by-pixel on pseudo-random sequence control CMOS, is then realized using row, column selection logic unit random Position samples, and obtains compression sampling data;Element number in the pseudo-random sequence cycle period is more than P*Q.
5. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The lenticule optical center Place plane is to the vector distance z of main lens focal plane>0, plane where compressed encoding sensor photosurface to lenticule optical center Distance g be more than single lenticule focal length f, 1/g+1/z=1/f;Main lens focal length F, the main lens clear aperature D of the present invention Meet D/ (F+z+g)=d/g with lenslet diameter d.
6. a kind of image generating method of high spatial resolution optical field acquisition device described in claim 1, it is characterised in that including Following step:
1) by main lens, the optical dimensions of microlens array and compressed encoding sensor, location information in the case where being unified into picture frame Parametric description is carried out, the cost function based on minimum re-projection error is built, and passes through global optimization method and solved;
2) the high resolution 2 d light field signal that angle is coupled with location information recovers, and step is as follows:
2.1) according to the parameterized model of imaging device internal reference, the centre coordinate of each lenticule imaging unit is asked for;Then carry It learns from else's experience the compressed observation signal of overcompression code sensor, rotation processing is carried out to compression image, makes every row imaging unit Centre coordinate is located at in one-row pixels;Geometry rectification finally is done to the twist distortion of image, makes observation signal and calculation matrix Correspondence be consistent;
2.2) extraction acts on the pseudo-random sequence of random coded, with reference to the measurement square in Preudo-Random Sequences Generation compressed sensing Battle array;The calculation matrix includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix;
2.3) the super-resolution light field two dimension letter coupled using BP methods, BPDN methods or TV methods reconstruction angle with location information Number;
3) angular samples number is calculated according to camera parameter model;Then believed according to angle information in two-dimension light field signal and position The coupled relation of breath calculates the spatial resolution size that each lenticule pixel of focal plane provides, and single from each imaging The block of pixels that the same position of member extracts same size is spliced, and synthesizes the focal plane imaging result under a visual angle;Finally The pixel decimation center of converter unit imaging, you can synthesize the two dimensional image under multiple visual angles, realize four-dimensional light field data structure;
4) digital refocusing method is used using light field data, generates the refocusing image of high spatial resolution.
CN201711263820.XA 2017-12-05 2017-12-05 High spatial resolution optical field acquisition device and image generating method Pending CN108093237A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201711263820.XA CN108093237A (en) 2017-12-05 2017-12-05 High spatial resolution optical field acquisition device and image generating method
CN201811439379.0A CN109413407B (en) 2017-12-05 2018-11-29 High spatial resolution light field acquisition device and image generation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711263820.XA CN108093237A (en) 2017-12-05 2017-12-05 High spatial resolution optical field acquisition device and image generating method

Publications (1)

Publication Number Publication Date
CN108093237A true CN108093237A (en) 2018-05-29

Family

ID=62173756

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201711263820.XA Pending CN108093237A (en) 2017-12-05 2017-12-05 High spatial resolution optical field acquisition device and image generating method
CN201811439379.0A Active CN109413407B (en) 2017-12-05 2018-11-29 High spatial resolution light field acquisition device and image generation method

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN201811439379.0A Active CN109413407B (en) 2017-12-05 2018-11-29 High spatial resolution light field acquisition device and image generation method

Country Status (1)

Country Link
CN (2) CN108093237A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109297925A (en) * 2018-10-09 2019-02-01 天津大学 A kind of Terahertz high-resolution fast imaging device based on splits' positions perception
CN110928113A (en) * 2019-12-03 2020-03-27 西北工业大学 Light field acquisition device with variable spatial resolution
CN111157500A (en) * 2020-01-06 2020-05-15 南开大学 Transient body imaging microscope system using light sheet crystal lattice array illumination
CN111273050A (en) * 2020-02-12 2020-06-12 清华大学 Signal acquisition processing method and device
CN111340888A (en) * 2019-12-23 2020-06-26 首都师范大学 Light field camera calibration method and system without white image
CN111610634A (en) * 2020-06-23 2020-09-01 京东方科技集团股份有限公司 Display system based on four-dimensional light field and display method thereof
CN113327211A (en) * 2021-07-28 2021-08-31 清华大学 Correction method and device for large-field-of-view high-resolution light field microscope system
WO2021254110A1 (en) * 2020-06-19 2021-12-23 京东方科技集团股份有限公司 Image processing method, apparatus and device, and storage medium

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112396670B (en) * 2019-08-15 2023-01-10 天津大学青岛海洋技术研究院 Image reconstruction method for novel binary image sensor
CN111598775B (en) * 2020-04-26 2023-02-03 西安理工大学 Light field video time domain super-resolution reconstruction method based on LSTM network
CN112945141B (en) * 2021-01-29 2023-03-14 中北大学 Structured light rapid imaging method and system based on micro-lens array

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7962033B2 (en) * 2008-01-23 2011-06-14 Adobe Systems Incorporated Methods and apparatus for full-resolution light-field capture and rendering
CN102739945B (en) * 2012-05-24 2014-09-03 上海理工大学 Optical field imaging device and method
EP3116216A4 (en) * 2014-03-03 2017-02-15 Panasonic Intellectual Property Management Co., Ltd. Image pickup apparatus
CN103871038B (en) * 2014-03-06 2014-11-05 中国人民解放军国防科学技术大学 Super-resolution omnidirectional image reconstruction method based on non-uniform measurement matrix
CN104036489B (en) * 2014-05-09 2017-10-03 北京工业大学 A kind of light filed acquisition method
CN105704371B (en) * 2016-01-25 2019-04-26 深圳市未来媒体技术研究院 A kind of light field refocusing method
CN106303228B (en) * 2016-08-04 2019-09-13 深圳市未来媒体技术研究院 A kind of rendering method and system of focus type light-field camera

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109297925A (en) * 2018-10-09 2019-02-01 天津大学 A kind of Terahertz high-resolution fast imaging device based on splits' positions perception
CN110928113A (en) * 2019-12-03 2020-03-27 西北工业大学 Light field acquisition device with variable spatial resolution
CN110928113B (en) * 2019-12-03 2021-10-08 西北工业大学 Light field acquisition device with variable spatial resolution
WO2021129437A1 (en) * 2019-12-23 2021-07-01 首都师范大学 Method and system for light calibration field camera without requiring white image
CN111340888A (en) * 2019-12-23 2020-06-26 首都师范大学 Light field camera calibration method and system without white image
CN111340888B (en) * 2019-12-23 2020-10-23 首都师范大学 Light field camera calibration method and system without white image
CN111157500A (en) * 2020-01-06 2020-05-15 南开大学 Transient body imaging microscope system using light sheet crystal lattice array illumination
CN111273050B (en) * 2020-02-12 2022-05-20 清华大学 Signal acquisition processing method and device
CN111273050A (en) * 2020-02-12 2020-06-12 清华大学 Signal acquisition processing method and device
WO2021254110A1 (en) * 2020-06-19 2021-12-23 京东方科技集团股份有限公司 Image processing method, apparatus and device, and storage medium
US11997397B2 (en) 2020-06-19 2024-05-28 Beijing Boe Optoelectronics Technology Co., Ltd. Method, apparatus, and device for processing images, and storage medium
CN111610634A (en) * 2020-06-23 2020-09-01 京东方科技集团股份有限公司 Display system based on four-dimensional light field and display method thereof
CN111610634B (en) * 2020-06-23 2022-05-27 京东方科技集团股份有限公司 Display system based on four-dimensional light field and display method thereof
US11789287B2 (en) 2020-06-23 2023-10-17 Beijing Boe Optoelectronics Technology Co., Ltd. Display system based on four-dimensional light field, and display method therefor
CN113327211A (en) * 2021-07-28 2021-08-31 清华大学 Correction method and device for large-field-of-view high-resolution light field microscope system
CN113327211B (en) * 2021-07-28 2021-11-02 清华大学 Correction method and device for large-field-of-view high-resolution light field microscope system
WO2023005671A1 (en) * 2021-07-28 2023-02-02 清华大学 Correction method and apparatus for large-field-of-view high-resolution light field microscopic system

Also Published As

Publication number Publication date
CN109413407A (en) 2019-03-01
CN109413407B (en) 2020-08-07

Similar Documents

Publication Publication Date Title
CN108093237A (en) High spatial resolution optical field acquisition device and image generating method
Wu et al. Light field image processing: An overview
Wang et al. End-to-end view synthesis for light field imaging with pseudo 4DCNN
Farrugia et al. Light field super-resolution using a low-rank prior and deep convolutional neural networks
Venkataraman et al. Picam: An ultra-thin high performance monolithic camera array
CN102959945B (en) The method and system of virtual output image is produced according to the data obtained by the array of image-capturing apparatus
US8290358B1 (en) Methods and apparatus for light-field imaging
WO2020117657A1 (en) Enhancing performance capture with real-time neural rendering
Ding et al. Dynamic fluid surface acquisition using a camera array
Wang et al. High-fidelity view synthesis for light field imaging with extended pseudo 4DCNN
Dąbała et al. Efficient Multi‐image Correspondences for On‐line Light Field Video Processing
Schwarz et al. A weighted optimization approach to time-of-flight sensor fusion
KR20160106045A (en) A light field processing method
CN108364309B (en) Space light field recovery method based on handheld light field camera
CN110662014B (en) Light field camera four-dimensional data large depth-of-field three-dimensional display method
US10762612B2 (en) Method and an apparatus for generating data representative of a pixel beam
CN1959341A (en) Optical imaging distance measuring device for single-aperture multiple imaging
CN109118544A (en) Synthetic aperture imaging method based on perspective transform
Vadathya et al. Learning light field reconstruction from a single coded image
US10909704B2 (en) Apparatus and a method for generating data representing a pixel beam
Neumann et al. Eyes from eyes: analysis of camera design using plenoptic video geometry
Bonatto et al. Multiview from micro-lens image of multi-focused plenoptic camera
CN116402908A (en) Dense light field image reconstruction method based on heterogeneous imaging
Tateishi et al. Time-multiplexed coded aperture and coded focal stack-comparative study on snapshot compressive light field imaging
CN115950534A (en) Focus-dividing plane type high-luminous-flux bionic multispectral polarization imaging detector

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180529