CN108093237A - High spatial resolution optical field acquisition device and image generating method - Google Patents
High spatial resolution optical field acquisition device and image generating method Download PDFInfo
- Publication number
- CN108093237A CN108093237A CN201711263820.XA CN201711263820A CN108093237A CN 108093237 A CN108093237 A CN 108093237A CN 201711263820 A CN201711263820 A CN 201711263820A CN 108093237 A CN108093237 A CN 108093237A
- Authority
- CN
- China
- Prior art keywords
- sensor
- lenticule
- light field
- imaging
- compressed encoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The present invention provides a kind of high spatial resolution optical field acquisition device and image generating methods, microlens array is positioned between main lens and compressed encoding sensor, microlens array is parallel to compressed encoding sensor, perpendicular to main lens optical axis, the photosurface of compressed encoding sensor is completely covered;Wherein each lenticule can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies;Target scene is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.The present invention obtains high spatial resolution light field data while light field angular resolution and thang-kng amount is not reduced, it is possible to reduce light field data amount alleviates storage pressure, promotes the network storage of light field video capture and light field data and propagates application development.
Description
Technical field
The present invention relates to computer vision, calculate camera shooting and optical engineering field, and in particular to a kind of to be compiled using compression
The optical field acquisition device and high-resolution light field image generation method that code sensor is combined with microlens array.
Background technology
The it is proposed of optical field imaging theory brings revolutionary advancement for digital imaging arts.It is sensed with tradition imaging only record
The Theory of Projections model of device plane light distribution is different, and optical field imaging model is by the light in scene by novel optical system
Line carries out corresponding according to certain given relation with sensor pixel.Optical field acquisition device can collect the position of light simultaneously
And angle information, and novel image is generated by digitized processing on this basis.But compared with conventional imaging techniques, light field
There is also certain limitations for imaging technique.Certain in sensor pixel, there are resolution tradeoffs for optical field imaging technology
Problem.It is very huge additionally, due to light field data amount, it is unfavorable for storage and network transmission on a large scale.This has become to limit light field
Camera and calculating photography are by further widely applied bottleneck problem.
In order to obtain high-resolution light field image, existing method mainly includes multiplexing method and computational methods.Stamford
128 camera arrays of university's design are the embodiments of method for spacial multiplex.The Massachusetts Institute of Technology design annular aperture camera be
Time-multiplexed method is realized.The existing method of frequency domain multiplexing there are light field image signal-to-noise ratio it is low the shortcomings that, and less focus on light field
The raising of angular resolution.Multiplexing method be by the compromise between light field position and angular resolution be converted to the time, space or
Compromise between thang-kng amount, but corresponding acquisition method existence time resolution ratio reduces, collecting device scale increases or light field figure
As the shortcomings such as signal-to-noise ratio is low, it is difficult to implement in the application.Computational methods need accurately to estimate inside and outside parameter or needs mostly
For the scene geometry of sub-pixel as priori, this is also relatively difficult to achieve in practical applications.It is existing to be modulated using encoding board
Light, which carries high-resolution device, there are problems that reducing luminous flux.This causes the signal-to-noise ratio for gathering signal low, the light field of reconstruction
The quality of data is poor.
Therefore, on the premise of angular resolution and luminous flux is not reduced, obtaining high spatial resolution light field becomes light field
Imaging field urgent problem to be solved.The high spatial resolution optical field acquisition device proposed in the present invention can efficiently solve this and ask
Topic can be widely used in the fields such as light field video capture, light-field data compression transmission.
The content of the invention
For overcome the deficiencies in the prior art, the present invention provides a kind of based on random convolution cmos sensor and lenticule battle array
The optical field acquisition device combined is arranged, compressed sensing technology can be utilized on the premise of angular resolution and luminous flux is not reduced
Acquisition includes the light field signal of more location informations, and passes through the algorithm for reconstructing structure high spatial resolution light field number of global optimization
According to.
The technical solution adopted by the present invention to solve the technical problems is:A kind of microlens array and compressed encoding sensor
With reference to light field data harvester, including main lens, microlens array and compressed encoding sensor.
The microlens array is positioned between main lens and compressed encoding sensor, and microlens array is parallel to compression
Perpendicular to main lens optical axis, the photosurface of compressed encoding sensor can be completely covered for code sensor;It is wherein each micro-
Mirror can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies;Target scene
It is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.
The arrangement of lenticule includes but are not limited to adjacent lines lenticule using linescan method in the microlens array
The square of optical center alignment and the arranged in regular hexagon shape of interlacing lenticule optical center alignment.
It is most short to be no more than twice of imaging sensor pixel for the Airy spot diameter of single lenticule in the microlens array
The length on side, the point range figure scope of single lenticule are no more than its Airy spot diameter.
The compressed encoding sensor uses random convolution cmos image sensor, and the physical resolution of sensor is P*
Q;Using random convolutional calculation is performed pixel-by-pixel on the pseudo-random sequence control CMOS stored in shift register, then utilize
Row, column selection logic unit realizes random site sampling, obtains compression sampling data;In the pseudo-random sequence cycle period
Element number be more than P*Q.
Plane where the lenticule optical center is to the vector distance z of main lens focal plane>0, compressed encoding sensor light
The distance g of quick face to plane where lenticule optical center is more than the focal length f, 1/g+1/z=1/f of single lenticule;The master of the present invention
Lens focus F, main lens clear aperature D and lenslet diameter d meet D/ (F+z+g)=d/g.
The present invention also provides a kind of method based on compressed sensing reconstruction high spatial resolution light field, including following step
Suddenly:
1) main lens, the optical dimensions of microlens array and compressed encoding sensor, location information are being unified into frame
Parametric description is carried out under frame, is built based on the cost function for minimizing re-projection error, and is carried out by global optimization method
It solves;
2) the high resolution 2 d light field signal that angle is coupled with location information recovers, and step is as follows:
2.1) according to the parameterized model of imaging device internal reference, the centre coordinate of each lenticule imaging unit is asked for;So
Extraction carries out rotation processing to compression image, makes every row imaging single by the compressed observation signal of compressed encoding sensor afterwards
The centre coordinate of member is located at in one-row pixels;Geometry rectification finally is done to the twist distortion of image, makes observation signal and measurement
The correspondence of matrix is consistent;
2.2) extraction acts on the pseudo-random sequence of random coded, with reference to the measurement in Preudo-Random Sequences Generation compressed sensing
Matrix;The calculation matrix includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix;
2.3) the super-resolution light field two coupled using BP methods, BPDN methods or TV methods reconstruction angle with location information
Dimensional signal;
3) angular samples number is calculated according to camera parameter model;Then according to angle information and position in two-dimension light field signal
The coupled relation of confidence breath, calculates the spatial resolution size that each lenticule pixel of focal plane provides, and from it is each into
As unit same position extract same size block of pixels spliced, synthesize a visual angle under focal plane imaging result;
The pixel decimation center of last converter unit imaging, you can synthesize the two dimensional image under multiple visual angles, realize four-dimensional light field data
Structure;
4) digital refocusing method is used using light field data, generates the refocusing image of high spatial resolution.
The beneficial effects of the invention are as follows:While light field angular resolution and thang-kng amount is not reduced, high spatial point is obtained
Resolution light field data, it is possible to reduce light field data amount alleviates storage pressure, promotes the net of light field video capture and light field data
Network stores and propagates application development.
Description of the drawings
Fig. 1 is the light path schematic diagram of optical field acquisition device;
Fig. 2 is optical field acquisition structure drawing of device;
Fig. 3 is microlens array physical parameter schematic diagram;
Fig. 4 is the calculating schematic diagram of angular samples number, wherein, (a) is angular samples number Computing Principle, and (b) is positive four side
The microlens array imaging recording interval of shape arrangement, (c) are the microlens array imaging recording interval of arranged in regular hexagon shape;
Fig. 5 is extracted for imaging unit and connecting method schematic diagram, wherein, (a) is the joining method of imaging unit, and (b) is
The extraction mode of square arrangement, (c) are the extraction mode of arranged in regular hexagon shape;
Fig. 6 rebuilds route for high spatial resolution light field data.
Specific embodiment
The present invention is further described with reference to the accompanying drawings and examples, and the present invention includes but are not limited to following implementations
Example.
The present invention provides the light field data acquisition dress that a kind of microlens array and random convolution cmos image sensor combine
It puts, including:Main lens, for object space light beam to be made to be refracted to camera internal;One microlens array passes through main lens for separating
The angle and location information of image plane light;One random convolution cmos image detecting sensor, after gathering coding compression
Light field signal.
Harvester of the present invention by one piece of microlens array be positioned over main lens and random convolution cmos image sensor it
Between, the photosurface of imaging sensor can be completely covered parallel to imaging sensor for microlens array.
Microlens array is positioned over main lens rear (close to image space), and parallel with primary mirror head plane, makes target scene
It is recorded after the real image that main lens are in can again be converged by microlens array by imaging sensor.Wherein each lenticule
Can focal imaging, and the imaging region of different lenticule is in sensor plane non-overlapping copies.
In harvester of the present invention, in microlens array the arrangement of lenticule use linescan method, include but are not limited to
The square of adjacent lines lenticule optical center alignment and the arranged in regular hexagon shape of interlacing lenticule optical center alignment.Microlens array
Physical parameter meets following constraint:The Airy spot diameter of single lenticule is no more than twice of imaging sensor pixel most short side
Length, the point range figure scope of single lenticule be no more than its Airy spot diameter.
In acquisition system of the present invention, compressed encoding sensor generates pseudo-random sequence and it is multiplied with sensor pixel and asks
With the compressed sensing for realizing signal.Specifically, a kind of random convolved image sensor is used in the present embodiment.The compression of the present invention
Code sensor includes but are not limited to a kind of this compression sensor.(often row is every for sensor for P*Q for the physical resolution of sensor
Row number of pixels is respectively P and Q).Initializing pseudo random sequence first, the element number in cycle period are more than sensor picture
Prime P * Q.Then will pseudo-random sequence be pressed into shift register in, using shift register control CMOS on perform pixel-by-pixel with
Then machine convolutional calculation realizes that random site samples using row, column selection logic unit.Compression sampling data are finally obtained,
Resolution ratio is M (0<M<P*Q).It needs to meet RIP matrix propertieses with reference to the calculation matrix of random convolution and down-sampled processing construction.
The arrangement of the angles and positions information of light on the image sensor has coupled relation, and the compression based on random convolution is down-sampled
Processing does not destroy the coupling of light field angle information and location information.
The arrangement model (Fig. 1) of each device of acquisition system of the present invention is:Microlens array and random convolution cmos image pass
Sensor is located at after the imaging plane of main lens, i.e., microlens array, random convolution cmos image sensor plane are each perpendicular to
Main lens optical axis, and be distributed successively along optical axis direction.It is main lens imaging plane, micro- due to the equal focal imaging of each lenticule
Lens array, the distance of random convolution cmos image sensor three meet Gaussian imaging equation, and plane where lenticule optical center
To vector distance (object distance) the z > 0 of main lens focal plane, the distance of plane where imaging sensor photosurface to lenticule optical center
Meet g > f between (image distance) g and the focal length f of single lenticule.The image device parameter selection of the present invention should be according to formula (1-1)
Requirement.
D/ (F+z+g)=d/g (1-1)
Wherein, F is main lens focus, and D is main lens clear aperture diameter, and d is lenslet diameter.Lenticule battle array of the present invention
The parameter selection of row should be according to the requirement of formula (1-2).
1/g+1/z=1/f (1-2)
The present invention also provides a kind of methods based on compressed sensing reconstruction high spatial resolution light field.Key link bag
It includes:Harvester intrinsic parameter is demarcated, and the high resolution 2 d light field signal that angle is coupled with location information recovers, high-space resolution
Rate four-dimension light field data is built, and high spatial resolution light field image is rebuild.The method comprises the steps of:
S1, the calibration of harvester intrinsic parameter.The location information of optical dimensions of each component in imaging device, each component is existed
It is unified under picture frame and carries out parametric description, build the cost function based on minimum re-projection error, and pass through global excellent
Change method is solved.
The high resolution 2 d light field signal that S2, angle are coupled with location information recovers.With reference to compressive sensing theory, to pressure
It reduces the staff the low resolution two-dimension light field signal after code and carries out super-resolution rebuilding.Locate in advance including the original signal based on camera parameter
Reason, the calculation matrix construction based on compressive sensing theory, the super-resolution light field 2D signal based on global optimum's method are rebuild.
S2.1, the original signal pretreatment based on camera parameter.First according to the parameterized model of imaging device internal reference, ask
Take the centre coordinate of each lenticule imaging unit;Then extraction is right by the compressed observation signal of compressed encoding sensor
It compresses image and carries out rotation processing, the centre coordinate of every row imaging unit is made to be located at in one-row pixels;Finally to the torsion of image
Geometry rectification is done in curved change, is consistent the correspondence of observation signal and calculation matrix.
S2.2, the calculation matrix construction based on compressive sensing theory.Extraction acts on the pseudo-random sequence of random coded, root
According to the operation principle of imaging sensor compressed encoding, with reference to the calculation matrix in Preudo-Random Sequences Generation compressed sensing.The present invention
The calculation matrix of middle construction, which includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix etc. is suitable for firmly
The compressed sensing calculation matrix that part is realized.
The super-resolution light field 2D signal that S2.3, angle are coupled with location information is rebuild.Super-resolution light field two dimension letter
Number reconstruction BP (Basis Pursuit) method, BPDN (Basis Pursuit De-Noising) method, TV may be employed
The compressed sensing reconstruction algorithms such as (Total Variation) method construct optimal compressed sensing according to the design of calculation matrix
Method for reconstructing utilizes observation signal and calculation matrix rebuilding super resolution light field 2D signal.
S3, high spatial resolution four-dimension light field data structure.It is first depending on camera parameter model and calculates angular samples number;
Then according to the coupled relation of angle information and location information in two-dimension light field signal, each lenticule picture of focal plane is calculated
The spatial resolution size that member provides, and the block of pixels of the same position extraction same size from each imaging unit is spelled
It connects, synthesizes the focal plane imaging result under a visual angle;The pixel decimation center of last converter unit imaging, you can synthesis is multiple
Two dimensional image under visual angle realizes four-dimensional light field data structure.
S4, high spatial resolution light field image are rebuild.Using light field data using digital refocusing method, high spatial is generated
The refocusing image of resolution ratio.
High spatial resolution optical field acquisition device provided in an embodiment of the present invention based on compressed encoding sensor includes three
A part, as shown in Figure 1, a main lens 101, one piece of microlens array 103, compressed encoding sensor uses in the present embodiment
One random convolution cmos sensor 104 (Fig. 2).Light path schematic diagram is as shown in Figure 1, scene light is reflected by main lens 101
After focus on main lens image plane 102, the light through image plane is initially separated, by the lenslet in microlens array 103
Refraction, converges on imaging sensor 104.
Specifically, microlens array 102 is as shown in figure 3, using arranged in regular hexagon shape, the diameter d of single lenticule 201 is
0.3mm, focal length 2.726mm are planoconvex spotlight.Microlens array 103 is complete by random 104 photosurface of convolved image sensor
Covering, by horizontal direction is no less than 120, vertical direction is no less than 92 lenticules and formed with arranged in regular hexagon shape.Random volume
Product cmos image sensor 104 and the distance g of microlens array meet formula (1-1).The random passive pixel sensings of convolution CMOS
Device is the P*Q binary arrays of a standard, and the effective area of sensor single pixel is 30 μm * 30 μm.What each pixel included
Photodiode maximum current is 200 μ A.
High spatial resolution light filed acquisition method proposed by the present invention with reference to shown in Fig. 6, comprises the following steps:
S1, the calibration of harvester intrinsic parameter.The present invention is by the optical dimensions of each component, location information unified into frame
Parametric description is carried out under frame, with reference to light field Two plane model, using meeting light field discrete sampling and continuous expression mapping relations
Intrinsic parameter scaling method demarcated.Harvester converts N number of posture in calibration process, and n is can extract on scaling boardcA spy
Sign.Internal reference matrix H, camera posture T and distortion parameter d are solved using error formula (1-3) is minimized.
Wherein | | | |pt-rayFor the re-projection error of light, that is, the error between light and scaling board priori point is decoded,For
Re-projection light, TnFor the Camera extrinsic under each posture, PcFor the priori point of scaling board, c values are from 1 to nc, n values from 1 to
N, s value are from 1 to P, and t values are from 1 to Q.
S2. the high resolution 2 d light field signal that angle is coupled with location information recovers.Using random volume in the present embodiment
Product sensor does compressed encoding to original signal;Then compressed signal is pre-processed using camera parameter;It is last according to
The operation principle construction calculation matrix Φ of machine convolution sensor, utilizes observation signal y and calculation matrix Φ rebuilding super resolution light
Field 2D signal.
Original signal pretreatments of the S2.1 based on camera parameter.According to the parameterized model of imaging device internal reference, ask for every
The centre coordinate of a lenticule imaging unit, Geometry rectification is done to original signal, makes observation signal is corresponding with calculation matrix to close
System is consistent.
Further, for asking for the centre coordinate of lenticule imaging unit.An image exposed in vain is shot first;So
Afterwards using inter-class variance maximum algorithm, white exposure diagram is subjected to binary conversion treatment;Finally using algorithm of region growing, picture is divided into
Unit asks for the mean center coordinate (formula 1-4) of each unit.
Wherein ciFor the centre coordinate of an imaging unit, (xi,j,yi,j) it is that an imaging unit is in binary picture
In point, mcFor pixel number in an imaging unit, the value of j is from 1 to mc。
Further, for the Geometry rectification of original signal.Extraction is by the compressed sight of compressed encoding sensor first
Survey signal y;Then the centre coordinate of each imaging unit is extracted, the slope of often row unit center is calculated using least square method,
Take average;Then rotation process is carried out to initial data y, makes the centre coordinate of every row cell imaging generally within same one-row pixels
On;Finally the deformation such as the distortion shear of image are done with Geometry rectification, and removal vignetting processing is done to edge imaging unit.
Calculation matrix constructions of the S2.2 based on compressive sensing theory.The present embodiment believes light field using random volume machine sensor
Number do compressed encoding processing.Therefore the signal x ∈ R before compressingPQ*1, by random convolution filter a treated expression formulas such as
Shown in formula 1-5.
Wherein r (i) ∈ { 1 ..., M }, i values are from 1 to M, and j values from 1 to P*Q, select at random in section by r (i) values
It selects, Φ is the calculation matrix being made of random convolution filter a.In the present embodiment, wave filter a values are chosen for ± 1, measurement
The form of matrix Φ is as follows.Therefore calculation matrix is Bernoulli Jacob's matrix that value is ± 1, meets RIP matrix propertieses, can use
In the reconstruction of compressed sensing.
S2.3, the super-resolution light field 2D signal method for reconstructing of global optimum.It is combined in the present embodiment using L1 norms
The algorithm for reconstructing of Total Variation.Using observation signal y and calculation matrix Φ as input, using shown in formula (1-7)
Method rebuilds the light field signal of two-dimensional super-resolution rate.Wherein | | | |TVFor gradient operator, ε is noise error.
S3, high spatial resolution four-dimension light field data structure.The present embodiment is according to angle in 2D signal and location information
Coupled relation, by extracting, the block of pixels in joining image-forming unit, obtain two dimension of the light field on focusing surface under a visual angle
Imaging.Change the pixel decimation center of cell imaging, the multi-view image on focusing surface can be obtained, so as to decode to obtain four-dimensional light
Field data.And the size of block of pixels is related to angular samples number.
Further, for calculating angular samples number m.Angular samples number m and imaging sensor photosurface to lenticule battle array
The distance g at row center, the distance z at microlens array center to main lens focal plane are related (Fig. 4), and to | z |/g is directly proportional, then
Angular samples number m is as shown in formula 1-7.
M=k × | z |/g (1-8)
The arrangement of wherein k and lenticule, interval, lenticule rear imaging region size are related.Lenticule battle array in the present invention
Row use arranged in regular hexagon shape, thereforeFig. 4 (a) is angular samples number Computing Principle, and Fig. 4 (b) arranges for square
The microlens array imaging recording interval of cloth, Fig. 4 (c) are the microlens array imaging recording interval of arranged in regular hexagon shape.Wherein r
For the radius of a lenticule rear imaging.
Further, for the block of pixels in extraction, joining image-forming unit.To every width imaging unit, distance center is intercepted
The block of pixels of same distance d/m sizes is spliced according to the arrangement mode of imaging unit.The splicing of image and extraction principle such as Fig. 5
Shown, (a) is the joining method of imaging unit, and (b) is the extraction mode of square arrangement, and (c) is the pumping of arranged in regular hexagon shape
Take mode.
S4, high spatial resolution light field image are rebuild.Digital refocusing method is combined using light field data, generates high spatial
The refocusing image of resolution ratio.Then edge effect caused by mitigating splicing using Gaussian filter.
Further, for digital refocusing method.According to the depth of focus of scene, focus point and microlens array are calculated
The distance between z.The size for extracting block of pixels is calculated with reference to formula (1-7).Using in extraction, joining image-forming unit in S3
The method of block of pixels synthesizes the imaging results at rendering plane.
Further, for gaussian filtering process.To mitigate the edge effect that splicing generates, [d/m] × [d/ is utilized
M] size, average be 6 Gaussian filter to entire image carry out two-dimensional convolution processing, inhibit splicing tape unsmooth effect
Fruit.
Claims (6)
1. a kind of high spatial resolution optical field acquisition device, special including main lens, microlens array and compressed encoding sensor
Sign is:The microlens array is positioned between main lens and compressed encoding sensor, and microlens array is parallel to compression
Perpendicular to main lens optical axis, the photosurface of compressed encoding sensor can be completely covered for code sensor;It is wherein each micro-
Mirror can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies;Target scene
It is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.
2. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The microlens array
The arrangement of middle lenticule includes but are not limited to square and the interlacing of adjacent lines lenticule optical center alignment using linescan method
The arranged in regular hexagon shape of lenticule optical center alignment.
3. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The microlens array
In the Airy spot diameter of single lenticule be no more than the length of twice imaging sensor pixel most short side, the point range of single lenticule
Figure scope is no more than its Airy spot diameter.
4. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The compressed encoding passes
Sensor uses random convolution cmos image sensor, and the physical resolution of sensor is P*Q;Utilize what is stored in shift register
Random convolutional calculation is performed pixel-by-pixel on pseudo-random sequence control CMOS, is then realized using row, column selection logic unit random
Position samples, and obtains compression sampling data;Element number in the pseudo-random sequence cycle period is more than P*Q.
5. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that:The lenticule optical center
Place plane is to the vector distance z of main lens focal plane>0, plane where compressed encoding sensor photosurface to lenticule optical center
Distance g be more than single lenticule focal length f, 1/g+1/z=1/f;Main lens focal length F, the main lens clear aperature D of the present invention
Meet D/ (F+z+g)=d/g with lenslet diameter d.
6. a kind of image generating method of high spatial resolution optical field acquisition device described in claim 1, it is characterised in that including
Following step:
1) by main lens, the optical dimensions of microlens array and compressed encoding sensor, location information in the case where being unified into picture frame
Parametric description is carried out, the cost function based on minimum re-projection error is built, and passes through global optimization method and solved;
2) the high resolution 2 d light field signal that angle is coupled with location information recovers, and step is as follows:
2.1) according to the parameterized model of imaging device internal reference, the centre coordinate of each lenticule imaging unit is asked for;Then carry
It learns from else's experience the compressed observation signal of overcompression code sensor, rotation processing is carried out to compression image, makes every row imaging unit
Centre coordinate is located at in one-row pixels;Geometry rectification finally is done to the twist distortion of image, makes observation signal and calculation matrix
Correspondence be consistent;
2.2) extraction acts on the pseudo-random sequence of random coded, with reference to the measurement square in Preudo-Random Sequences Generation compressed sensing
Battle array;The calculation matrix includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix;
2.3) the super-resolution light field two dimension letter coupled using BP methods, BPDN methods or TV methods reconstruction angle with location information
Number;
3) angular samples number is calculated according to camera parameter model;Then believed according to angle information in two-dimension light field signal and position
The coupled relation of breath calculates the spatial resolution size that each lenticule pixel of focal plane provides, and single from each imaging
The block of pixels that the same position of member extracts same size is spliced, and synthesizes the focal plane imaging result under a visual angle;Finally
The pixel decimation center of converter unit imaging, you can synthesize the two dimensional image under multiple visual angles, realize four-dimensional light field data structure;
4) digital refocusing method is used using light field data, generates the refocusing image of high spatial resolution.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711263820.XA CN108093237A (en) | 2017-12-05 | 2017-12-05 | High spatial resolution optical field acquisition device and image generating method |
CN201811439379.0A CN109413407B (en) | 2017-12-05 | 2018-11-29 | High spatial resolution light field acquisition device and image generation method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711263820.XA CN108093237A (en) | 2017-12-05 | 2017-12-05 | High spatial resolution optical field acquisition device and image generating method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN108093237A true CN108093237A (en) | 2018-05-29 |
Family
ID=62173756
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201711263820.XA Pending CN108093237A (en) | 2017-12-05 | 2017-12-05 | High spatial resolution optical field acquisition device and image generating method |
CN201811439379.0A Active CN109413407B (en) | 2017-12-05 | 2018-11-29 | High spatial resolution light field acquisition device and image generation method |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811439379.0A Active CN109413407B (en) | 2017-12-05 | 2018-11-29 | High spatial resolution light field acquisition device and image generation method |
Country Status (1)
Country | Link |
---|---|
CN (2) | CN108093237A (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109297925A (en) * | 2018-10-09 | 2019-02-01 | 天津大学 | A kind of Terahertz high-resolution fast imaging device based on splits' positions perception |
CN110928113A (en) * | 2019-12-03 | 2020-03-27 | 西北工业大学 | Light field acquisition device with variable spatial resolution |
CN111157500A (en) * | 2020-01-06 | 2020-05-15 | 南开大学 | Transient body imaging microscope system using light sheet crystal lattice array illumination |
CN111273050A (en) * | 2020-02-12 | 2020-06-12 | 清华大学 | Signal acquisition processing method and device |
CN111340888A (en) * | 2019-12-23 | 2020-06-26 | 首都师范大学 | Light field camera calibration method and system without white image |
CN111610634A (en) * | 2020-06-23 | 2020-09-01 | 京东方科技集团股份有限公司 | Display system based on four-dimensional light field and display method thereof |
CN113327211A (en) * | 2021-07-28 | 2021-08-31 | 清华大学 | Correction method and device for large-field-of-view high-resolution light field microscope system |
WO2021254110A1 (en) * | 2020-06-19 | 2021-12-23 | 京东方科技集团股份有限公司 | Image processing method, apparatus and device, and storage medium |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112396670B (en) * | 2019-08-15 | 2023-01-10 | 天津大学青岛海洋技术研究院 | Image reconstruction method for novel binary image sensor |
CN111598775B (en) * | 2020-04-26 | 2023-02-03 | 西安理工大学 | Light field video time domain super-resolution reconstruction method based on LSTM network |
CN112945141B (en) * | 2021-01-29 | 2023-03-14 | 中北大学 | Structured light rapid imaging method and system based on micro-lens array |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7962033B2 (en) * | 2008-01-23 | 2011-06-14 | Adobe Systems Incorporated | Methods and apparatus for full-resolution light-field capture and rendering |
CN102739945B (en) * | 2012-05-24 | 2014-09-03 | 上海理工大学 | Optical field imaging device and method |
EP3116216A4 (en) * | 2014-03-03 | 2017-02-15 | Panasonic Intellectual Property Management Co., Ltd. | Image pickup apparatus |
CN103871038B (en) * | 2014-03-06 | 2014-11-05 | 中国人民解放军国防科学技术大学 | Super-resolution omnidirectional image reconstruction method based on non-uniform measurement matrix |
CN104036489B (en) * | 2014-05-09 | 2017-10-03 | 北京工业大学 | A kind of light filed acquisition method |
CN105704371B (en) * | 2016-01-25 | 2019-04-26 | 深圳市未来媒体技术研究院 | A kind of light field refocusing method |
CN106303228B (en) * | 2016-08-04 | 2019-09-13 | 深圳市未来媒体技术研究院 | A kind of rendering method and system of focus type light-field camera |
-
2017
- 2017-12-05 CN CN201711263820.XA patent/CN108093237A/en active Pending
-
2018
- 2018-11-29 CN CN201811439379.0A patent/CN109413407B/en active Active
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109297925A (en) * | 2018-10-09 | 2019-02-01 | 天津大学 | A kind of Terahertz high-resolution fast imaging device based on splits' positions perception |
CN110928113A (en) * | 2019-12-03 | 2020-03-27 | 西北工业大学 | Light field acquisition device with variable spatial resolution |
CN110928113B (en) * | 2019-12-03 | 2021-10-08 | 西北工业大学 | Light field acquisition device with variable spatial resolution |
WO2021129437A1 (en) * | 2019-12-23 | 2021-07-01 | 首都师范大学 | Method and system for light calibration field camera without requiring white image |
CN111340888A (en) * | 2019-12-23 | 2020-06-26 | 首都师范大学 | Light field camera calibration method and system without white image |
CN111340888B (en) * | 2019-12-23 | 2020-10-23 | 首都师范大学 | Light field camera calibration method and system without white image |
CN111157500A (en) * | 2020-01-06 | 2020-05-15 | 南开大学 | Transient body imaging microscope system using light sheet crystal lattice array illumination |
CN111273050B (en) * | 2020-02-12 | 2022-05-20 | 清华大学 | Signal acquisition processing method and device |
CN111273050A (en) * | 2020-02-12 | 2020-06-12 | 清华大学 | Signal acquisition processing method and device |
WO2021254110A1 (en) * | 2020-06-19 | 2021-12-23 | 京东方科技集团股份有限公司 | Image processing method, apparatus and device, and storage medium |
US11997397B2 (en) | 2020-06-19 | 2024-05-28 | Beijing Boe Optoelectronics Technology Co., Ltd. | Method, apparatus, and device for processing images, and storage medium |
CN111610634A (en) * | 2020-06-23 | 2020-09-01 | 京东方科技集团股份有限公司 | Display system based on four-dimensional light field and display method thereof |
CN111610634B (en) * | 2020-06-23 | 2022-05-27 | 京东方科技集团股份有限公司 | Display system based on four-dimensional light field and display method thereof |
US11789287B2 (en) | 2020-06-23 | 2023-10-17 | Beijing Boe Optoelectronics Technology Co., Ltd. | Display system based on four-dimensional light field, and display method therefor |
CN113327211A (en) * | 2021-07-28 | 2021-08-31 | 清华大学 | Correction method and device for large-field-of-view high-resolution light field microscope system |
CN113327211B (en) * | 2021-07-28 | 2021-11-02 | 清华大学 | Correction method and device for large-field-of-view high-resolution light field microscope system |
WO2023005671A1 (en) * | 2021-07-28 | 2023-02-02 | 清华大学 | Correction method and apparatus for large-field-of-view high-resolution light field microscopic system |
Also Published As
Publication number | Publication date |
---|---|
CN109413407A (en) | 2019-03-01 |
CN109413407B (en) | 2020-08-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108093237A (en) | High spatial resolution optical field acquisition device and image generating method | |
Wu et al. | Light field image processing: An overview | |
Wang et al. | End-to-end view synthesis for light field imaging with pseudo 4DCNN | |
Farrugia et al. | Light field super-resolution using a low-rank prior and deep convolutional neural networks | |
Venkataraman et al. | Picam: An ultra-thin high performance monolithic camera array | |
CN102959945B (en) | The method and system of virtual output image is produced according to the data obtained by the array of image-capturing apparatus | |
US8290358B1 (en) | Methods and apparatus for light-field imaging | |
WO2020117657A1 (en) | Enhancing performance capture with real-time neural rendering | |
Ding et al. | Dynamic fluid surface acquisition using a camera array | |
Wang et al. | High-fidelity view synthesis for light field imaging with extended pseudo 4DCNN | |
Dąbała et al. | Efficient Multi‐image Correspondences for On‐line Light Field Video Processing | |
Schwarz et al. | A weighted optimization approach to time-of-flight sensor fusion | |
KR20160106045A (en) | A light field processing method | |
CN108364309B (en) | Space light field recovery method based on handheld light field camera | |
CN110662014B (en) | Light field camera four-dimensional data large depth-of-field three-dimensional display method | |
US10762612B2 (en) | Method and an apparatus for generating data representative of a pixel beam | |
CN1959341A (en) | Optical imaging distance measuring device for single-aperture multiple imaging | |
CN109118544A (en) | Synthetic aperture imaging method based on perspective transform | |
Vadathya et al. | Learning light field reconstruction from a single coded image | |
US10909704B2 (en) | Apparatus and a method for generating data representing a pixel beam | |
Neumann et al. | Eyes from eyes: analysis of camera design using plenoptic video geometry | |
Bonatto et al. | Multiview from micro-lens image of multi-focused plenoptic camera | |
CN116402908A (en) | Dense light field image reconstruction method based on heterogeneous imaging | |
Tateishi et al. | Time-multiplexed coded aperture and coded focal stack-comparative study on snapshot compressive light field imaging | |
CN115950534A (en) | Focus-dividing plane type high-luminous-flux bionic multispectral polarization imaging detector |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WD01 | Invention patent application deemed withdrawn after publication | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180529 |