CN108093237A - High spatial resolution optical field acquisition device and image generating method - Google Patents

Abstract

The present invention provides a high spatial resolution light field acquisition device and an image generation method. The microlens array is placed between the main lens and the compression encoding sensor, the microlens array is parallel to the compression encoding sensor and perpendicular to the optical axis of the main lens. The photosensitive surface of the compression coding sensor is completely covered; each microlens can focus imaging, and the imaging areas of different microlenses do not overlap each other on the compression coding sensor plane; the real image presented by the target scene through the main lens is re- Collected and recorded by compression-encoded sensors. The invention obtains light field data with high spatial resolution without reducing the angular resolution and light flux of the light field, which can reduce the amount of light field data, relieve storage pressure, and promote light field video shooting and network storage and transmission of light field data application development.

Description

High spatial resolution light field acquisition device and image generation method

technical field

The invention relates to the fields of computer vision, computational photography and optical engineering, in particular to a light field acquisition device and a high-resolution light field image generation method using a compression coding sensor combined with a microlens array.

Background technique

The theory of light field imaging has brought revolutionary progress to the field of digital imaging. Different from the projection theory model of traditional imaging that only records the light intensity distribution on the sensor plane, the light field imaging model uses a novel optical system to correspond the light in the scene to the sensor pixels according to a given relationship. The light field acquisition device can simultaneously collect the position and angle information of light, and on this basis, generate novel images through digital processing. However, compared with traditional imaging techniques, light field imaging techniques also have certain limitations. In the case of certain sensor pixels, there is a problem of resolution compromise in light field imaging technology. In addition, due to the huge amount of light field data, it is not conducive to large-scale storage and network transmission. This has become a bottleneck problem that limits the further widespread application of light field cameras and computational photography.

In order to obtain high-resolution light field images, existing methods mainly include multiplexing methods and computational methods. The 128-camera array designed by Stanford University is an embodiment of the spatial multiplexing method. The annular aperture camera designed by the Massachusetts Institute of Technology is realized by the time multiplexing method. The existing methods of frequency domain multiplexing have the disadvantage of low signal-to-noise ratio of light field images, and pay less attention to the improvement of the angular resolution of light field. The multiplexing method is to convert the trade-off between light field position and angular resolution into a trade-off between time, space or light flux, but the corresponding acquisition method has the disadvantages of reduced time resolution, increased acquisition equipment scale, or light field image signal. The disadvantages such as low noise ratio are difficult to implement in applications. Most of the calculation methods need to accurately estimate the internal and external parameters, or need the sub-pixel-level scene geometry as a priori, which is also difficult to achieve in practical applications. The existing devices that use coded plates to modulate light to increase resolution have the problem of reducing luminous flux. This results in a low signal-to-noise ratio of the acquired signal and poor quality of the reconstructed light field data.

Therefore, obtaining a high spatial resolution light field without reducing the angular resolution and luminous flux has become an urgent problem to be solved in the field of light field imaging. The high spatial resolution light field acquisition device proposed in the present invention can effectively solve this problem, and can be widely used in fields such as light field video shooting, light field data compression and transmission, and the like.

Contents of the invention

In order to overcome the deficiencies of the prior art, the present invention provides a light field acquisition device based on the combination of a random convolution CMOS sensor and a microlens array, which can use compressed sensing technology to acquire more light fields without reducing the angular resolution and luminous flux. Light field signals with multi-position information, and construct high spatial resolution light field data through a globally optimized reconstruction algorithm.

The technical solution adopted by the present invention to solve the technical problem is: a light field data acquisition device combining a microlens array and a compression encoding sensor, including a main lens, a microlens array and a compression encoding sensor.

The microlens array is placed between the main lens and the compression encoding sensor, the microlens array is parallel to the compression encoding sensor, perpendicular to the optical axis of the main lens, and can completely cover the photosensitive surface of the compression encoding sensor; wherein each microlens It can focus imaging, and the imaging areas of different microlenses do not overlap each other on the plane of the compression coding sensor; the real image of the target scene passing through the main lens is re-converged by the microlens array and then recorded by the compression coding sensor.

The microlenses in the microlens array are arranged in row and column, including but not limited to a regular quadrilateral arrangement in which the optical centers of the microlenses in adjacent rows are aligned, and a regular hexagonal arrangement in which the optical centers of the microlenses in alternate rows are aligned.

The diameter of the Airy disk of a single microlens in the microlens array is not more than twice the length of the shortest side of the image sensor pixel, and the range of the spot diagram of a single microlens is not more than the diameter of the Airy disk.

The compression coding sensor adopts a random convolution CMOS image sensor, and the physical resolution of the sensor is P*Q; utilize the pseudo-random sequence stored in the shift register to control the random convolution calculation pixel by pixel on the CMOS, and then use row and column The logic unit is selected to realize sampling at random positions to obtain compressed sampling data; the number of elements in the cycle period of the pseudo-random sequence is greater than P*Q.

The vector distance z>0 from the plane where the optical center of the microlens is located to the focal plane of the main lens, the distance g from the photosensitive surface of the compression encoding sensor to the plane where the optical center of the microlens is located is greater than the focal length f of a single microlens, 1/g+1/ z=1/f; the focal length F of the main lens, the clear aperture D of the main lens and the diameter d of the microlens of the present invention satisfy D/(F+z+g)=d/g.

The present invention also provides a method for reconstructing a high spatial resolution light field based on compressed sensing technology, comprising the following steps:

1) Parametrically describe the optical size and position information of the main lens, microlens array, and compression-encoded sensor under a unified imaging framework, construct a cost function based on minimizing the reprojection error, and solve it through a global optimization method;

2) High-resolution two-dimensional light field signal recovery coupled with angle and position information, the steps are as follows:

2.1) Calculate the center coordinates of each microlens imaging unit according to the parameterized model of the internal parameters of the imaging device; then extract the observation signal compressed by the compression coding sensor, and rotate the compressed image so that the center coordinates of each row of imaging units Located on the same row of pixels; finally, geometrically correct the distortion and deformation of the image, so that the corresponding relationship between the observed signal and the measurement matrix is consistent;

2.2) extract the pseudo-random sequence that acts on the random code, and combine the pseudo-random sequence to construct the measurement matrix in the compressed sensing; the measurement matrix includes but not limited to Gaussian random matrix, Bernoulli matrix, uniform distribution matrix;

2.3) Using BP method, BPDN method or TV method to reconstruct the super-resolution light field two-dimensional signal coupled with angle and position information;

3) Calculate the number of angle samples according to the parameterized model of the camera; then calculate the spatial resolution provided by each microlens pixel at the focal plane according to the coupling relationship between the angle information and the position information in the two-dimensional light field signal, and from each The pixel blocks of the same size are extracted from the same position of the two imaging units for splicing, and the focal plane imaging result under one viewing angle is synthesized; finally, the pixel extraction center of the imaging unit is transformed to synthesize two-dimensional images under multiple viewing angles, realizing a four-dimensional light field data construction;

4) A digital refocusing method is used using the light field data to generate a refocused image with high spatial resolution.

The beneficial effects of the present invention are: without reducing the angular resolution of the light field and the amount of light passing through, the light field data with high spatial resolution can be obtained, the amount of light field data can be reduced, the storage pressure can be relieved, and the shooting of light field video and light field data can be promoted. The development of network storage and dissemination applications.

Description of drawings

1 is a schematic diagram of the optical path of the light field acquisition device;

Fig. 2 is a structural diagram of the light field acquisition device;

Fig. 3 is a schematic diagram of the physical parameters of the microlens array;

Figure 4 is a schematic diagram of the calculation of the number of angle samples, where (a) is the calculation principle of the number of angle samples, (b) is the imaging and recording range of the microlens array arranged in a regular quadrilateral, and (c) is the microlens arranged in a regular hexagon Array imaging recording range;

5 is a schematic diagram of the imaging unit extraction and splicing method, wherein (a) is the splicing method of the imaging unit, (b) is the extraction method of the square arrangement, and (c) is the extraction method of the regular hexagonal arrangement;

Figure 6 shows the reconstruction route of high spatial resolution light field data.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, and the present invention includes but not limited to the following embodiments.

The present invention provides a light field data acquisition device combining a microlens array and a random convolution CMOS image sensor, comprising: a main lens, used to refract the object beam into the camera; a microlens array, used to separate the The angle and position information of the light on the mirror plane; a random convolution CMOS image sensing sensor for collecting encoded and compressed light field signals.

The acquisition device of the present invention places a microlens array between the main lens and the random convolution CMOS image sensor, the microlens array is parallel to the image sensor, and can completely cover the photosensitive surface of the image sensor.

The microlens array is placed behind the main lens (closer to the image side) and parallel to the plane of the main lens, so that the real image of the target scene passing through the main lens can be collected again by the microlens array and then recorded by the image sensor. Each microlens can be focused and imaged, and the imaging areas of different microlenses do not overlap each other on the sensor plane.

In the collection device of the present invention, the microlenses in the microlens array are arranged in rows and columns, including but not limited to a regular quadrilateral arrangement in which the optical centers of the microlenses in adjacent rows are aligned and a regular hexagonal arrangement in which the optical centers of the microlenses in alternate rows are aligned. The physical parameters of the microlens array meet the following constraints: the diameter of the Airy disk of a single microlens is not more than twice the length of the shortest side of the image sensor pixel, and the spot diagram range of a single microlens does not exceed the diameter of its Airy disk.

In the acquisition system of the present invention, the compression coding sensor generates a pseudo-random sequence and multiplies it with the sensor pixels to realize the compressed sensing of the signal. Specifically, a random convolution image sensor is used in this embodiment. Compression-encoded sensors of the present invention include, but are not limited to, such a compression sensor. The physical resolution of the sensor is P*Q (the number of pixels in each row and column of the sensor is P and Q respectively). First initialize the pseudo-random sequence, the number of elements in its cycle is greater than the number of sensor pixels P*Q. Then press the pseudo-random sequence into the shift register, use the shift register to control the CMOS to perform random convolution calculation pixel by pixel, and then use the row and column selection logic unit to realize random position sampling. Finally, the compressed sampling data is obtained, and its resolution is M (0<M<P*Q). The measurement matrix constructed by combining random convolution and downsampling needs to satisfy the RIP matrix property. The arrangement of the light angle and position information on the image sensor has a coupling relationship, and the compression and downsampling processing based on random convolution does not destroy the coupling between the light field angle information and position information.

The arrangement mode (Fig. 1) of each device of the acquisition system of the present invention is: the microlens array and the random convolution CMOS image sensor are located behind the imaging plane of the main lens, that is, the microlens array and the random convolution CMOS image sensor plane are all perpendicular to the main lens. The optical axis of the lens is distributed along the optical axis in turn. Since each microlens is focused and imaged, the distances between the imaging plane of the main lens, the microlens array, and the random convolution CMOS image sensor satisfy the Gaussian imaging formula, and the vector distance from the plane where the optical center of the microlens is to the focal plane of the main lens (object distance) z>0, the distance (image distance) g between the photosensitive surface of the image sensor and the plane where the optical center of the microlens is located and the focal length f of a single microlens satisfy g>f. The parameter selection of the imaging device of the present invention should be based on the requirements of formula (1-1).

D/(F+z+g)=d/g (1-1)

Among them, F is the focal length of the main lens, D is the diameter of the clear aperture of the main lens, and d is the diameter of the microlens. The parameter selection of the microlens array of the present invention should be based on the requirements of formula (1-2).

1/g+1/z=1/f (1-2)

The invention also provides a method for reconstructing a light field with high spatial resolution based on compressed sensing technology. The main links include: parameter calibration of the acquisition device, high-resolution two-dimensional light field signal recovery coupled with angle and position information, high spatial resolution four-dimensional light field data construction, and high spatial resolution light field image reconstruction. Said method comprises the following steps:

S1. Calibration of internal parameters of the acquisition device. The optical size and position information of each component in the imaging device are parametrically described under a unified imaging framework, and a cost function based on minimizing the reprojection error is constructed and solved by a global optimization method.

S2. High-resolution two-dimensional light field signal recovery coupled with angle and position information. Combined with the theory of compressed sensing, super-resolution reconstruction is performed on the compressed and encoded low-resolution two-dimensional light field signal. Including raw signal preprocessing based on camera parameters, measurement matrix construction based on compressed sensing theory, super-resolution light field two-dimensional signal reconstruction based on global optimal method.

S2.1. Raw signal preprocessing based on camera parameters. Firstly, according to the parameterized model of the internal parameters of the imaging device, the center coordinates of each microlens imaging unit are obtained; then the observation signal compressed by the compression coding sensor is extracted, and the compressed image is rotated so that the center coordinates of each row of imaging units are located at On the same row of pixels; finally, the geometric correction is made to the distortion of the image, so that the corresponding relationship between the observed signal and the measurement matrix is consistent.

S2.2. Construction of measurement matrix based on compressive sensing theory. The pseudo-random sequence that acts on the random code is extracted, and according to the working principle of image sensor compression coding, the measurement matrix in compressed sensing is constructed by combining the pseudo-random sequence. The measurement matrix constructed in the present invention includes but not limited to Gaussian random matrix, Bernoulli matrix, uniform distribution matrix and other compressed sensing measurement matrices suitable for hardware implementation.

S2.3. Two-dimensional signal reconstruction of super-resolution light field coupled with angle and position information. The reconstruction of super-resolution light field two-dimensional signal can use compressed sensing reconstruction algorithms such as BP (Basis Pursuit) method, BPDN (Basis Pursuit De-Noising) method, TV (Total Variation) method, and construct the optimal A compressed sensing reconstruction method uses observation signals and measurement matrices to reconstruct super-resolution light field two-dimensional signals.

S3. Construction of high spatial resolution 4D light field data. First, the number of angle samples is calculated according to the camera parameterization model; then, according to the coupling relationship between angle information and position information in the two-dimensional light field signal, the spatial resolution provided by each microlens pixel at the focal plane is calculated, and from each Pixel blocks of the same size are extracted from the same position of the imaging unit for splicing to synthesize the focal plane imaging result under one viewing angle; finally, the pixel extraction center of the imaging unit is transformed to synthesize two-dimensional images under multiple viewing angles to realize four-dimensional light field data Construct.

S4. High spatial resolution light field image reconstruction. A digital refocusing method is employed using light field data to generate refocused images with high spatial resolution.

The high spatial resolution light field acquisition device based on the compression coding sensor provided by the embodiment of the present invention includes three parts, as shown in Figure 1, a main lens 101, and a microlens array 103. In this embodiment, the compression coding sensor adopts A random convolution CMOS sensor 104 (FIG. 2). The schematic diagram of the optical path is shown in Figure 1. The scene light is refracted by the main lens 101 and then focused on the image plane 102 of the main lens. The light passing through the image plane begins to separate, refracts through the small lenses in the microlens array 103, and converges on the image sensor 104. .

Specifically, as shown in FIG. 3 , the microlens array 102 is arranged in a regular hexagon. A single microlens 201 has a diameter d of 0.3 mm and a focal length of 2.726 mm, and is a plano-convex lens. The microlens array 103 completely covers the photosensitive surface of the random convolution image sensor 104, and consists of no less than 120 microlenses in the horizontal direction and no less than 92 microlenses in the vertical direction arranged in a regular hexagon. The distance g between the random convolution CMOS image sensor 104 and the microlens array satisfies formula (1-1). The random convolution CMOS passive pixel sensor is a standard P*Q binary array, and the effective area of a single pixel of the sensor is 30μm*30μm. Each pixel contains a photodiode with a maximum current of 200µA.

The high spatial resolution light field acquisition method proposed by the present invention, as shown in Figure 6, includes the following steps:

S1. Calibration of internal parameters of the acquisition device. The present invention parametrically describes the optical size and position information of each component under a unified imaging framework, combines the light field biplane model, and adopts an internal parameter calibration method that satisfies the mapping relationship between discrete sampling and continuous expression of the light field for calibration. During the calibration process, the acquisition device transforms N postures, and n _c features can be extracted on the calibration board. Use the minimized error formula (1-3) to solve the internal parameter matrix H, camera attitude T and distortion parameter d.

Where ||·|| ^pt-ray is the reprojection error of the ray, that is, the error between the decoded ray and the prior point of the calibration plate, is the reprojection ray, T _n is the camera extrinsic parameter in each pose, P _c is the prior point of the calibration board, c takes a value from 1 to n _c , n takes a value from 1 to N, and s takes a value from 1 to P, t take values from 1 to Q.

S2. High-resolution two-dimensional light field signal recovery coupled with angle and position information. In this embodiment, the random convolution sensor is used to compress and encode the original signal; then the compressed signal is preprocessed using camera parameters; finally, the measurement matrix Φ is constructed according to the working principle of the random convolution sensor, and the observation signal y and the measurement matrix Φ are used to reconstruct Super-resolution of light-field 2D signals.

S2.1 Raw signal preprocessing based on camera parameters. According to the parametric model of the internal parameters of the imaging device, the center coordinates of each microlens imaging unit are obtained, and the geometric correction is made to the original signal, so that the corresponding relationship between the observed signal and the measurement matrix is consistent.

Further, for calculating the central coordinates of the microlens imaging unit. First, take a white-exposed image; then use the maximum variance algorithm between classes to binarize the white-exposure image; finally use the region growing algorithm to segment the imaging unit, and obtain the average center coordinates of each unit (Formula 1- 4).

Where c _i is the center coordinate of an imaging unit, (xi _,j ,y _i,j ) is a point in an imaging unit in the binarization map, m _c is the number of pixels in an imaging unit, j's Takes values from 1 to m _c .

Further, geometric correction of the original signal. First extract the observation signal y compressed by the compression coding sensor; then extract the center coordinates of each imaging unit, use the least square method to calculate the slope of the center of each row of units, and take the average value; then rotate the original data y to make each row The center coordinates of unit imaging are basically located on the same row of pixels; finally, geometric correction is made to the deformation of the image such as distortion and shear, and vignetting is removed for the edge imaging unit.

S2.2 Construction of measurement matrix based on compressive sensing theory. In this embodiment, the random winding sensor is used to compress and encode the light field signal. Therefore, the expression of the uncompressed signal ^x∈RPQ*1 after being processed by the random convolution filter a is shown in formula 1-5.

Where r(i)∈{1,...,M}, i ranges from 1 to M, j ranges from 1 to P*Q, the value of r(i) is randomly selected in the interval, Φ is determined by random The measurement matrix formed by the convolution filter a. In this embodiment, the value of the filter a is selected as ±1, and the form of the measurement matrix Φ is as follows. Therefore, the measurement matrix is a Bernoulli matrix with a value of ±1, which satisfies the property of the RIP matrix and can be used for compressed sensing reconstruction.

S2.3. A globally optimal super-resolution light field two-dimensional signal reconstruction method. In this embodiment, a reconstruction algorithm combining L1 norm and Total Variation is used. Taking the observation signal y and the measurement matrix Φ as input, the two-dimensional super-resolution light field signal is reconstructed using the method shown in formula (1-7). Where ||·|| _TV is the gradient operator, ε is the noise error.

S3. Construction of high spatial resolution 4D light field data. In this embodiment, according to the coupling relationship between the angle and the position information in the two-dimensional signal, by extracting and splicing the pixel blocks in the imaging unit, two-dimensional imaging of the light field on the focal plane under one viewing angle is obtained. By changing the pixel extraction center of unit imaging, multi-view images on the focal plane can be obtained, and thus four-dimensional light field data can be decoded. The size of the pixel block is related to the number of angle samples.

Further, for calculating the angle sampling number m. The angle sampling number m is related to the distance g from the photosensitive surface of the image sensor to the center of the microlens array, and the distance z from the center of the microlens array to the focal plane of the main lens (Figure 4), and is proportional to |z|/g, then the angle sampling number m is shown in formula 1-7.

m=k×|z|/g (1-8)

Among them, k is related to the arrangement and spacing of the microlenses, and the size of the imaging area behind the microlenses. In the present invention, the microlens array is arranged in a regular hexagon, so Figure 4(a) is the calculation principle of the angle sampling number, Figure 4(b) is the imaging and recording range of the microlens array arranged in a regular quadrilateral, and Figure 4(c) is the imaging and recording range of the microlens array arranged in a regular hexagon. Where r is the radius of the rear image of a microlens.

Further, for extracting and splicing the pixel blocks in the imaging unit. For each imaging unit, intercept pixel blocks with the same distance d/m from the center, and stitch them together according to the arrangement of the imaging units. The principles of image splicing and extraction are shown in Figure 5, (a) is the splicing method of imaging units, (b) is the extraction method of square arrangement, and (c) is the extraction method of regular hexagonal arrangement.

S4. High spatial resolution light field image reconstruction. Using light field data combined with a digital refocusing method, a refocused image with high spatial resolution is generated. Gaussian filters are then used to mitigate edge effects caused by splicing.

Further, for the digital refocusing method. According to the depth of focus of the scene, the distance z between the focus point and the microlens array is calculated. Combining formula (1-7) to calculate the size of the extracted pixel block. Using the method of extracting and splicing the pixel blocks in the imaging unit in S3, the imaging result at the rendering plane is synthesized.

Further, for Gaussian filtering processing. In order to reduce the edge effect caused by splicing, a Gaussian filter with a size of [d/m]×[d/m] and an average value of 6 is used to perform two-dimensional convolution processing on the entire image to suppress the non-smooth effect caused by splicing.

Claims (6)

1. a kind of high spatial resolution optical field acquisition device, special including main lens, microlens array and compressed encoding sensor Sign is：The microlens array is positioned between main lens and compressed encoding sensor, and microlens array is parallel to compression Perpendicular to main lens optical axis, the photosurface of compressed encoding sensor can be completely covered for code sensor；It is wherein each micro- Mirror can focal imaging, and the imaging region of different lenticule is in compressed encoding sensor plane non-overlapping copies；Target scene It is recorded after the real image that main lens are in converges again via microlens array by compressed encoding sensor.

2. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that：The microlens array The arrangement of middle lenticule includes but are not limited to square and the interlacing of adjacent lines lenticule optical center alignment using linescan method The arranged in regular hexagon shape of lenticule optical center alignment.

3. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that：The microlens array In the Airy spot diameter of single lenticule be no more than the length of twice imaging sensor pixel most short side, the point range of single lenticule Figure scope is no more than its Airy spot diameter.

4. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that：The compressed encoding passes Sensor uses random convolution cmos image sensor, and the physical resolution of sensor is P*Q；Utilize what is stored in shift register Random convolutional calculation is performed pixel-by-pixel on pseudo-random sequence control CMOS, is then realized using row, column selection logic unit random Position samples, and obtains compression sampling data；Element number in the pseudo-random sequence cycle period is more than P*Q.

5. high spatial resolution optical field acquisition device according to claim 1, it is characterised in that：The lenticule optical center Place plane is to the vector distance z of main lens focal plane>0, plane where compressed encoding sensor photosurface to lenticule optical center Distance g be more than single lenticule focal length f, 1/g+1/z=1/f；Main lens focal length F, the main lens clear aperature D of the present invention Meet D/ (F+z+g)=d/g with lenslet diameter d.

6. a kind of image generating method of high spatial resolution optical field acquisition device described in claim 1, it is characterised in that including Following step：

1) by main lens, the optical dimensions of microlens array and compressed encoding sensor, location information in the case where being unified into picture frame Parametric description is carried out, the cost function based on minimum re-projection error is built, and passes through global optimization method and solved；

2) the high resolution 2 d light field signal that angle is coupled with location information recovers, and step is as follows：

2.1) according to the parameterized model of imaging device internal reference, the centre coordinate of each lenticule imaging unit is asked for；Then carry It learns from else's experience the compressed observation signal of overcompression code sensor, rotation processing is carried out to compression image, makes every row imaging unit Centre coordinate is located at in one-row pixels；Geometry rectification finally is done to the twist distortion of image, makes observation signal and calculation matrix Correspondence be consistent；

2.2) extraction acts on the pseudo-random sequence of random coded, with reference to the measurement square in Preudo-Random Sequences Generation compressed sensing Battle array；The calculation matrix includes but are not limited to gaussian random matrix, Bernoulli Jacob's matrix, is uniformly distributed matrix；

2.3) the super-resolution light field two dimension letter coupled using BP methods, BPDN methods or TV methods reconstruction angle with location information Number；

3) angular samples number is calculated according to camera parameter model；Then believed according to angle information in two-dimension light field signal and position The coupled relation of breath calculates the spatial resolution size that each lenticule pixel of focal plane provides, and single from each imaging The block of pixels that the same position of member extracts same size is spliced, and synthesizes the focal plane imaging result under a visual angle；Finally The pixel decimation center of converter unit imaging, you can synthesize the two dimensional image under multiple visual angles, realize four-dimensional light field data structure；

4) digital refocusing method is used using light field data, generates the refocusing image of high spatial resolution.