CN111182191A - Wide-field high-resolution camera shooting equipment and method based on aberration compensation calculation - Google Patents
Wide-field high-resolution camera shooting equipment and method based on aberration compensation calculation Download PDFInfo
- Publication number
- CN111182191A CN111182191A CN202010133526.2A CN202010133526A CN111182191A CN 111182191 A CN111182191 A CN 111182191A CN 202010133526 A CN202010133526 A CN 202010133526A CN 111182191 A CN111182191 A CN 111182191A
- Authority
- CN
- China
- Prior art keywords
- scanning
- resolution
- lens
- image
- wide
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Abstract
The invention provides a wide-field high-resolution camera shooting device and method based on aberration compensation calculation. A wide-field high-resolution image pickup apparatus based on calculation aberration compensation according to the present invention includes: the scanning micro-lens array comprises an optical input module, a scanning micro-lens array module and an imaging module; wherein the optical input module comprises an optical lens; the scanning micro-lens array module comprises a displacement device and a micro-lens array; the imaging module comprises an image sensor; the optical input module shoots a scene through any optical lens to collect optical information, the scanning micro-lens array module drives the micro-lens array to perform periodic moving scanning through the displacement device, and the optical information collected by the optical lens is further transmitted to an image sensor in the imaging module.
Description
Technical Field
The invention relates to the fields of light field imaging, phase space imaging and macroscopic optical imaging, in particular to wide-field high-resolution camera equipment and a method based on aberration compensation.
Background
In recent years, ultra-high pixel count imaging has gradually entered the field of view of people. Along with popularization and diffusion of industries or equipment such as machine vision, unmanned planes, high-definition monitoring systems and the like, the requirements of people for high-resolution and large-pixel-number imaging are increasingly obvious.
The three limiting conditions for determining the imaging quality or the imaging resolution of the imaging system are as follows: one is the sampling rate of the image sensor: the popular image sensors are mainly divided into two types of CCD and CMOS, the number of pixels is increased, and the small size of the pixels is beneficial to the generation of images with higher resolution; second, the optical diffraction limit of the system: according to Rayleigh resolution criterion, the numerical aperture of each lens in the imaging system determines the resolution capability of the system; and thirdly, aberration, which is commonly existed in natural environment, such as atmospheric scattering and the like. Meanwhile, aberration also exists in the lens, and because the lens belongs to an artificially ground device, deviation from an ideal lens model in theoretical optics exists. On the other hand, as the size of the lens gradually increases, paraxial optical theory in an ideal optical system is no longer applicable, and the trajectory of off-axis rays is difficult to predict as easily as paraxial rays. The above three points all limit the imaging capability of the system, and further, the development of billion pixel imaging is stopped.
Thanks to the industrial development, the diffraction limit which can be reached by the optical system can meet the requirement of macroscopic scene shooting, and simultaneously, the technology is enough to manufacture a high-resolution image sensor with large area array and small pixel size. It is the presence of aberrations that discourages further development of these processes. In the physical model, as the size of a single lens increases, the aberration gradually increases. Therefore, as the number of pixels increases, the number of effective pixels is limited to a limited scale. Under such conditions, no matter how the number of pixels of the image sensor is increased or the numerical aperture of the lens is increased, the image is difficult to further improve the resolution and the definition.
Under such conditions, the development of billions of pixels has gone through two stages. In the first stage, the size of an optical aperture is reduced, the existence of aberration is reduced, and the limitation of the number of effective pixels of the original common camera can be broken through. However, this solution leads to a decrease in the amount of light transmitted, an increase in the exposure time, and a significant decrease in the signal-to-noise ratio. And in the second stage, billion pixel imaging is formed by shooting and splicing multiple lenses. The scheme reduces the corresponding increasing amplitude of the aberration along with the increase of the size of the single lens by increasing the number of the lenses, breaks through the aberration limitation better than the mode of reducing the aperture, and achieves better imaging effect. However, neither of the two solutions actually solves the problem of aberration, and the best imaging effect that can be achieved is far less than the optical diffraction limit performance.
Disclosure of Invention
The invention aims to solve the technical problem of providing a wide-field high-resolution shooting technology based on aberration compensation calculation aiming at the defects in the prior art. The system simultaneously acquires the angle information and the spatial information of light rays through the micro lens array, further fuses images of the same scene at different angles in a phase space, and corrects aberration. On the other hand, the system also has the function of obtaining an image with higher resolution than the image sensor, and an image with a higher number of pixels is obtained by shooting with a camera with a lower number of pixels.
According to the present invention, there is provided a wide-field high-resolution image pickup apparatus based on calculation aberration compensation comprising: the scanning micro-lens array comprises an optical input module, a scanning micro-lens array module and an imaging module; wherein the optical input module comprises an optical lens; the scanning micro-lens array module comprises a displacement device and a micro-lens array; the imaging module comprises an image sensor; the optical input module shoots a scene through any optical lens to collect optical information, the scanning micro-lens array module drives the micro-lens array to perform periodic moving scanning through the displacement device, and the optical information collected by the optical lens is further transmitted to an image sensor in the imaging module.
Preferably, the microlens array of the scanning microlens array module collects spatial information as well as angular information through scanning for the calculation of the digitally calculated aberration compensation algorithm and image reconstruction algorithm of the image sensor.
Preferably, the optical lens is in focus or in a predetermined defocus range, the microlens array is placed at a back focal plane of the optical lens, and the image sensor is within a predetermined axial range of a back focal point of the microlens array.
Preferably, the micro lens array is arranged at an image point of the lens, and obtains the spatial information of the illumination collected by the lens and the angle information of the light beam.
Preferably, the scanning speed of the scanning microlens module matches the acquisition speed of the image sensor in the imaging module.
Preferably, the size of the microlens array just covers the area of the effective collection area of the image sensor.
Preferably, the numerical aperture of the optical lens in the optical input module matches the numerical aperture of each microlens in the microlens array.
Preferably, in the periodic moving scanning, each scanning period is based on the number of pixels covered by a single micro lens, and the image sensor acquires a shot image corresponding to each movement in one period so as to correct aberration through a digital algorithm and reconstruct a clear image.
Preferably, the wide-field high-resolution image pickup apparatus based on the calculation aberration compensation further includes: and the processing unit is used for rearranging the light field data into phase space images with different light incidence angles through a digital self-adaptive algorithm for correcting aberration, and reconstructing to obtain an aberration-free image or video with the resolution greater than or equal to that of the original image by combining the calculated ideal point diffusion function or the point diffusion function obtained by direct calibration.
According to the present invention, there is also provided a wide-field high-resolution imaging method based on calculation of aberration compensation, comprising:
establishing a simulation model of the wide-field high-resolution camera equipment based on the calculated aberration compensation, designing the size of a micro-lens array according to system requirements, and simultaneously performing simulation calculation on a point spread image of an object space and an ideal point spread function of a phase space;
shooting light field data containing one or more scanning periods by using the simulation model parameters;
through a digital self-adaptive algorithm, the light field data is rearranged into phase space images with different light incidence angles for correcting aberration, and the phase space images are combined with the calculated ideal point diffusion function or the point diffusion function obtained by direct calibration to reconstruct and obtain an aberration-free image or video with the resolution greater than or equal to that of the original image.
Preferably, when the light field data containing one or more scanning periods is shot, after the shot scene is selected, the optical input module collects light beams of the target scene, adjusts the frame rate of the cameras in the scanning microlens array module and the imaging module and the scanning speed of the microlens array, and then images are formed through the image sensor.
The invention provides a wide-field high-resolution shooting technology based on aberration compensation calculation. Unlike ordinary light field cameras, the present invention compensates for spatial information lost to collecting angular information by scanning of the microlens array. Through the acquisition of information at different angles and the digital image processing, imaging patterns at different light angles are obtained. And then eliminating the aberration by calculating an aberration compensation algorithm, and fusing the images into an image without the aberration.
The invention has another advantage of breaking through the limitation of the number of pixels of the image sensor and achieving the number of pixels corresponding to the diffraction limit resolution. By adjusting the moving step length in the scanning process of the micro lens array and processing the step length by an algorithm, the image or the video with higher resolution and more pixels than the original image or the video can be obtained. Also, high resolution images or videos reconstructed in this way are aberration-free.
The invention improves the imaging performance of a macro camera system, so that a common camera breaks through the limitation of aberration and achieves the diffraction limit resolution.
Drawings
A more complete understanding of the present invention, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:
fig. 1 schematically shows a system block diagram of a wide-field high-resolution image pickup apparatus based on calculating aberration compensation according to a preferred embodiment of the present invention.
Fig. 2 schematically shows a schematic configuration of a wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to a preferred embodiment of the present invention.
Fig. 3 schematically shows a flow chart of a wide field of view high resolution imaging method based on calculating aberration compensation according to a preferred embodiment of the invention.
FIG. 4 is a diagram illustrating an effect of implementing a function of removing aberration according to an embodiment of the present invention.
Fig. 5 is a diagram illustrating the effect of increasing the imaging resolution according to an embodiment of the present invention.
It is to be noted, however, that the appended drawings illustrate rather than limit the invention. It is noted that the drawings representing structures may not be drawn to scale. Also, in the drawings, the same or similar elements are denoted by the same or similar reference numerals.
Detailed Description
In order that the present disclosure may be more clearly and readily understood, reference will now be made in detail to the present disclosure as illustrated in the accompanying drawings.
< first embodiment >
Fig. 1 schematically shows a system block diagram of a wide-field high-resolution image pickup apparatus based on calculating aberration compensation according to a preferred embodiment of the present invention. Fig. 2 schematically shows a schematic configuration of a wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to a preferred embodiment of the present invention.
As shown in fig. 1 and 2, the wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to the preferred embodiment of the present invention includes: an optical input module 100, a scanning microlens array module 200, and an imaging module 300; the optical input module 100 includes an optical lens 110; the scanning microlens array module 200 includes a displacement device 210 and a microlens array 220; the imaging module 300 includes an image sensor 310.
Specifically, the optical input module 100 captures a scene through an arbitrary optical lens 110, and collects optical information. The scanning microlens array module 200 drives the microlens array 220 to perform a periodic moving scan by the displacement device 210, and further transmits the optical information collected by the optical lens 110 to the image sensor 310 in the imaging module 300.
Further, the wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to the preferred embodiment of the present invention further includes: and the processing unit is used for rearranging the light field data into phase space images with different light incidence angles through a digital self-adaptive algorithm for correcting aberration, and reconstructing to obtain an aberration-free image or video with the resolution greater than or equal to that of the original image by combining the calculated ideal point diffusion function or the point diffusion function obtained by direct calibration.
For example, optical lenses include, but are not limited to: a long-focus lens, a short-focus lens, a middle-focus lens, a wide-angle lens and a fisheye lens. And (3) collecting a plurality of light field data by combining a micro lens array for moving scanning, and processing and fusing by using a later-stage digital algorithm to obtain a high-resolution aberration-free image.
For example, the optical input module may acquire any scene and under a common lighting condition by using an optical lens.
For example, the displacement device and the microlens array of the scanning microlens array module, the microlens array with the size capable of just covering the effective collection area of the image sensor is selected and placed at the front focal point of the image sensor. Each scan displacement is in the order of pixel length, depending on resolution requirements.
The acquisition rate of the image sensor of the imaging module needs to be matched with the moving rate of the micro lens. The number of pixels of the image sensor is selected based on the final image resolution requirements and the optical diffraction limit of the system.
Further, in one embodiment of the present invention, sufficient spatial information as well as angular information is collected by the scanning of the microlens array by the scanning microlens array module 200 for use in conjunction with digitally computing aberration compensation algorithms and image reconstruction algorithms. Preferably, the micro lens array is arranged at an image point of the lens, and obtains the spatial information of the illumination collected by the lens and the angle information of the light beam. By constant scanning of the microlens array, the spatial resolution can be compensated for because of the collection of angular information and sacrifice.
Preferably, the Numerical Aperture (NA) of the optical lens in the optical input module needs to match the Numerical Aperture (NA) of each microlens in the microlens array.
In the scanning microlens array module, the displacement device is not particularly limited, as long as the microlens can be driven to move at a required speed and step length. The impulse response function of the optical system can be obtained by simulating a Point Spread Function (PSF) under an aberration-free condition or by acquiring the Point Spread Function (PSF) in a calibration manner.
The optical diffraction limit of the system determines the highest resolution at which an image or video can be reconstructed.
The microlens array is translated in a single pixel step length to reconstruct an aberration-free image or video with the resolution equal to the number of pixels of the image sensor; the microlens array is translated in sub-pixel step size to reconstruct the image with resolution equal to the number N of pixels of the image sensor2Multiple resolution images or videos, where the value of N depends on the step size of the sub-pixel translation. Specifically, each scanning period is based on the number of pixels covered by a single microlens, and the image sensor needs to acquire a shot image corresponding to each movement in one period, so that aberration is corrected through a digital algorithm, and a clear image is reconstructed. If the diffraction limit performance of the system is to be achieved, an image sensor with dense sampling and large pixel number can be selected. When the pixel size is equal to 1/N of the diffraction limit resolution, an image with the resolution degree equal to the diffraction limit resolution, namely N, can be obtained by a method of scanning the micro-lens array by sub-pixel step sizes2The image or video with the number of times of the pixel number of the image sensor is not interfered by aberration, thereby achieving the purpose ofHigh pixel order of magnitude.
Further, in one embodiment of the present invention, the step size of the scanning by the scanning microlens array module 200 is determined by the resolution desired to be obtained finally, and can be calculated by simulation process simulation.
Further, in one embodiment of the present invention, the image sensor 310 in the imaging module 300 has no specific parameter limitations, including but not limited to any of: the image sensors with parameters such as the number of pixels, the size of the pixels, the shutter mode, the frame rate of acquisition, etc. can be used in the system. Different parametric image sensors correspond to different imaging qualities.
Further, in one embodiment of the present invention, the reconstruction of the image or video is an increase in the aberration, resolution of the image or video captured by the combination of the existing optical module and the imaging module.
Further, in one embodiment of the present invention, a resolution greater than the pixel size of image sensor 310, and a number of images or videos greater than the pixel size of image sensor 310, may be obtained by adjusting the step size of the scanning movement.
Further, in an embodiment of the present invention, the optical lens 110 does not need to be in focus, and does not affect the reconstruction result within a certain defocus range. The microlens array 220 needs to be placed at the back focal plane of the optical lens 110. The image sensor 310 does not have to be at the back focus of the microlens array 220, and does not affect the reconstruction result in a certain axial range.
Further, in one embodiment of the present invention, various parameters of the system can be simulated by the simulation model, but the actual parameters are subject to the real optical parameters of the system.
Specifically, for example, the optical input module selects macro imaging lenses with different focal lengths according to shooting requirements; the scanning micro-lens module consists of a displacement device and a micro-lens array, the micro-lens array is driven by the displacement device to move in a two-dimensional scale, and the micro-lens array is placed at the back focal plane of the imaging lens; and the imaging module is used for placing the image sensor at the back focal plane of the micro lens array.
In the embodiment of the present invention, the scanning speed of the scanning microlens module needs to be matched with the acquisition speed of the image sensor in the imaging module, and images formed by microlens arrays at different positions in the scanning process are acquired in real time.
In the embodiment of the invention, a simulation model is established, a micro-lens array with proper parameters is selected by simulating the imaging effect of an optical imaging system, ideal point diffusion patterns of object spaces at different positions are obtained, and the point diffusion patterns of a phase space are calculated. After the simulation design is finished, the size of the image sensor is selected through a shooting scene selection lens and the requirement on the number of pixels.
In the embodiment of the invention, the image or video with the pixel number and the resolution more than or equal to the pixel number of the original image sensor can be reconstructed by controlling the displacement step length of the scanning micro-lens module and combining the RL deconvolution algorithm.
In the embodiment of the invention, the image acquired by the image sensor in the process of scanning the micro lens can eliminate the aberration caused by a lens or a macroscopic scene through a digital calculation aberration compensation technology, so that the imaging resolution reaches the optical diffraction limit of the system. The embodiment of the invention aims to establish a miniaturized aberration-free photographing system, and the photo with the resolution ratio larger than the size of an image sensor is formed by fusing a plurality of light field photos, so that the optical diffraction limit is reached, and the high-resolution imaging in the true sense is realized.
< second embodiment >
Fig. 3 schematically shows a flow chart of a wide field of view high resolution imaging method based on calculating aberration compensation according to a preferred embodiment of the invention.
The wide-field high-resolution imaging method based on calculating aberration compensation according to the preferred embodiment of the present invention as shown in fig. 3 includes:
establishing a simulation model of the wide-field high-resolution camera equipment based on the calculated aberration compensation shown in the figures 1 and 2, designing the size of a micro-lens array according to the system requirement, and simultaneously simulating and calculating a point spread image of an object space and an ideal point spread function of a phase space; specifically, a numerical simulation model of optical elements such as a lens, a micro-lens array, an image sensor and the like and imaging elements is established, a point spread pattern of an object space is obtained through simulation, and an ideal point spread function of a phase space is further obtained through calculation;
capturing light field data comprising one or more scan cycles; the camera can be calibrated and the light field data can be collected by combining with the scanning micro-lens array;
through a digital algorithm, the light field data are rearranged into phase space images with different light incidence angles for correcting aberration, and a high-resolution aberration-free image or video is reconstructed by combining the calculated ideal point spread function or the point spread function obtained by direct calibration. Wherein the aberration can be compensated using the collected light field data in combination with a digitally computed aberration compensation algorithm; and reconstructing a high-resolution image or a plurality of frames of reconstructed images by combining a deconvolution algorithm and fusing the reconstructed images into a video.
As an example, the following steps may be specifically performed:
1) an optical system simulation model is established, a micro-lens array with proper parameters is selected by simulating the imaging effect of an optical imaging system, ideal point diffusion images of object spaces at different positions are obtained, and the point diffusion images of a phase space are calculated. The phase space is calculated in one of the most common representations, the wigner distribution is described as follows:
W(r,k)＝∫<ψ*(r+ξ/2)ψ(r-ξ/2)＞eikξdξ.
where r ═ x, y, and k ═ u, v correspond to the two-dimensional space vector and the two-dimensional space frequency domain vector, respectively. ψ (r) is a corresponding wave function. In a single module (optical input module and scanning microlens array module), the lens is a fourier transformer with a pupil:
where k is the wavenumber of light, f is the focal length of the lens, P (x, y) is 1 (within the lens aperture) and P (x, y) is 0 (elsewhere).
In a single module (scanning microlens array module), the spatial distance between the microlenses and the imaging module is simulated using diffraction of fresnel propagation simulated light.
2) And according to the simulation model parameters, constructing a wide-field high-resolution shooting technology based on aberration compensation calculation. After the shooting scene is selected, the optical input module collects light beams of the target scene, adjusts the frame rates of cameras in the scanning micro-lens array module and the imaging module and the scanning speed of the micro-lens array, and then images are carried out through the image sensor. Firstly, calibrating a camera as required, and further shooting and collecting a target scene.
3) And carrying out calculation processing on the collected multiple groups of light field data. And removing image aberrations corresponding to different angles through a digital self-adaptive algorithm, combining the scanning times, fusing the light field data with the removed aberrations in each scanning period, and reconstructing to obtain an image or video with the resolution greater than or equal to that of the original image.
The wide-field high-resolution camera shooting technology based on aberration compensation calculation provided by the invention has better performance than other systems in the aspects of aberration removal, original definition degree improvement, original resolution improvement and the like, and the great advantages of the wide-field high-resolution camera shooting technology in an optical imaging system are verified.
Thus, referring to fig. 4, the imaging process of a street view photograph is simulated. And outputting the aberration image and combining the optimized image of the numerical calculation aberration compensation algorithm and the deconvolution reconstruction algorithm, wherein the signal-to-noise ratio is improved from 20% to 85%.
Further, referring to FIG. 5, in one embodiment of the present invention, a USAF1951 resolution test plate without aberration added was simulated, and an image matching the diffraction limit resolution of the system was reconstructed by changing the step size of the movement of the scanning microlens array module 200. The resolution is improved by 3 times, and the number of pixels is improved by 9 times.
According to the wide-field high-resolution camera shooting technology based on aberration compensation calculation, the system breaks through the limitation that the general camera shooting system cannot reach the optical diffraction limit resolution due to the existence of aberration, and has the following advantages: (1) by calculating aberration compensation, the aberration existing in the shooting scene and the optical element is eliminated, and more effective pixel numbers are achieved. The clearer shooting quality is obtained for the images or videos with the same pixel number; (2) by changing the step length of the movement of the scanning microlens array, the aberration-free number higher than the number of pixels of the image sensor is obtained
Images or videos, to achieve the diffraction limit resolution of the system; (3) the whole system can be formed by only inserting a scanning micro-lens array between a common optical lens and an image sensor. Complex calculation and compensation are realized in a digital mode, so that miniaturization in a real sense is realized; (4) the quantity of the collected data is equivalent to the data storage quantity of the normally shot image, data redundancy is not caused, and the requirement on storage space is not too high.
In the embodiment of the invention, a simulation model is established, a micro-lens array with proper parameters is selected by simulating the imaging effect of an optical imaging system, ideal point diffusion patterns of object spaces at different positions are obtained, and the point diffusion patterns of a phase space are calculated. After the simulation design is finished, the size of the image sensor is selected through a shooting scene selection lens and the requirement on the number of pixels.
In the embodiment of the invention, the image or video with the pixel number and the resolution more than or equal to the pixel number of the original image sensor can be reconstructed by controlling the displacement step length of the scanning micro-lens module and combining the RL deconvolution algorithm.
In the embodiment of the invention, the image acquired by the image sensor in the process of scanning the micro lens can eliminate the aberration caused by a lens or a macroscopic scene through a digital calculation aberration compensation technology, so that the imaging resolution reaches the optical diffraction limit of the system. The embodiment of the invention aims to establish a small-sized aberration-free photographing method, and the photo with the resolution ratio larger than the size of an image sensor is formed by fusing a plurality of light field photos, so that the optical diffraction limit is reached, and high-resolution imaging in the true sense is realized.
According to the wide-field high-resolution shooting technology based on the calculation aberration compensation, which is different from a common light field camera, the invention compensates the spatial information lost due to the acquisition of the angle information through the scanning of the micro lens array. Through the acquisition of information at different angles and the digital image processing, imaging patterns at different light angles are obtained. And then eliminating the aberration by calculating an aberration compensation algorithm, and fusing the image or the video without the aberration. The invention also aims to break through the limitation of the number of pixels of the image sensor and achieve the number of pixels corresponding to the diffraction limit resolution. By adjusting the moving step length in the scanning process of the micro lens array and processing the step length by an algorithm, images or videos with higher resolution and more pixels than the original image can be obtained. Also, high resolution images or videos reconstructed in this way are aberration-free.
It should be noted that the terms "first", "second", "third", and the like in the description are used for distinguishing various components, elements, steps, and the like in the description, and are not used for indicating a logical relationship or a sequential relationship between the various components, elements, steps, and the like, unless otherwise specified.
It should be noted that in the description of the present invention, unless otherwise specified or limited, the terms "mounted," "connected," "coupled," and the like are used broadly and can be, for example, a removably secured connection. Connected or integrally connected; mechanical or electrical connections are also possible; or may be a direct connection or an indirect connection through intermediate structures; or may be an intrinsic exchange of the two elements. The specific meaning of the above terms will be understood by those skilled in the art as the case may be.
It should also be noted that in the description of the present invention, the reconstructed image and the captured image each include a concept of "video". The video is the fast play of a plurality of continuous images. Are within the scope of application of the technology.
In the description, it is to be understood that terms such as "central," "longitudinal," "lateral," "front," "rear," "right," "left," "inner," "outer," "lower," "upper," "horizontal," "vertical," "above," "below," "upper," "top," "bottom," and derivatives thereof (e.g., "horizontal," "downward," "upward," etc.) are to be interpreted as referring to the orientation as subsequently described or as shown in the drawing under discussion.
It is to be understood that while the present invention has been described in conjunction with the preferred embodiments thereof, it is not intended to limit the invention to those embodiments. It will be apparent to those skilled in the art from this disclosure that many changes and modifications can be made, or equivalents modified, in the embodiments of the invention without departing from the scope of the invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.
Reference throughout this specification to "an embodiment," "some embodiments," "an example," "a particular example" or "some examples" means that a particular feature, structure, material, or characteristic described in connection with the embodiment is included in at least one embodiment or example of the invention. In this specification, an exemplary description of the foregoing terms does not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.
Claims (10)
1. A wide-field high-resolution image pickup apparatus based on calculation of aberration compensation, characterized by comprising: the scanning micro-lens array comprises an optical input module, a scanning micro-lens array module and an imaging module; wherein the optical input module comprises an optical lens; the scanning micro-lens array module comprises a displacement device and a micro-lens array; the imaging module comprises an image sensor; the optical input module shoots a scene through any optical lens to collect optical information, the scanning micro-lens array module drives the micro-lens array to perform periodic moving scanning through the displacement device, and the optical information collected by the optical lens is further transmitted to an image sensor in the imaging module.
2. The wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to claim 1, wherein the microlens array of the scanning microlens array module collects spatial information as well as angular information by scanning for calculation of the numerical calculation aberration compensation algorithm and image reconstruction algorithm of the image sensor.
3. The wide-field high-resolution image pickup apparatus based on the calculation of aberration compensation according to claim 1 or 2, wherein the optical lens is in an in-focus state or in a predetermined defocus range, the microlens array is placed at a back focal plane of the optical lens, and the image sensor is in a predetermined axial range of a back focal point of the microlens array.
4. The wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to claim 1 or 2, wherein the microlens array is disposed at an image point of the lens, and the angular information of the light beam is obtained while obtaining spatial information of the illumination collected by the lens.
5. The wide-field high-resolution imaging apparatus based on calculation aberration compensation according to claim 1 or 2, wherein the scanning speed of the scanning microlens module matches the acquisition speed of the image sensor in the imaging module.
6. The wide-field high-resolution imaging apparatus based on calculation aberration compensation according to claim 1 or 2, wherein the size of the microlens array just covers the area of the effective collection area of the image sensor.
7. The wide-field high-resolution image pickup apparatus based on the calculated aberration compensation according to claim 1 or 2, wherein a numerical aperture of an optical lens in the optical input module is matched with a numerical aperture of each microlens in the microlens array.
8. The wide-field high-resolution image pickup apparatus according to claim 1 or 2, wherein in the periodic moving scan, each scanning period is based on the number of pixels covered by a single microlens, and the image sensor acquires the corresponding captured image for each movement in one period, so as to correct the aberration by a digital algorithm and reconstruct a clear image.
9. The wide-field high-resolution image pickup apparatus based on calculation of aberration compensation according to claim 1 or 2, characterized by further comprising: and the processing unit is used for rearranging the light field data into phase space images with different light incidence angles through a digital self-adaptive algorithm for correcting aberration, and reconstructing to obtain an aberration-free image or video with the resolution greater than or equal to that of the original image by combining the calculated ideal point diffusion function or the point diffusion function obtained by direct calibration.
10. A wide-field high-resolution imaging method based on calculation aberration compensation is characterized by comprising the following steps:
establishing a simulation model of the wide-field high-resolution camera equipment based on the calculation of aberration compensation according to one of claims 1 to 8, designing the size of a micro-lens array according to system requirements, and simultaneously simulating and calculating a point spread image of an object space and an ideal point spread function of a phase space;
shooting light field data containing one or more scanning periods by using the simulation model parameters; after a shooting scene is selected, the optical input module collects light beams of a target scene, adjusts the frame rates of cameras in the scanning micro-lens array module and the imaging module and the scanning speed of the micro-lens array, and then images are formed through the image sensor.
Through a digital self-adaptive algorithm, the light field data is rearranged into phase space images with different light incidence angles for correcting aberration, and the phase space images are combined with the calculated ideal point diffusion function or the point diffusion function obtained by direct calibration to reconstruct and obtain an aberration-free image or video with the resolution greater than or equal to that of the original image.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2019111264222 | 2019-11-15 | ||
| CN201911126422 | 2019-11-15 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111182191A true CN111182191A (en) | 2020-05-19 |
| CN111182191B CN111182191B (en) | 2021-08-20 |
Family
ID=70658363
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010133526.2A Active CN111182191B (en) | 2019-11-15 | 2020-02-28 | Wide-field high-resolution camera shooting equipment and method based on aberration compensation calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111182191B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112304443A (en) * | 2020-10-30 | 2021-02-02 | 中国科学院光电技术研究所 | Hartmann wavefront sensor with variable spatial resolution |
| CN113630517A (en) * | 2021-10-08 | 2021-11-09 | 清华大学 | Intelligent imaging method and device for light-electric inductance calculation integrated light field |
| CN115086550A (en) * | 2022-05-30 | 2022-09-20 | 元潼(北京)技术有限公司 | Meta-imaging method and system |
| CN115174789A (en) * | 2022-07-04 | 2022-10-11 | 上海乂义实业有限公司 | Micro-lens array focusing type light field camera based on micro-motion platform |
| CN115278065A (en) * | 2022-07-18 | 2022-11-01 | 奕目(上海)科技有限公司 | Light field imaging method, light field imaging system, light field camera and storage medium |
| CN115314619A (en) * | 2022-08-04 | 2022-11-08 | 上海乂义实业有限公司 | Micro-lens array non-focusing type light field camera based on micro-scanning technology |
| WO2023280292A1 (en) * | 2021-07-08 | 2023-01-12 | 清华大学 | Fast-scanning and three-dimensional imaging method and device for large-volume scattered sample |
| CN115967852A (en) * | 2022-02-11 | 2023-04-14 | 四川大学 | Image processing method based on optical compensation optimization algorithm |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050128308A1 (en) * | 2003-12-06 | 2005-06-16 | Tholl Hans D. | Imaging device for the stabilized imaging of an object onto a detector |
| CN102439979A (en) * | 2009-04-22 | 2012-05-02 | 雷特利克斯股份有限公司 | Digital imaging system, plenoptic optical device and image data processing method |
| CN102739945A (en) * | 2012-05-24 | 2012-10-17 | 上海理工大学 | Optical field imaging device and method |
| CN104363369A (en) * | 2014-11-17 | 2015-02-18 | 清华大学深圳研究生院 | Image restoration method and device for optical field camera |
| CN106488148A (en) * | 2016-11-01 | 2017-03-08 | 首都师范大学 | A kind of super-resolution image sensor and its building method |
| CN106842178A (en) * | 2017-01-13 | 2017-06-13 | 清华大学深圳研究生院 | A kind of light field method for estimating distance and optical field imaging system |
| CN109541791A (en) * | 2019-01-30 | 2019-03-29 | 清华大学 | High-resolution light field micro imaging system and method based on sub-pix translation |
| CN109615651A (en) * | 2019-01-29 | 2019-04-12 | 清华大学 | Three-dimensional microscopy method and system based on light field microscopic system |
| CN110441271A (en) * | 2019-07-15 | 2019-11-12 | 清华大学 | Light field high-resolution deconvolution method and system based on convolutional neural networks |
-
2020
- 2020-02-28 CN CN202010133526.2A patent/CN111182191B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050128308A1 (en) * | 2003-12-06 | 2005-06-16 | Tholl Hans D. | Imaging device for the stabilized imaging of an object onto a detector |
| CN102439979A (en) * | 2009-04-22 | 2012-05-02 | 雷特利克斯股份有限公司 | Digital imaging system, plenoptic optical device and image data processing method |
| CN102739945A (en) * | 2012-05-24 | 2012-10-17 | 上海理工大学 | Optical field imaging device and method |
| CN104363369A (en) * | 2014-11-17 | 2015-02-18 | 清华大学深圳研究生院 | Image restoration method and device for optical field camera |
| CN106488148A (en) * | 2016-11-01 | 2017-03-08 | 首都师范大学 | A kind of super-resolution image sensor and its building method |
| CN106842178A (en) * | 2017-01-13 | 2017-06-13 | 清华大学深圳研究生院 | A kind of light field method for estimating distance and optical field imaging system |
| CN109615651A (en) * | 2019-01-29 | 2019-04-12 | 清华大学 | Three-dimensional microscopy method and system based on light field microscopic system |
| CN109541791A (en) * | 2019-01-30 | 2019-03-29 | 清华大学 | High-resolution light field micro imaging system and method based on sub-pix translation |
| CN110441271A (en) * | 2019-07-15 | 2019-11-12 | 清华大学 | Light field high-resolution deconvolution method and system based on convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 朱茜等: "基于数字微镜器件并行共焦成像的光点阵列优化", 《光学学报》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112304443A (en) * | 2020-10-30 | 2021-02-02 | 中国科学院光电技术研究所 | Hartmann wavefront sensor with variable spatial resolution |
| WO2023280292A1 (en) * | 2021-07-08 | 2023-01-12 | 清华大学 | Fast-scanning and three-dimensional imaging method and device for large-volume scattered sample |
| CN113630517A (en) * | 2021-10-08 | 2021-11-09 | 清华大学 | Intelligent imaging method and device for light-electric inductance calculation integrated light field |
| US11425292B1 (en) | 2021-10-08 | 2022-08-23 | Tsinghua University | Method and apparatus for camera-free light field imaging with optoelectronic intelligent computing |
| CN115967852A (en) * | 2022-02-11 | 2023-04-14 | 四川大学 | Image processing method based on optical compensation optimization algorithm |
| CN115967852B (en) * | 2022-02-11 | 2024-02-06 | 四川大学 | Image processing method based on optical compensation optimization algorithm |
| CN115086550A (en) * | 2022-05-30 | 2022-09-20 | 元潼(北京)技术有限公司 | Meta-imaging method and system |
| CN115086550B (en) * | 2022-05-30 | 2023-04-28 | 元潼(北京)技术有限公司 | Meta imaging system |
| WO2023231139A1 (en) * | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Meta-imaging method and system |
| CN115174789A (en) * | 2022-07-04 | 2022-10-11 | 上海乂义实业有限公司 | Micro-lens array focusing type light field camera based on micro-motion platform |
| CN115278065A (en) * | 2022-07-18 | 2022-11-01 | 奕目(上海)科技有限公司 | Light field imaging method, light field imaging system, light field camera and storage medium |
| CN115314619A (en) * | 2022-08-04 | 2022-11-08 | 上海乂义实业有限公司 | Micro-lens array non-focusing type light field camera based on micro-scanning technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111182191B (en) | 2021-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111182191B (en) | Wide-field high-resolution camera shooting equipment and method based on aberration compensation calculation | |
| CN111182238B (en) | High-resolution mobile electronic equipment imaging device and method based on scanning light field | |
| US10127682B2 (en) | System and methods for calibration of an array camera | |
| CN102326380B (en) | There are image sensor apparatus and the method for the efficient lens distortion calibration function of row buffer | |
| US9482850B2 (en) | Monocentric imaging | |
| US7723662B2 (en) | Microscopy arrangements and approaches | |
| CN109413407B (en) | High spatial resolution light field acquisition device and image generation method | |
| US20070285553A1 (en) | Method and apparatus for image capturing capable of effectively reproducing quality image and electronic apparatus using the same | |
| CN110636277B (en) | Detection apparatus, detection method, and image pickup apparatus | |
| KR20130039676A (en) | Image-capturing device, image processing apparatus, image processing method, and method of manufacturing image-capturing device | |
| WO2014074202A2 (en) | Monocentric lens designs and associated imaging systems having wide field of view and high resolution | |
| JP6003578B2 (en) | Image generation method and apparatus | |
| KR101689534B1 (en) | Multiscale Imaging System | |
| CN113487658B (en) | Dynamic scene shooting method and device for scanning light field imaging system | |
| KR20200140791A (en) | Constant resolution continuous hybrid zoom system | |
| WO2012061571A1 (en) | Camera systems and methods for gigapixel computational imaging | |
| CN108805921B (en) | Image acquisition system and method | |
| JP4369867B2 (en) | A system to increase image resolution by rotating the sensor | |
| CN103081481A (en) | Stereography device and stereography method | |
| KR20160143138A (en) | Camera and control method thereof | |
| JP6168220B2 (en) | Image generation apparatus, image processing apparatus, image generation method, and image processing program | |
| JP6330955B2 (en) | Imaging apparatus and imaging method | |
| Oberdörster et al. | Correcting distortion and braiding of micro-images from multi-aperture imaging systems | |
| CN108205236A (en) | Panoramic camera and its camera lens | |
| JP6569769B2 (en) | Arbitrary viewpoint image composition method and image processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210618 Address after: Room 301, building 3, 1390 cangxing street, Cangqian street, Yuhang District, Hangzhou City, Zhejiang Province Applicant after: Zhejiang Hehu Technology Co.,Ltd. Address before: 100089 115 / F, block C, Zhizao street, Zhongguancun, 45 Chengfu Road, Haidian District, Beijing Applicant before: Beijing chaofang Information Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |