WO2023284792A1 - 用于扫描光场成像系统的镜头标定方法及装置 - Google Patents
用于扫描光场成像系统的镜头标定方法及装置 Download PDFInfo
- Publication number
- WO2023284792A1 WO2023284792A1 PCT/CN2022/105495 CN2022105495W WO2023284792A1 WO 2023284792 A1 WO2023284792 A1 WO 2023284792A1 CN 2022105495 W CN2022105495 W CN 2022105495W WO 2023284792 A1 WO2023284792 A1 WO 2023284792A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- light field
- image
- image pair
- lens
- images
- Prior art date
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000004075 alteration Effects 0.000 claims abstract description 72
- 238000009826 distribution Methods 0.000 claims abstract description 60
- 238000012545 processing Methods 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 abstract description 18
- 238000012937 correction Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 125000001475 halogen functional group Chemical group 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Definitions
- the present application relates to the technical field of structured light imaging, in particular to a lens calibration method and device for a scanning light field imaging system.
- the diffraction limit that the optical system can achieve is enough to meet the needs of our macro scene shooting, and there is also enough technology to produce a high-resolution image sensor with a large area array and a small pixel size.
- the aberration will gradually increase. Therefore, as the number of pixels increases, the number of effective pixels will be limited to a limited scale. That is to say, no matter what Increasing the number of pixels of the image sensor or the numerical aperture of the lens will make it difficult to further improve the resolution and clarity of the image, that is, the existence of aberrations hinders the further development of the optical system.
- the development of gigapixels has gone through the following two stages: In the first stage, by reducing the size of the optical aperture and reducing the existence of aberrations, it can break through the limitation of the number of effective pixels of the original ordinary camera. This solution will reduce the amount of light passing through, increase the exposure time, and greatly reduce the signal-to-noise ratio; in the second stage, multi-lens shooting and splicing will form a billion-pixel imaging. This solution reduces aberrations by increasing the number of lenses The magnitude of the corresponding increase with the increase of the size of the single lens is better than the method of reducing the aperture to break through the aberration limit and achieve better imaging effect, but the imaging effect is far less than the performance of the optical diffraction limit.
- the existing technology can further correct the aberration in the phase space by collecting the angle information and spatial information of the light at the same time.
- the uniform halo area only corresponds to a local image with a small number of pixels, so it takes a lot of computing resources and computing time to achieve global aberration correction.
- This application aims to solve one of the technical problems in the related art at least to a certain extent.
- the first purpose of this application is to propose a lens calibration method for scanning light field imaging systems, so as to reduce the time and complexity of aberration correction in multiple small areas, and actually solve the problem of aberrations. Make the imaging effect as close as possible to the optical diffraction limit performance.
- the second purpose of the present application is to propose a lens calibration device for a scanning light field imaging system.
- the third object of the present application is to propose a computer device.
- the fourth objective of the present application is to provide a non-transitory computer-readable storage medium.
- the embodiment of the first aspect of the present application proposes a lens calibration method for a scanning light field imaging system, including:
- a non-globally consistent point spread function is generated according to the global distribution of different order aberrations of the image, and the lens is calibrated according to the solution of the non-globally consistent point spread function.
- the multiple light field original images are processed into three-dimensional image stacks with different angle information.
- the plurality of light field original images are divided into central perspective images and non-central perspective images, and each central perspective image is paired with each non-central perspective image to form image pair, and obtain the global distribution of different order aberrations of the image pair, specifically including:
- the global distribution of different order aberrations of the image pair is obtained.
- the global distribution of different order aberrations of the image pair is acquired, specifically including:
- a fitting process is performed on the aberration distribution of the image pair to obtain a global distribution of aberrations of different orders of the image pair.
- the embodiment of the second aspect of the present application proposes a lens calibration device for a scanning light field imaging system, including:
- a light field image acquisition module configured to acquire a plurality of light field original images containing different angle information
- a light field image processing module configured to divide the plurality of light field original images into central perspective images and non-central perspective images, pair each central perspective image with each non-central perspective image to form an image pair, and obtain The global distribution of the image to different orders of aberrations;
- the light field lens calibration module is used to generate a non-globally consistent point spread function according to the global distribution of the image to different order aberrations, and to calibrate the lens according to the solution of the non-globally consistent point spread function.
- the lens calibration device further includes:
- the light field image initial processing module is configured to process the plurality of light field original images into a three-dimensional image stack with different angle information.
- the light field image processing module specifically includes:
- the first light field image processing unit is configured to perform non-rigid registration or distortion stretch processing on the image pair to obtain the aberration distribution of the image pair;
- the second light field image processing unit is configured to acquire global distributions of different order aberrations of the image pair according to the aberration distribution of the image pair.
- the embodiment of the third aspect of the present application proposes a computer device, including: including a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor When the computer program is executed, the method as described in the embodiment of the first aspect of the present application is realized.
- the embodiment of the fourth aspect of the present application proposes a non-transitory computer-readable storage medium, on which a computer program is stored.
- the computer program is executed by a processor, the implementation of the first aspect of the present application can be realized. The method described in the example.
- the embodiment of the present application provides a lens calibration method for a scanning light field imaging system, a lens calibration device, a computer device, and a non-transitory computer-readable storage medium.
- Light field original images multiple light field original images are divided into central perspective images and non-central perspective images, each central perspective image is paired with each non-central perspective image to form an image pair, and image pairs of different order are obtained
- Global distribution of aberrations according to the global distribution of different orders of aberrations in the image, a non-globally consistent point spread function is generated, and the lens is calibrated according to the solution of the non-globally consistent point spread function.
- a non-globally consistent point spread function can be generated according to the global distribution of different order aberrations of the optical lens, and the lens can be calibrated according to the solution of the non-globally consistent point spread function, and then quickly passed Computational method to obtain diffraction-limited imaging results and realize global aberration correction requires a lot of computing resources and computing time.
- FIG. 1 is a flowchart of a lens calibration method for a scanning light field imaging system provided in Embodiment 1 of the present application;
- FIG. 2 is a flow chart of a lens calibration method for a scanning light field imaging system provided in Embodiment 2 of the present application.
- FIG. 3 is a schematic structural diagram of a lens calibration device for a scanning light field imaging system provided in Embodiment 3 of the present application.
- the three limiting conditions that determine the imaging quality or imaging resolution of the imaging system are as follows: First, the sampling rate of the image sensor: the popular image sensors are mainly divided into two types: CCD and CMOS. Being smaller is conducive to the generation of higher-resolution images; the second is the optical diffraction limit of the system: according to the Rayleigh resolution criterion, the numerical aperture of each lens in the imaging system determines the resolution of the system; the third is aberration, image Differences generally exist in the natural environment, such as atmospheric scattering.
- the lens is an artificially polished device, there will be deviations from the ideal lens model in theoretical optics, that is to say, aberrations also exist in the lens; in addition, with the gradual increase in lens size, Large, the theory of paraxial optics in an ideal optical system is no longer applicable, and the trajectories of off-axis rays are difficult to predict as easily as those of paraxial rays. All in all, the above three points limit the imaging capability of the optical imaging system and hinder the further development of gigapixel imaging.
- FIG. 1 is a flowchart of a lens calibration method for a scanning light field imaging system provided by an embodiment of the present application.
- the embodiment of the present application provides a lens calibration method for a scanning light field imaging system to reduce the time and complexity of aberration correction in multiple small areas, practically solve the problem of aberration, and maximize the imaging effect.
- the method comprises the following steps:
- Step 110 collecting a plurality of light field original images containing different angle information.
- light field original images containing different angle information are collected through a light field system or a scanning light field system, wherein the light field original images with different angle information are images of different angles and different spaces.
- Step 120 Divide a plurality of light field original images into central perspective images and non-central perspective images, pair each central perspective image with each non-central perspective image to form an image pair, and obtain image pairs with different order aberrations global distribution.
- step 120 in the embodiment of the present application specifically includes the following steps:
- Step 121 perform non-rigid registration or distortion stretching processing on the image pair to obtain the aberration distribution of the image pair.
- this application uses the central perspective image as an example, and performs two-by-two non-rigid registration or distortion stretching on the example and non-central perspective images as image pairs, so that images of different perspectives can be obtained according to the transformation of coordinates. Aberration distribution.
- Step 122 according to the aberration distribution of the image pair, obtain the global distribution of different order aberrations of the image pair.
- the global distribution of the phase difference of different orders of the image pair may be obtained by performing fitting processing on the phase difference distribution of the image pair. Specifically, according to the aberration distributions of different viewing angles, the global distribution of aberrations of different orders is obtained through Zernike polynomial fitting or other fitting methods. The images thus obtained have a more accurate global distribution of phase differences of different orders, paving the way for precise calibration of the lens, and then fundamentally solve the aberration problem, making the imaging effect as close as possible to the optical diffraction limit performance.
- Step 130 generating a non-globally consistent point spread function according to the global distribution of different orders of aberrations in the image, and calibrating the lens according to the solution of the non-globally consistent point spread function.
- the embodiment of the present application processes and divides information images of different angles and spaces acquired by a light field imaging system or a scanning light field imaging system to obtain central perspective images and non-central perspective images, with the central angle as the
- the example and the non-central viewing angle images are used as an image pair, and the global non-rigid registration estimation is used to obtain the corresponding parallax distribution maps under different viewing angles, so as to restore the system aberration distribution of the optical lens.
- the system aberration distribution of the optical lens can be obtained, and then the diffraction-limited imaging result can be quickly obtained by calculation, reducing the time complexity of calculation and the waste of resources.
- FIG. 2 is a flowchart of a lens calibration method for a scanning light field imaging system provided by an embodiment of the present application.
- the embodiment of the present application provides a lens calibration method for a scanning light field imaging system to reduce the time and complexity of aberration correction in multiple small areas, practically solve the problem of aberration, and maximize the imaging effect.
- the method comprises the following steps:
- Step 210 collecting multiple light field original images containing different angle information.
- Step 220 processing multiple light field original images into a three-dimensional image stack with different angle information.
- Step 230 Divide a plurality of light field original images into central perspective images and non-central perspective images, pair each central perspective image with each non-central perspective image to form an image pair, and obtain the image pairs with different orders of difference global distribution.
- Step 240 generate a non-globally consistent point spread function according to the global distribution of different orders of aberrations in the image, and calibrate the lens according to the solution of the non-globally consistent point spread function.
- steps 210, 230 and 240 in the embodiment of the present application are the same as the steps 110, 120 and 130 in the embodiment 1 of the present application, in order to avoid repetition, they will not be repeated here, and only step S220 will be further described.
- the original image of the light field is transformed into a three-dimensional image stack with different angle information by means of a rearrangement method.
- the embodiment of the present application sorts the original light field images in order, and temporarily stores the sorted original light field images in the 3D image stack, which is beneficial to the processing of subsequent steps.
- the design of the 3D image stack has It is beneficial to the processing speed of dividing a plurality of light field original images into central perspective images and non-central perspective images in step 230. Since the three-dimensional image stack temporarily stores the sorted light field original images and has addresses, it can be quickly processed after the data processing is completed. free memory.
- the embodiment of the present application processes and divides information images of different angles and spaces acquired by a light field imaging system or a scanning light field imaging system to obtain central perspective images and non-central perspective images, with the central angle as the
- the example and the non-central viewing angle images are used as an image pair, and the global non-rigid registration estimation is used to obtain the corresponding parallax distribution maps under different viewing angles, so as to restore the system aberration distribution of the optical lens.
- the system aberration distribution of the optical lens can be obtained, and then the diffraction-limited imaging result can be quickly obtained by calculation, reducing the time complexity of calculation and the waste of resources.
- the present application also proposes a lens calibration device for a scanning light field imaging system.
- FIG. 3 is a schematic structural diagram of a lens calibration device for a scanning light field imaging system provided by an embodiment of the present application.
- the lens calibration device for the scanning light field imaging system includes:
- a light field image acquisition module 10 configured to acquire a plurality of light field original images containing different angle information
- the light field image processing module 20 is configured to divide a plurality of light field original images into central perspective images and non-central perspective images, pair each central perspective image with each non-central perspective image to form an image pair, and acquire the image Global distribution of difference for different orders;
- the light field lens calibration module 30 is configured to generate a non-globally consistent point spread function according to the global distribution of phase differences of different orders in the image, and calibrate the lens according to the solution of the non-globally consistent point spread function.
- the embodiment of the present application further includes: a light field image initial processing module, configured to process the plurality of light field original images into a three-dimensional image stack with different angle information.
- the light field image processing module specifically includes:
- the first light field image processing unit is used to perform non-rigid registration or distortion stretch processing on the image pair to obtain the phase difference distribution of the image pair;
- the second light field image processing unit is configured to obtain the global distribution of phase differences of different orders of the image pair according to the phase difference distribution of the image pair.
- the present application also proposes a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor.
- a computer program stored on the memory and operable on the processor.
- the present application also proposes a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the methods described in the above embodiments are implemented.
- the present application also proposes a computer program product.
- an instruction processor in the computer program product executes, an artificial intelligence-based method is executed, and the method includes: .
- first and second are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features.
- the features defined as “first” and “second” may explicitly or implicitly include at least one of these features.
- “plurality” means at least two, such as two, three, etc., unless otherwise specifically defined.
- a "computer-readable medium” may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device, or device.
- computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM).
- the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since the program can be read, for example, by optically scanning the paper or other medium, followed by editing, interpretation or other suitable processing if necessary.
- the program is processed electronically and stored in computer memory.
- each part of the present application may be realized by hardware, software, firmware or a combination thereof.
- various steps or methods may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system.
- a suitable instruction execution system For example, if implemented in hardware as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: a discrete Logic circuits, ASICs with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.
- each functional unit in each embodiment of the present application may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module.
- the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.
- the storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
一种用于扫描光场成像系统的镜头标定方法及装置，方法包括：采集包含不同角度信息的多个光场原始图像；将多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取图像对不同阶像差的全局分布；根据图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定。上述方法可以根据光学镜头的不同阶像差的全局分布生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定，进而快速获得衍射极限成像结果，解决全局的像差矫正需要花费大量的计算资源与计算时间的问题。
Description
相关申请的交叉引用
本申请要求清华大学于2021年07月14日提交的、申请名称为“用于扫描光场成像系统的镜头标定方法及装置”的、中国专利申请号“202110792338.5”的优先权。
本申请涉及结构光成像技术领域，尤其涉及一种用于扫描光场成像系统的镜头标定方法及装置。
近年来，超高像素数目成像逐渐进入人们的视野，随着机器视觉、无人机、高清监视系统等行业或设备的普及与扩散，人们对于高分辨率、大像素数目成像的需求日益明显。
得益于工业的发展，光学系统可以达到的衍射极限已经足以满足我们宏观场景拍摄的需求，同时也有足够的工艺制作出大面阵小像元尺寸的高分辨率像感器。但从物理模型上分析，随着单镜头尺寸的增长，像差会随之逐步提升，因此随着像素数目的提升、有效像素数目会被限制在一个有限的尺度上，也就是说，无论如何增加像感器的像素数目或是镜头的数值孔径，图像都很难进一步提升分辨率与清晰度，即像差的存在阻碍了光学系统的一步发展。
在这样的条件下，十亿像素的发展经历了下述两个阶段：第一个阶段通过缩小光学孔径尺寸，减小像差的存在，可以突破原有普通相机的有效像素数目局限，但这种方案会导致通光量减小，曝光时间增大，信噪比大幅度降低；第二个阶段通过多镜头拍摄拼接，形成十亿像素成像，该种方案通过增多镜头数目的方式减小像差随单镜头尺寸增大而相应增大的幅度，比减小孔径的方式更好的突破了像差限制，达到较好的成像效果，但成像效果也远远小于光学衍射极限性能。由此可知，现有技术能通过同时采集光线的角度信息和空间信息，进一步实现在相位空间矫正像差，但是对于高像素数目成像，比如亿级像素成像等存在全局不一致的像差，像差一致的等晕区只对应像素数目很少的图像局部，因此实现全局的像差矫正需要花费大量的计算资源与计算时间。
发明内容
本申请旨在至少在一定程度上解决相关技术中的技术问题之一。
为此，本申请的第一个目的在于提出一种用于扫描光场成像系统的镜头标定方法，以减少多个小区域像差矫正的时间和复杂度，从实际上解决象差的问题，使成像效果尽可能的接近光学衍射极限性能。
本申请的第二个目的在于提出一种用于扫描光场成像系统的镜头标定装置。
本申请的第三个目的在于提出一种计算机设备。
本申请的第四个目的在于提出一种非临时性计算机可读存储介质。
为达上述目的，本申请第一方面实施例提出了一种用于扫描光场成像系统的镜头标定方法，包括：
采集包含不同角度信息的多个光场原始图像；
将所述多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶像差的全局分布；
根据所述图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据所述非全局一致的点扩散函数的求解对镜头进行标定。
可选的，在本申请实施例中，在所述采集包含不同角度信息的多个光场原始图像之后，还包括：
将所述多个光场原始图像处理为具有不同角度信息的三维图像堆栈。
可选的，在本申请实施例中，所述将所述多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶像差的全局分布，具体包括：
对所述图像对进行非刚性配准或畸变拉伸处理，得到所述图像对的像差分布；
根据所述图像对的像差分布，获取所述图像对不同阶像差的全局分布。
可选的，在本申请实施例中，根据所述图像对的像差分布，获取所述图像对不同阶像差的全局分布，具体包括：
对所述图像对的像差分布进行拟合处理，以获取所述图像对不同阶像差的全局分布。
为达上述目的，本申请第二方面实施例提出了一种用于扫描光场成像系统的镜头标定装置，包括：
光场图像采集模块，用于采集包含不同角度信息的多个光场原始图像；
光场图像处理模块，用于将所述多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶像差的全局分布；
光场镜头标定模块，用于根据所述图像对不同阶像差的全局分布，生成非全局一致的 点扩散函数，并根据所述非全局一致的点扩散函数的求解对镜头进行标定。
可选的，在本申请实施例中，所述镜头标定装置，还包括：
光场图像初始处理模块，用于将所述多个光场原始图像处理为具有不同角度信息的三维图像堆栈。
可选的，在本申请实施例中，所述光场图像处理模块，具体包括：
第一光场图像处理单元，用于对所述图像对进行非刚性配准或畸变拉伸处理，得到所述图像对的像差分布；
第二光场图像处理单元，用于根据所述图像对的像差分布，获取所述图像对不同阶像差的全局分布。
为达上述目的，本申请第三方面实施例提出了一种计算机设备，包括：包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现如本申请第一方面实施例所述的方法。
为了实现上述目的，本申请第四方面实施例提出了一种非临时性计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时，实现如本申请第一方面实施例所述的方法。
综上所述，本申请实施例提供的用于扫描光场成像系统的镜头标定方法、镜头标定装置、计算机设备和非临时性计算机可读存储介质，该方案包括：采集包含不同角度信息的多个光场原始图像；将多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取图像对不同阶像差的全局分布；根据图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定。由此可知，通过本申请实施例可以根据获得光学镜头的不同阶像差的全局分布生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定，进而快速通过计算方式获得衍射极限成像结果，实现全局的像差矫正需要花费大量的计算资源与计算时间。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为本申请实施例1所提供的一种用于扫描光场成像系统的镜头标定方法的流程图；
图2为本申请实施例2所提供的一种用于扫描光场成像系统的镜头标定方法的流程图；以及
图3为本申请实施例3所提供的一种用于扫描光场成像系统的镜头标定装置的结构示意图。
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
为了便于本领域技术人员更好的理解本申请实施例，现简单介绍一下影响成像系统成像质量或成像分辨率的三方面限制条件，详情如下：
决定成像系统成像质量或成像分辨率的三方面限制条件如下：一是像感器的采样率：普遍流行的像感器时下主要分为CCD和CMOS两类，将像素数目做多，像元尺寸做小都有利于更高清的分辨率图像生成；二是系统的光学衍射极限：根据瑞利分辨率判据，成像系统中各透镜的数值孔径决定了系统的分辨能力；三是像差，像差普遍存在于自然环境中，比如大气的散射等。再者，由于镜头属于人工磨制的器件，会存在与理论光学中的理想透镜模型的偏差，也就是说，像差也存在于镜头之中；除此之外，随着镜头尺寸的逐渐增大，理想光学系统中的近轴光学理论不再适用，离轴光线的轨迹难以像近轴光线那样易于预测。总而言之，以上三点限制了光学成像系统的成像能力，阻碍了十亿像素成像的进一步发展。
在本申请背景技术中已介绍象差的存在，导致无论如何增加像感器的像素数目或是镜头的数值孔径，图像都很难进一步提升分辨率与清晰度。尤其对于高像素数目成像，比如亿级像素成像等存在全局不一致的像差，像差一致的等晕区只对应像素数目很少的图像局部，因此实现全局的像差矫正需要花费大量的计算资源与计算时间。换言之，方案都没有实际上解决像差的问题，可以达到的最佳成像效果也远远小于光学衍射极限性能。
下面参考附图描述本申请实施例的用于扫描光场成像系统的镜头标定方法和装置。
实施例1
图1为本申请实施例所提供的一种用于扫描光场成像系统的镜头标定方法的流程图。
针对上述问题，本申请实施例提供了用于扫描光场成像系统的镜头标定方法，以减少多个小区域像差矫正的时间和复杂度，从实际上解决象差的问题，使成像效果尽可能的接近光学衍射极限性能，如图1所示，该方法包括以下步骤：
步骤110，采集包含不同角度信息的多个光场原始图像。
在本申请实施例中，通过光场系统或扫描光场系统采集包含不同角度信息的光场原始图像，其中，不同角度信息的光场原始图像为不同角度和不同空间的图像。
步骤120，将多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取图像对不同阶像差的全局分布。
进一步地，本申请实施例中的步骤120，具体包括以下步骤：
步骤121，对图像对进行非刚性配准或畸变拉伸处理，得到图像对的像差分布。
具体而言，本申请通过将中心视角图像作为范例，将范例与非中心视角图像作为图像对分别进行两两的非刚性配准或畸变拉伸，可以实现根据坐标的变换情况获得不同视角图像的像差分布。
步骤122，根据图像对的像差分布，获取图像对不同阶像差的全局分布。
在本申请实施例中，可以通过对图像对的相差分布进行拟合处理，来获取图像对不同阶相差的全局分布。具体而言，根据不同视角的像差分布经过泽尼克多项式拟合或其它拟合方式获得不同阶像差的全局分布情况。由此获得的图像对不同阶相差的全局分布更为精准，为实现对镜头的精准标定做铺垫，进而从根本解决像差问题，使成像效果尽可能的接近光学衍射极限性能。
步骤130，根据图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定。
综上所述，本申请实施例通过对由光场成像系统或扫描光场成像系统采集获得的不同角度和空间的信息图像进行处理划分，得到中心视角图像和非中心视角图像，以中心角度作为范例，将范例与非中心视角图像作为图像对，使用全局的非刚性配准估计获得不同视角下对应的视差分布图，以此恢复光学镜头的系统像差分布。由此，可以获得光学镜头的系统像差分布，进而快速通过计算方式获得衍射极限成像结果，减少计算的时间复杂度和资源的浪费。
实施例2
图2为本申请实施例所提供的一种用于扫描光场成像系统的镜头标定方法的流程图。
针对上述问题，本申请实施例提供了用于扫描光场成像系统的镜头标定方法，以减少多个小区域像差矫正的时间和复杂度，从实际上解决象差的问题，使成像效果尽可能的接近光学衍射极限性能，如图2所示，该方法包括以下步骤：
步骤210，采集包含不同角度信息的多个光场原始图像。
步骤220，将多个光场原始图像处理为具有不同角度信息的三维图像堆栈。
步骤230，将多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶相差的全局分布。
步骤240，根据图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定。
由于本申请实施例中的步骤210、230和240分别于本申请实施例1的步骤110、120和130相同，为了避免重复，在此不再赘述，现只针对步骤S220进行进一步说明，
进一步地，本申请实施例中的步骤220，通过重排列的方法，将光场原始图像变换成具有不同角度信息的三维图像堆栈。具体而言，本申请实施例按序对光场原始图像进行排序，并将排序后的光场原始图像临时存储在三维图像堆栈中，有利于后续步骤的处理，比如，三维图像堆栈的设计有利于步骤230中将多个光场原始图像划分为中心视角图像和非中心视角图像的处理速度，由于三维图像堆栈是暂时存放排序后的光场原始图像，且有地址，数据处理完成后能快速释放内存。
综上所述，本申请实施例通过对由光场成像系统或扫描光场成像系统采集获得的不同角度和空间的信息图像进行处理划分，得到中心视角图像和非中心视角图像，以中心角度作为范例，将范例与非中心视角图像作为图像对，使用全局的非刚性配准估计获得不同视角下对应的视差分布图，以此恢复光学镜头的系统像差分布。由此，可以获得光学镜头的系统像差分布，进而快速通过计算方式获得衍射极限成像结果，减少计算的时间复杂度和资源的浪费。
实施例3
为了实现上述实施例，本申请还提出一种用于扫描光场成像系统的镜头标定装置。
图3为本申请实施例提供的一种用于扫描光场成像系统的镜头标定装置的结构示意图。
如图3所示，本申请实施例提供的用于扫描光场成像系统的镜头标定装置，包括：
光场图像采集模块10，用于采集包含不同角度信息的多个光场原始图像；
光场图像处理模块20，用于将多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取图像对不同阶相差的全局分布；
光场镜头标定模块30，用于根据图像对不同阶相差的全局分布，生成非全局一致的点扩散函数，并根据非全局一致的点扩散函数的求解对镜头进行标定。
进一步地，本申请实施例，还包括：光场图像初始处理模块，用于将所述多个光场原始图像处理为具有不同角度信息的三维图像堆栈。
进一步地，在本申请实施例中，光场图像处理模块，具体包括：
第一光场图像处理单元，用于对图像对进行非刚性配准或畸变拉伸处理，得到图像对的相差分布；
第二光场图像处理单元，用于根据图像对的相差分布，获取图像对不同阶相差的全局分布。
为了实现上述实施例，本申请还提出一种计算机设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现上述实施例所述的方法。
为了实现上述实施例，本申请还提出一种非临时性计算机可读存储介质，所述计算机程序被处理器执行时实现上述实施例所述的方法。
为了实现上述实施例，本申请还提出一种计算机程序产品，当所述计算机程序产品中的指令处理器执行时，执行一种基于人工智能的方法，所述方法包括：。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行 系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。如，如果用硬件来实现和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。
Claims (9)
- 一种用于扫描光场成像系统的镜头标定方法，其特征在于，包括：采集包含不同角度信息的多个光场原始图像；将所述多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶像差的全局分布；根据所述图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据所述非全局一致的点扩散函数的求解对镜头进行标定。
- 如权利要求书1所述的用于扫描光场成像系统的镜头标定方法，其特征在于，在所述采集包含不同角度信息的多个光场原始图像之后，还包括：将所述多个光场原始图像处理为具有不同角度信息的三维图像堆栈。
- 如权利要求书1或2所述的用于扫描光场成像系统的镜头标定方法，其特征在于，所述将所述多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶像差的全局分布，具体包括：对所述图像对进行非刚性配准或畸变拉伸处理，得到所述图像对的像差分布；根据所述图像对的像差分布，获取所述图像对不同阶像差的全局分布。
- 如权利要求书3所述的用于扫描光场成像系统的镜头标定方法，其特征在于，根据所述图像对的像差分布，获取所述图像对不同阶像差的全局分布，具体包括：对所述图像对的像差分布进行拟合处理，以获取所述图像对不同阶像差的全局分布。
- 一种用于扫描光场成像系统的镜头标定装置，其特征在于，包括：光场图像采集模块，用于采集包含不同角度信息的多个光场原始图像；光场图像处理模块，用于将所述多个光场原始图像划分为中心视角图像和非中心视角图像，将每个中心视角图像与每个非中心视角图像两两配对形成图像对，并获取所述图像对不同阶像差的全局分布；光场镜头标定模块，用于根据所述图像对不同阶像差的全局分布，生成非全局一致的点扩散函数，并根据所述非全局一致的点扩散函数的求解对镜头进行标定。
- 如权利要求书5所述的用于扫描光场成像系统的镜头标定装置，其特征在于，所述镜头标定装置，还包括：光场图像初始处理模块，用于将所述多个光场原始图像处理为具有不同角度信息的三维图像堆栈。
- 如权利要求书5或6所述的用于扫描光场成像系统的镜头标定装置，其特征在于，所述光场图像处理模块，具体包括：第一光场图像处理单元，用于对所述图像对进行非刚性配准或畸变拉伸处理，得到所述图像对的像差分布；第二光场图像处理单元，用于根据所述图像对的像差分布，获取所述图像对不同阶像差的全局分布。
- 一种计算机设备，其特征在于，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现如权利要求1-4中任一所述的方法。
- 一种非临时性计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-4中任一所述的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110792338.5 | 2021-07-14 | ||
| CN202110792338.5A CN113256741B (zh) | 2021-07-14 | 2021-07-14 | 用于扫描光场成像系统的镜头标定方法及装置 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023284792A1 true WO2023284792A1 (zh) | 2023-01-19 |
Family
ID=77191203
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2022/105495 WO2023284792A1 (zh) | 2021-07-14 | 2022-07-13 | 用于扫描光场成像系统的镜头标定方法及装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113256741B (zh) |
| WO (1) | WO2023284792A1 (zh) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113256741B (zh) * | 2021-07-14 | 2021-10-22 | 清华大学 | 用于扫描光场成像系统的镜头标定方法及装置 |
| CN113487658B (zh) * | 2021-08-31 | 2021-11-16 | 清华大学 | 一种用于扫描光场成像系统的动态场景拍摄方法及装置 |
| CN114494258B (zh) * | 2022-04-15 | 2022-08-30 | 清华大学 | 镜头的像差预测和图像重建方法及装置 |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6567570B1 (en) * | 1998-10-30 | 2003-05-20 | Hewlett-Packard Development Company, L.P. | Optical image scanner with internal measurement of point-spread function and compensation for optical aberrations |
| CN102460444A (zh) * | 2009-06-09 | 2012-05-16 | 皇家飞利浦电子股份有限公司 | 用于对存储的图像进行排序的装置和方法 |
| CN103856723A (zh) * | 2014-02-25 | 2014-06-11 | 中国人民解放军国防科学技术大学 | 一种单透镜成像的psf快速标定方法 |
| CN104680491A (zh) * | 2015-02-28 | 2015-06-03 | 西安交通大学 | 一种基于深度神经网络的图像非均匀运动模糊去除方法 |
| CN111402127A (zh) * | 2020-02-18 | 2020-07-10 | 清华大学 | 基于光场信息去除光学像差的方法及装置 |
| CN112035774A (zh) * | 2020-09-01 | 2020-12-04 | 平安付科技服务有限公司 | 网络页面生成方法、装置、计算机设备及可读存储介质 |
| CN113256741A (zh) * | 2021-07-14 | 2021-08-13 | 清华大学 | 用于扫描光场成像系统的镜头标定方法及装置 |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107492127B (zh) * | 2017-09-18 | 2021-05-11 | 丁志宇 | 光场相机参数标定方法、装置、存储介质和计算机设备 |
| CN108776980B (zh) * | 2018-05-14 | 2021-07-09 | 南京工程学院 | 一种面向微透镜光场相机的标定方法 |
| CN109146794B (zh) * | 2018-09-29 | 2019-11-05 | 哈尔滨工业大学 | 一种光场图像旋转误差校正方法 |
| US11030776B2 (en) * | 2019-02-01 | 2021-06-08 | Molecular Devices (Austria) GmbH | Calibration of a light-field imaging system |
-
2021
- 2021-07-14 CN CN202110792338.5A patent/CN113256741B/zh active Active
-
2022
- 2022-07-13 WO PCT/CN2022/105495 patent/WO2023284792A1/zh unknown
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6567570B1 (en) * | 1998-10-30 | 2003-05-20 | Hewlett-Packard Development Company, L.P. | Optical image scanner with internal measurement of point-spread function and compensation for optical aberrations |
| CN102460444A (zh) * | 2009-06-09 | 2012-05-16 | 皇家飞利浦电子股份有限公司 | 用于对存储的图像进行排序的装置和方法 |
| CN103856723A (zh) * | 2014-02-25 | 2014-06-11 | 中国人民解放军国防科学技术大学 | 一种单透镜成像的psf快速标定方法 |
| CN104680491A (zh) * | 2015-02-28 | 2015-06-03 | 西安交通大学 | 一种基于深度神经网络的图像非均匀运动模糊去除方法 |
| CN111402127A (zh) * | 2020-02-18 | 2020-07-10 | 清华大学 | 基于光场信息去除光学像差的方法及装置 |
| CN112035774A (zh) * | 2020-09-01 | 2020-12-04 | 平安付科技服务有限公司 | 网络页面生成方法、装置、计算机设备及可读存储介质 |
| CN113256741A (zh) * | 2021-07-14 | 2021-08-13 | 清华大学 | 用于扫描光场成像系统的镜头标定方法及装置 |
Non-Patent Citations (1)
| Title |
|---|
| WU XIAO-TIAN, LÜ BO, LIU BO, YANG HANG: "Combined exposure computational imaging system and image restoration method", OPTICS AND PRECISION ENGINEERING, ZHONGGUO KEXUEYUAN CHANGCHUN JINGMI JIXIE YANJIUSUO, CHANGCHUN, CN, vol. 29, no. 2, 1 January 2021 (2021-01-01), CN , pages 452 - 462, XP093024052, ISSN: 1004-924X, DOI: 10.37188/OPE.20212902.0452 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113256741B (zh) | 2021-10-22 |
| CN113256741A (zh) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2023284792A1 (zh) | 用于扫描光场成像系统的镜头标定方法及装置 | |
| CN108122191B (zh) | 鱼眼图像拼接成全景图像和全景视频的方法及装置 | |
| US9286680B1 (en) | Computational multi-camera adjustment for smooth view switching and zooming | |
| JP4159986B2 (ja) | デジタル画像から変換された画像を計算するための方法およびシステム | |
| WO2020010945A1 (zh) | 图像处理方法和装置、电子设备、计算机可读存储介质 | |
| US11403739B2 (en) | Methods and apparatus for retargeting and prioritized interpolation of lens profiles | |
| CN109615663A (zh) | 全景视频校正方法及终端 | |
| CN105488810A (zh) | 一种聚焦光场相机内外参数标定方法 | |
| CN110345921B (zh) | 立体视场视觉测量及垂轴像差和轴向像差校正方法及系统 | |
| JP2009135921A (ja) | 画像投影装置及び画像投影方法 | |
| WO2023029520A1 (zh) | 一种用于扫描光场成像系统的动态场景拍摄方法及装置 | |
| CN102540477B (zh) | 双光路系统交互校正方法及使用该方法的光学系统 | |
| WO2019065260A1 (ja) | 情報処理装置、情報処理方法、及び、プログラム、並びに、交換レンズ | |
| EP2422524A1 (en) | Spatially-varying spectral response calibration data | |
| CN109163888A (zh) | 光学中心测试方法、装置和设备 | |
| US10230911B1 (en) | Preview generation for plenoptic imaging systems | |
| US11838643B2 (en) | Information processing method and electronic apparatus | |
| CN111757086A (zh) | 有源双目相机、rgb-d图像确定方法及装置 | |
| KR102253320B1 (ko) | 집적영상 현미경 시스템에서의 3차원 영상 디스플레이 방법 및 이를 구현하는 집적영상 현미경 시스템 | |
| CN108122199A (zh) | 一种全景相机的原始图像颜色调整方法及装置 | |
| Su et al. | Calibrating the orientation between a microlens array and a sensor based on projective geometry | |
| WO2023231139A1 (zh) | 元成像方法与系统 | |
| Oberdörster et al. | Correcting distortion and braiding of micro-images from multi-aperture imaging systems | |
| CN115272124A (zh) | 一种畸变图像校正方法、装置 | |
| CN115209000A (zh) | 用于遥感成像动态相差估计方法及系统 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22841425 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |