CN118154411A - Digital Adaptive Optics Architecture and System - Google Patents
Digital Adaptive Optics Architecture and System Download PDFInfo
- Publication number
- CN118154411A CN118154411A CN202410573482.3A CN202410573482A CN118154411A CN 118154411 A CN118154411 A CN 118154411A CN 202410573482 A CN202410573482 A CN 202410573482A CN 118154411 A CN118154411 A CN 118154411A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- aperture
- wavefront
- sampling rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 20
- 238000005070 sampling Methods 0.000 claims abstract description 67
- 230000004075 alteration Effects 0.000 claims abstract description 64
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims description 35
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 230000008707 rearrangement Effects 0.000 claims description 7
- 230000010354 integration Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000003287 optical effect Effects 0.000 abstract description 15
- 238000003384 imaging method Methods 0.000 abstract description 14
- 238000001514 detection method Methods 0.000 abstract description 11
- 230000015572 biosynthetic process Effects 0.000 abstract description 3
- 238000003786 synthesis reaction Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 101000827703 Homo sapiens Polyphosphoinositide phosphatase Proteins 0.000 description 2
- 102100023591 Polyphosphoinositide phosphatase Human genes 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 239000008358 core component Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 1
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
本发明公开了数字自适应光学架构与系统,包括对原始光场图像进行像素重排得到低空间采样率的子孔径图像;利用TIS算法对低空间采样率的子孔径图像进行处理得到高空间采样率的子孔径图像;利用由粗粒度到细粒度的波前梯度估计算法对高空间采样率的子孔径图像的空间偏移量进行估计得到第一波前梯度,并利用训练好的MLP模型将第一波前梯度转换为第一波前像差;基于第一波前像差对高空间采样率的子孔径图像进行相空间解卷积运算得到局部图像的图像重建结果。本发明能够进行高速宽视场波前检测,采用湍流诱导扫描算法提高采样率,使用非相干孔径合成算法实现去像差、高分辨成像。
The present invention discloses a digital adaptive optical architecture and system, including rearranging pixels of an original light field image to obtain a sub-aperture image with a low spatial sampling rate; processing the sub-aperture image with a low spatial sampling rate using a TIS algorithm to obtain a sub-aperture image with a high spatial sampling rate; estimating the spatial offset of the sub-aperture image with a high spatial sampling rate using a wavefront gradient estimation algorithm from coarse granularity to fine granularity to obtain a first wavefront gradient, and converting the first wavefront gradient into a first wavefront aberration using a trained MLP model; performing a phase space deconvolution operation on the sub-aperture image with a high spatial sampling rate based on the first wavefront aberration to obtain an image reconstruction result of a local image. The present invention can perform high-speed wide-field wavefront detection, use a turbulence-induced scanning algorithm to increase the sampling rate, and use an incoherent aperture synthesis algorithm to achieve aberration removal and high-resolution imaging.
Description
技术领域Technical Field
本发明涉及光场成像技术领域,特别是涉及数字自适应光学架构与系统。The present invention relates to the field of light field imaging technology, and in particular to a digital adaptive optics architecture and system.
背景技术Background technique
光学像差普遍存在于自然环境与成像系统中,大气湍流运动、不完美透镜等都会导致光线轨迹发生偏折,从而产生光学像差,导致图像模糊和信号失真。自适应光学(AO)技术旨在实时测量和校正大气湍流或其他介质引起的光学像差,这种技术主要应用于天文观测、激光通信、视觉科学等领域,可以显著提高图像质量和信号稳定性。Optical aberrations are common in natural environments and imaging systems. Atmospheric turbulence and imperfect lenses can cause light trajectories to bend, resulting in optical aberrations, blurry images and distorted signals. Adaptive optics (AO) technology aims to measure and correct optical aberrations caused by atmospheric turbulence or other media in real time. This technology is mainly used in astronomical observations, laser communications, visual science and other fields, and can significantly improve image quality and signal stability.
AO系统的核心部件包括波前传感器、可变形镜和先进的控制系统。波前传感器用于实时检测进入系统的光波前的畸变情况;可变形镜则根据波前传感器的测量结果进行形状调整,以补偿光波前的畸变;控制系统则协调波前传感器和变形镜的工作,确保畸变得到有效校正。The core components of the AO system include wavefront sensors, deformable mirrors, and advanced control systems. The wavefront sensor is used to detect the distortion of the light wavefront entering the system in real time; the deformable mirror adjusts its shape according to the measurement results of the wavefront sensor to compensate for the distortion of the light wavefront; the control system coordinates the work of the wavefront sensor and the deformable mirror to ensure that the distortion is effectively corrected.
现有的AO系统主要在视场范围、系统复杂度两个方面有一定局限性。光学像差具有空间非一致性,现有的AO系统主要采用Shack-Hartmann波前传感器,只能针对较小视场(FOV)内的单一光学像差进行测量,无法解决空间非一致问题。先进的AO技术,如MCAO和GLAO,采用多传感器方案,扩大视场范围到数个角分,但牺牲了一定的波前探测精度,导致系统复杂度提高。Existing AO systems have certain limitations in terms of field of view and system complexity. Optical aberrations have spatial inconsistency. Existing AO systems mainly use Shack-Hartmann wavefront sensors, which can only measure single optical aberrations within a small field of view (FOV) and cannot solve the problem of spatial inconsistency. Advanced AO technologies, such as MCAO and GLAO, use multi-sensor solutions to expand the field of view to several arc minutes, but sacrifice a certain degree of wavefront detection accuracy, resulting in increased system complexity.
发明内容Summary of the invention
本发明旨在至少在一定程度上解决相关技术中的技术问题之一。The present invention aims to solve one of the technical problems in the related art at least to a certain extent.
为此,本发明提出了一种数字自适应图像重建方法,解决宽视场、高分辨率成像,在光学像差较大的问题,实现高分辨成像。To this end, the present invention proposes a digital adaptive image reconstruction method to solve the problem of wide field of view, high-resolution imaging and achieve high-resolution imaging when optical aberrations are large.
本发明的另一个目的在于提出一种数字自适应图像重建系统。Another object of the present invention is to provide a digital adaptive image reconstruction system.
为达上述目的,本发明一方面提出一种数字自适应图像重建方法,包括:To achieve the above object, the present invention provides a digital adaptive image reconstruction method, comprising:
对原始光场图像进行像素重排得到低空间采样率的子孔径图像;Rearrange the pixels of the original light field image to obtain a sub-aperture image with a low spatial sampling rate;
利用TIS算法对所述低空间采样率的子孔径图像进行处理得到高空间采样率的子孔径图像;Processing the sub-aperture image with low spatial sampling rate by using TIS algorithm to obtain a sub-aperture image with high spatial sampling rate;
利用由粗粒度到细粒度的波前梯度估计算法对所述高空间采样率的子孔径图像的空间偏移量进行估计得到第一波前梯度,并利用训练好的MLP模型将所述第一波前梯度转换为第一波前像差;Using a wavefront gradient estimation algorithm from coarse granularity to fine granularity to estimate the spatial offset of the sub-aperture image with a high spatial sampling rate to obtain a first wavefront gradient, and using a trained MLP model to convert the first wavefront gradient into a first wavefront aberration;
基于所述第一波前像差对所述高空间采样率的子孔径图像进行相空间解卷积运算得到局部图像的图像重建结果。A phase space deconvolution operation is performed on the sub-aperture image with a high spatial sampling rate based on the first wavefront aberration to obtain an image reconstruction result of a local image.
本发明实施例的数字自适应图像重建方法还可以具有以下附加技术特征:The digital adaptive image reconstruction method of the embodiment of the present invention may also have the following additional technical features:
在本发明的一个实施例中,在得到低空间采样率的子孔径图像之后,所述方法还包括:In one embodiment of the present invention, after obtaining the sub-aperture image with a low spatial sampling rate, the method further includes:
获取不同时刻、同一子孔径的所述低空间采样率的子孔径图像的图像序列;Acquire an image sequence of sub-aperture images of the same sub-aperture at different times and with the low spatial sampling rate;
根据所述图像序列之间的相对偏移量计算参考时刻的各子孔径图像之间的相对坐标。The relative coordinates between the sub-aperture images at the reference time are calculated according to the relative offset between the image sequences.
在本发明的一个实施例中,利用TIS算法对所述低空间采样率的子孔径图像进行处理得到高空间采样率的子孔径图像,包括:In one embodiment of the present invention, the sub-aperture image with a low spatial sampling rate is processed by using a TIS algorithm to obtain a sub-aperture image with a high spatial sampling rate, including:
通过TIS算法对所述图像序列中的多个子孔径图像按照参考时刻的子孔径图像的相对坐标进行拼接得到坐标拼接结果;splicing the multiple sub-aperture images in the image sequence according to the relative coordinates of the sub-aperture images at a reference time using a TIS algorithm to obtain a coordinate splicing result;
基于所述坐标拼接结果得到参考时刻的高空间采样率的子孔径图像。A sub-aperture image with a high spatial sampling rate at a reference time is obtained based on the coordinate stitching result.
在本发明的一个实施例中,利用训练好的MLP模型将所述第一波前梯度转换为第一波前像差,包括:In one embodiment of the present invention, using a trained MLP model to convert the first wavefront gradient into a first wavefront aberration includes:
获取由粗粒度到细粒度的波前梯度估计算法得到的第二波前梯度;Obtaining a second wavefront gradient obtained by a coarse-grained to fine-grained wavefront gradient estimation algorithm;
利用二维积分算法将所述第二波前梯度转化为第二波前像差,以拟合第二波前像差对应的待拟合的Zernike多项式系数;The second wavefront gradient is converted into a second wavefront aberration by using a two-dimensional integration algorithm, so as to fit the Zernike polynomial coefficients to be fitted corresponding to the second wavefront aberration;
利用第二波前梯度训练MLP模型以得到训练好的MLP模型;其中,模型输入层对应第二波前梯度的维度,模型输出层的节点数等于待拟合的Zernike多项式系数的数量;The MLP model is trained using the second wavefront gradient to obtain a trained MLP model; wherein the model input layer corresponds to the dimension of the second wavefront gradient, and the number of nodes in the model output layer is equal to the number of Zernike polynomial coefficients to be fitted;
将第一波前梯度输入所述训练好的MLP模型输出相应的Zernike多项式系数,以计算所述第一波前像差。The first wavefront gradient is input into the trained MLP model and the corresponding Zernike polynomial coefficients are output to calculate the first wavefront aberration.
在本发明的一个实施例中,在得到局部图像的图像重建结果之后,所述方法,还包括:In one embodiment of the present invention, after obtaining the image reconstruction result of the local image, the method further includes:
对每个局部图像进行重建得到所有局部图像的图像重建结果;Reconstruct each local image to obtain image reconstruction results of all local images;
对所述所有局部图像的图像重建结果进行拼接以得到全局图像的图像重建结果。The image reconstruction results of all the local images are spliced to obtain an image reconstruction result of the global image.
为达上述目的,本发明另一方面提出一种时空角融合动态光场重建系统,包括:To achieve the above object, the present invention provides a spatiotemporal angular fusion dynamic light field reconstruction system, comprising:
图像像素重排模块,用于对原始光场图像进行像素重排得到低空间采样率的子孔径图像;An image pixel rearrangement module, used for rearranging pixels of the original light field image to obtain a sub-aperture image with a low spatial sampling rate;
孔径图像处理模块,用于利用TIS算法对所述低空间采样率的子孔径图像进行处理得到高空间采样率的子孔径图像;An aperture image processing module, used for processing the sub-aperture image with low spatial sampling rate by using TIS algorithm to obtain a sub-aperture image with high spatial sampling rate;
波前像差转换模块,用于利用由粗粒度到细粒度的波前梯度估计算法对所述高空间采样率的子孔径图像的空间偏移量进行估计得到第一波前梯度,并利用训练好的MLP模型将所述第一波前梯度转换为第一波前像差;A wavefront aberration conversion module, used to estimate the spatial offset of the sub-aperture image with a high spatial sampling rate by using a wavefront gradient estimation algorithm from coarse granularity to fine granularity to obtain a first wavefront gradient, and convert the first wavefront gradient into a first wavefront aberration by using a trained MLP model;
局部图像重建模块,用于基于所述第一波前像差对所述高空间采样率的子孔径图像进行相空间解卷积运算得到局部图像的图像重建结果。A local image reconstruction module is used to perform a phase space deconvolution operation on the sub-aperture image with a high spatial sampling rate based on the first wavefront aberration to obtain an image reconstruction result of a local image.
本发明实施例的数字自适应图像重建方法与系统,通过即插即用的宽视场波前传感器进行高速宽视场波前检测,采用湍流诱导扫描算法提高采样率,使用非相干孔径合成算法,可以实现低成本、宽视场、高分辨率成像。The digital adaptive image reconstruction method and system of the embodiment of the present invention performs high-speed wide-field-of-view wavefront detection through a plug-and-play wide-field-of-view wavefront sensor, adopts a turbulence-induced scanning algorithm to improve the sampling rate, and uses an incoherent aperture synthesis algorithm to achieve low-cost, wide-field-of-view, and high-resolution imaging.
本发明附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本发明的实践了解到。Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
本发明上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
图1是根据本发明实施例的数字自适应图像重建方法的流程图;1 is a flow chart of a digital adaptive image reconstruction method according to an embodiment of the present invention;
图2是根据本发明实施例的数字自适应图像重建方法的框架图;FIG2 is a framework diagram of a digital adaptive image reconstruction method according to an embodiment of the present invention;
图3是根据本发明实施例的由粗粒度到细粒度的波前梯度估计算法流程图;3 is a flow chart of a wavefront gradient estimation algorithm from coarse granularity to fine granularity according to an embodiment of the present invention;
图4是根据本发明实施例的 ISA算法流程图;FIG4 is a flow chart of an ISA algorithm according to an embodiment of the present invention;
图5是根据本发明实施例的数字自适应图像重建系统的结构示意图。FIG. 5 is a schematic structural diagram of a digital adaptive image reconstruction system according to an embodiment of the present invention.
具体实施方式Detailed ways
需要说明的是,在不冲突的情况下,本发明中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本发明。It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
为了使本技术领域的人员更好地理解本发明方案,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分的实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都应当属于本发明保护的范围。In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
下面参照附图描述根据本发明实施例提出的数字自适应图像重建方法和系统。The digital adaptive image reconstruction method and system according to the embodiments of the present invention are described below with reference to the accompanying drawings.
图1是本发明实施例的数字自适应图像重建方法的流程图。FIG. 1 is a flow chart of a digital adaptive image reconstruction method according to an embodiment of the present invention.
如图1所示,该方法包括但不限于以下步骤:As shown in FIG1 , the method includes but is not limited to the following steps:
S1,对原始光场图像进行像素重排得到低空间采样率的子孔径图像;S1, rearrange the pixels of the original light field image to obtain a sub-aperture image with a low spatial sampling rate;
S2,利用TIS算法对低空间采样率的子孔径图像进行处理得到高空间采样率的子孔径图像;S2, using the TIS algorithm to process the sub-aperture image with a low spatial sampling rate to obtain a sub-aperture image with a high spatial sampling rate;
S3,利用由粗粒度到细粒度的波前梯度估计算法对高空间采样率的子孔径图像的空间偏移量进行估计得到第一波前梯度,并利用训练好的MLP模型将第一波前梯度转换为第一波前像差;S3, using a wavefront gradient estimation algorithm from coarse-grained to fine-grained to estimate the spatial offset of the sub-aperture image with a high spatial sampling rate to obtain a first wavefront gradient, and using the trained MLP model to convert the first wavefront gradient into a first wavefront aberration;
S4,基于第一波前像差对高空间采样率的子孔径图像进行相空间解卷积运算得到局部图像的图像重建结果。S4, performing a phase space deconvolution operation on the sub-aperture image with a high spatial sampling rate based on the first wavefront aberration to obtain an image reconstruction result of the local image.
可以理解的是,本发明提出了一种新的DAO架构,其核心部件是即插即用的宽视场波前传感器(WWS)。硬件方面,WWS只是在传统2D CMOS图像传感器的基础上做较小改动,具体来说,只需将微透镜阵列(MLA)置于原始像平面,再将2D CMOS图像传感器置于MLA的后焦面上即可,系统复杂度和实现成本较低;算法方面,本发明采用一种由粗粒度到细粒度的波前梯度估计算法,结合多层感知机(MLP)映射,实现高速的(30Hz)、宽视场(超过1100角秒)的空间非一致像差探测。It is understandable that the present invention proposes a new DAO architecture, the core component of which is a plug-and-play wide-field wavefront sensor (WWS). In terms of hardware, WWS is just a minor modification of the traditional 2D CMOS image sensor. Specifically, it is only necessary to place the microlens array (MLA) on the original image plane and then place the 2D CMOS image sensor on the back focal plane of the MLA. The system complexity and implementation cost are low. In terms of algorithms, the present invention adopts a wavefront gradient estimation algorithm from coarse-grained to fine-grained, combined with multi-layer perceptron (MLP) mapping, to achieve high-speed (30Hz), wide-field (over 1100 arc seconds) spatial non-uniform aberration detection.
图2为本发明的数字自适应图像重建方法的框架图,如图2所示,包括预处理阶段、湍流诱导扫描(TIS)算法流程、像差波前检测和非相干孔径合成(ISA)算法流程。FIG2 is a framework diagram of the digital adaptive image reconstruction method of the present invention, which includes a preprocessing stage, a turbulence induced scanning (TIS) algorithm flow, an aberration wavefront detection and an incoherent aperture synthesis (ISA) algorithm flow as shown in FIG2 .
在本发明的一个实施例中,预处理阶段即像素重排操作。In one embodiment of the present invention, the pre-processing stage is a pixel rearrangement operation.
具体地,将原始光场图像转化为对应的数个子孔径图像,每个子孔径图像代表不同角度光场的投影。可以通过尺寸调整和旋转操作矫正CMOS和MLA之间的微小偏差。Specifically, the original light field image is converted into several corresponding sub-aperture images, each of which represents the projection of the light field at a different angle. The slight deviation between CMOS and MLA can be corrected by resizing and rotating operations.
在本发明的一个实施例中,TIS即图2左侧分支,实现了亚像素级的图像对齐和拼接。In one embodiment of the present invention, TIS, i.e., the left branch of FIG. 2 , implements sub-pixel level image alignment and stitching.
可以理解的是,光场成像用2D CMOS记录4D光场信息,在记录角度信息的同时牺牲了空间采样率,因此预处理得到的子孔径图像的空间采样率过低,TIS即用于恢复其采样率。It is understandable that light field imaging uses 2D CMOS to record 4D light field information, sacrificing the spatial sampling rate while recording the angle information. Therefore, the spatial sampling rate of the sub-aperture image obtained by preprocessing is too low, and TIS is used to restore its sampling rate.
具体地,首先估计不同时刻、同一子孔径的图像序列的相对偏移,得到各子孔径图像之间的相对坐标;之后通过TIS算法对序列中的多个子孔径图像按照参考时刻的子孔径图像的相对坐标进行拼接,即可得到参考时刻的、高空间采样率的子孔径图像。采用TIS算法得到的图像质量较高,不具有明显的动态伪影。Specifically, the relative offset of the image sequence of the same sub-aperture at different times is first estimated to obtain the relative coordinates between the sub-aperture images; then, the multiple sub-aperture images in the sequence are spliced according to the relative coordinates of the sub-aperture images at the reference time by the TIS algorithm, so as to obtain the sub-aperture image at the reference time with a high spatial sampling rate. The image obtained by the TIS algorithm has high quality and no obvious dynamic artifacts.
本发明实施例的TIS算法,实现了亚像素级的图像对齐和拼接,算法输入是多张低空间采样率的图片,输出是一张高空间采样率的图片。空间采样率较低时,快速变化的大气湍流导致的光学像差可视为空间非一致的快速扫描,TIS算法可以融合扫描信息,提高空间采样率。The TIS algorithm of the embodiment of the present invention realizes sub-pixel image alignment and stitching. The algorithm input is multiple pictures with low spatial sampling rate, and the output is a picture with high spatial sampling rate. When the spatial sampling rate is low, the optical aberration caused by the rapidly changing atmospheric turbulence can be regarded as a fast scan of spatial inconsistency. The TIS algorithm can fuse the scanning information and improve the spatial sampling rate.
TIS算法流程可用公式表示如下:The TIS algorithm process can be expressed by the following formula:
式中,表示用于TIS的图像帧数(要求为奇数,则参考时刻/>),表示时刻/>对应的第/>个子孔径图像,/>表示参考时刻的子孔径图像的TIS结果,/>表示图像平面上的二维坐标,/>表示/>与/>之间的相对坐标偏移,表示对子孔径图像/>在坐标位置/>采用双三次插值(BI)的结果,/>表示均方误差,/>表示散点插值(SI)。In the formula, Indicates the number of image frames used for TIS (required to be an odd number, then the reference time/> ), Indicates time /> The corresponding /> sub-aperture images, /> represents the TIS result of the sub-aperture image at the reference time,/> represents the two-dimensional coordinates on the image plane,/> Indicates/> With/> The relative coordinate offset between Represents the sub-aperture image/> At coordinate position /> The result of using bicubic interpolation (BI),/> represents the mean square error, /> Represents scattered point interpolation (SI).
式(1)的目的是计算不同时刻子孔径图像的相对坐标偏移,该优化问题采用随机梯度下降算法(SGD)求解;式(2)的含义是将横向坐标随机分布的多张子孔径图像采用散点插值的方式求出其对应在均匀网格位置上的灰度值并将其融合为一张高分辨率的子孔径图像。The purpose of formula (1) is to calculate the relative coordinate offset of the sub-aperture images at different times. The optimization problem is solved by the stochastic gradient descent algorithm (SGD). The meaning of formula (2) is to use the scattered point interpolation method to obtain the grayscale values corresponding to the uniform grid positions of multiple sub-aperture images with randomly distributed lateral coordinates and fuse them into a high-resolution sub-aperture image.
在本发明的一个实施例中,像差波前检测即图2右侧分支。In one embodiment of the present invention, the aberration wavefront detection is the right branch of FIG. 2 .
具体地,用由粗粒度到细粒度的波前梯度估计算法,对特定时刻的多个不同子孔径图像的空间非一致偏移进行估计,该偏移量即为波前梯度场。“由粗粒度到细粒度”的优化策略可以显著提高算法的稳健性和运行速度。算法具体流程如图3所示,下面简要介绍其流程:Specifically, the wavefront gradient estimation algorithm from coarse granularity to fine granularity is used to estimate the spatial non-uniform offset of multiple sub-aperture images at a specific moment. The offset is the wavefront gradient field. The "coarse granularity to fine granularity" optimization strategy can significantly improve the robustness and running speed of the algorithm. The specific process of the algorithm is shown in Figure 3, and its process is briefly described below:
(1)该算法用于估计从目标子孔径图像到参考子孔径图像/>的空间非一致横向偏移,具体到本发明,对同一时刻拍摄得到的不同子孔径图像,选定某一参考子孔径图像/>,估计其他所有子孔径图像/>到/>的空间非一致横向偏移,将其按照孔径面的位置拼接起来就得到了空间非一致的波前梯度场;(1) This algorithm is used to estimate the target sub-aperture image To the reference sub-aperture image/> In the present invention, for different sub-aperture images captured at the same time, a reference sub-aperture image is selected. , estimate all other sub-aperture images/> To/> The spatially non-uniform lateral offsets are spliced together according to the position of the aperture plane to obtain the spatially non-uniform wavefront gradient field;
(2)首先参考图3上半部分的流程对与/>进行标准化,以去除其强度分布,避免强度分布不一致导致估计算法失效,之后将在标准化子孔径图像/>与/>上估计波前梯度,优化问题表述如下:(2) First, refer to the process in the upper part of Figure 3. With/> Standardization is performed to remove its intensity distribution to avoid inconsistent intensity distribution causing failure of the estimation algorithm. After that, the sub-aperture image is normalized. With/> The wavefront gradient is estimated and the optimization problem is expressed as follows:
式中表示均方误差,/>表示双三次插值,/>表示双三次上采样,/>表示图像平面上的二维坐标(大小与图像相同),/>表示图像平面上的二维坐标(大小与局部空间一致区域的个数一致,比图像尺寸小),/>表示横向偏移量;In the formula represents the mean square error, /> represents bicubic interpolation, /> represents bicubic upsampling,/> represents the two-dimensional coordinates on the image plane (same size as the image),/> Represents the two-dimensional coordinates on the image plane (the size is the same as the number of local spatial consistent regions, smaller than the image size), /> Indicates the lateral offset;
(3)参考图3下半部分的流程,首先优化粗粒度波前梯度,之后将/>的值用于初始化细粒度波前梯度/>,再进一步优化/>,待迭代过程收敛后,/>即所求的/>。粗粒度与细粒度波前梯度均采用随机梯度下降算法进行优化,优化器采用Adam。(3) Referring to the process in the lower part of Figure 3, first optimize the coarse-grained wavefront gradient , then /> The value of is used to initialize the fine-grained wavefront gradient/> , and further optimize/> , after the iterative process converges,/> That is what you want/> Both coarse-grained and fine-grained wavefront gradients are optimized using the stochastic gradient descent algorithm, and the optimizer is Adam.
进一步地,得到波前梯度之后,可以采用积分算法由波前梯度得到波前像差。在实际应用场景中,积分算法速度较慢,无法满足实时观测需求,为此,本发明设计并训练一个MLP模型以实现梯度到像差的快速转换。采用有监督模式训练MLP,并用训练好的MLP代替积分过程进行推理,具体流程如下:Furthermore, after obtaining the wavefront gradient, the wavefront aberration can be obtained from the wavefront gradient using an integral algorithm. In practical application scenarios, the integral algorithm is slow and cannot meet the real-time observation requirements. For this reason, the present invention designs and trains an MLP model to achieve rapid conversion from gradient to aberration. The MLP is trained in a supervised mode, and the trained MLP is used to replace the integral process for reasoning. The specific process is as follows:
1)准备输入数据:输入数据即由粗粒度到细粒度的波前梯度估计算法计算得到的第二波前梯度;1) Prepare input data: The input data is the second wavefront gradient calculated by the coarse-grained to fine-grained wavefront gradient estimation algorithm;
2)准备待拟合输出:对第二波前梯度运行二维积分算法得到第二波前像差,再用最小二乘法拟合第二波前像差对应的Zernike多项式系数,这些Zernike多项式系数即为待拟合输出;2) Prepare the output to be fitted: Run a two-dimensional integration algorithm on the second wavefront gradient to obtain the second wavefront aberration, and then use the least squares method to fit the Zernike polynomial coefficients corresponding to the second wavefront aberration. These Zernike polynomial coefficients are the output to be fitted;
3)初始化:初始化MLP网络,采用双隐层的结构,输入层即波前梯度,第一隐层有300个节点,第二隐层有500个节点,输出层即Zernike多项式系数,参数随机初始化;3) Initialization: Initialize the MLP network with a double hidden layer structure. The input layer is the wavefront gradient. The first hidden layer has 300 nodes, the second hidden layer has 500 nodes, and the output layer is the Zernike polynomial coefficients. The parameters are randomly initialized.
4)训练:损失函数采用MSE,优化器采用Adam,迭代训练直至MSE损失收敛即可;4) Training: The loss function uses MSE, the optimizer uses Adam, and the training is iterated until the MSE loss converges;
5)推理:对MLP输入第一波前梯度,则MLP可以输出第一波前像差的Zernike多项式系数。5) Reasoning: If the first wavefront gradient is input to the MLP, the MLP can output the Zernike polynomial coefficients of the first wavefront aberration.
由此,实验发现,MLP模型能够较好的拟合训练数据,并在验证数据上的表现与训练数据十分接近,具有良好的泛化能力。Therefore, the experiment found that the MLP model can fit the training data well, and its performance on the validation data is very close to the training data, and it has good generalization ability.
在本发明的一个实施例中,ISA的算法具体流程如图4所示,下面简要介绍其流程:In one embodiment of the present invention, the specific process of the ISA algorithm is shown in FIG4 , and the process is briefly described below:
考虑到像差的空间非一致性,采用分块解卷积的方式进行重建,解卷积方法采用相空间解卷积算法;Considering the spatial inconsistency of aberrations, the reconstruction is performed by block deconvolution, and the deconvolution method uses the phase space deconvolution algorithm;
对于局部区域,认为像差一致,因此可以得到的波前像差对得到的TIS后局部子孔径图像运行相空间解卷积算法,得到局部图像的重建结果;For the local area, the aberration is considered to be consistent, so the obtained wavefront aberration is used to run the phase space deconvolution algorithm on the obtained local sub-aperture image after TIS to obtain the reconstruction result of the local image;
对每个局部进行重建,得到所有局部区域的重建结果,只需将其拼接即可得到全局图像的重建结果,实现了全局分辨率的提升。By reconstructing each local area and obtaining the reconstruction results of all local areas, the reconstruction result of the global image can be obtained by simply splicing them together, thus achieving an improvement in global resolution.
可以知道的是,本发明采用ISA算法实现空间非一致光学像差的去除工作,显著提高成像分辨率。It can be known that the present invention adopts the ISA algorithm to remove spatially non-uniform optical aberrations, thereby significantly improving imaging resolution.
示例性地,深度神经网络或许可以替代目前的像差波前检测算法,其精度与速度有待验证。深度神经网络或许可以替代ISA中分块解卷积的操作,其成像质量与速度有待验证。For example, deep neural networks may replace the current aberration wavefront detection algorithm, and its accuracy and speed need to be verified. Deep neural networks may replace the block deconvolution operation in ISA, and its imaging quality and speed need to be verified.
综上所述,本发明带来的有益效果以及应用场景包括:In summary, the beneficial effects and application scenarios brought by the present invention include:
1)低成本、宽视场、高分辨率成像:本发明主要解决的问题就是宽视场、高分辨率成像,在光学像差较大的情形尤为有效。此外,本发明低成本的特点有利于其被广泛应用于天文观测、远距离成像等场景,为科学探索铺路架桥。1) Low cost, wide field of view, high resolution imaging: The main problem solved by the present invention is wide field of view, high resolution imaging, which is particularly effective in cases where optical aberrations are large. In addition, the low cost feature of the present invention is conducive to its wide application in astronomical observation, long-distance imaging and other scenarios, paving the way for scientific exploration.
2)湍流解析:宽视场的波前梯度检测有利于分析大气湍流运动。例如,在梯度探测与测距(Slope Detection and Ranging, SLODAR)中,宽视场的波前梯度数据有利于提高湍流层析(profiling)的纵向分辨率以及分析结果的稳健性。2) Turbulence analysis: Wide-field wavefront gradient detection is beneficial for analyzing atmospheric turbulence motion. For example, in Slope Detection and Ranging (SLODAR), wide-field wavefront gradient data is beneficial for improving the longitudinal resolution of turbulence profiling and the robustness of analysis results.
3)湍流预测:根据泰勒冻流假说,宽视场的湍流像差数据有利于提高湍流像差预测的精度。在自由空间光通信中,湍流像差的预补偿是提高信噪比、降低误码率的重要环节,本发明有利于提高湍流像差的预测精度,从而提高通信质量。3) Turbulence prediction: According to the Taylor frozen flow hypothesis, wide-field turbulence aberration data is conducive to improving the accuracy of turbulence aberration prediction. In free-space optical communication, pre-compensation of turbulence aberration is an important link to improve the signal-to-noise ratio and reduce the bit error rate. The present invention is conducive to improving the prediction accuracy of turbulence aberration, thereby improving the communication quality.
根据本发明实施例的数字自适应图像重建方法,通过即插即用的宽视场波前传感器进行高速宽视场波前检测,采用TIS算法实现了亚像素级的图像对齐和拼接,算法输入是多张低空间采样率的图片,输出是一张高空间采样率的图片。空间采样率较低时,快速变化的大气湍流导致的光学像差可视为空间非一致的快速扫描,TIS算法可以融合扫描信息,提高空间采样率。采用ISA算法实现空间非一致光学像差的去除工作,显著提高成像分辨率。According to the digital adaptive image reconstruction method of the embodiment of the present invention, high-speed wide-field-of-view wavefront detection is performed through a plug-and-play wide-field-of-view wavefront sensor, and sub-pixel image alignment and stitching are achieved using the TIS algorithm. The algorithm input is a plurality of pictures with low spatial sampling rates, and the output is a picture with high spatial sampling rates. When the spatial sampling rate is low, the optical aberrations caused by the rapidly changing atmospheric turbulence can be regarded as rapid scans of spatial inconsistency. The TIS algorithm can fuse the scanning information and improve the spatial sampling rate. The ISA algorithm is used to remove spatially inconsistent optical aberrations, significantly improving the imaging resolution.
为了实现上述实施例,如图5所示,本实施例中还提供了数字自适应图像重建系统10,该系统10包括,图像像素重排模块100、孔径图像处理模块200、波前像差转换模块300和局部图像重建模块400。In order to implement the above embodiment, as shown in Figure 5, a digital adaptive image reconstruction system 10 is also provided in this embodiment. The system 10 includes an image pixel rearrangement module 100, an aperture image processing module 200, a wavefront aberration conversion module 300 and a local image reconstruction module 400.
图像像素重排模块100,用于对原始光场图像进行像素重排得到低空间采样率的子孔径图像;An image pixel rearrangement module 100 is used to rearrange pixels of an original light field image to obtain a sub-aperture image with a low spatial sampling rate;
孔径图像处理模块200,用于利用TIS算法对低空间采样率的子孔径图像进行处理得到高空间采样率的子孔径图像;The aperture image processing module 200 is used to process the sub-aperture image with a low spatial sampling rate using the TIS algorithm to obtain a sub-aperture image with a high spatial sampling rate;
波前像差转换模块300,用于利用由粗粒度到细粒度的波前梯度估计算法对所述高空间采样率的子孔径图像的空间偏移量进行估计得到第一波前梯度,并利用训练好的MLP模型将第一波前梯度转换为第一波前像差;The wavefront aberration conversion module 300 is used to estimate the spatial offset of the sub-aperture image with a high spatial sampling rate by using a wavefront gradient estimation algorithm from coarse granularity to fine granularity to obtain a first wavefront gradient, and convert the first wavefront gradient into a first wavefront aberration by using a trained MLP model;
局部图像重建模块400,用于基于第一波前像差对所述高空间采样率的子孔径图像进行相空间解卷积运算得到局部图像的图像重建结果。The local image reconstruction module 400 is used to perform a phase space deconvolution operation on the sub-aperture image with a high spatial sampling rate based on the first wavefront aberration to obtain an image reconstruction result of the local image.
进一步地,在图像像素重排模块100之后,还包括:坐标计算模块,用于:Furthermore, after the image pixel rearrangement module 100, it further includes: a coordinate calculation module, which is used to:
获取不同时刻、同一子孔径的低空间采样率的子孔径图像的图像序列;Acquire an image sequence of sub-aperture images of the same sub-aperture at different times and with a low spatial sampling rate;
根据图像序列之间的相对偏移量计算参考时刻的各子孔径图像之间的相对坐标。The relative coordinates between the sub-aperture images at the reference time are calculated according to the relative offset between the image sequences.
进一步地,孔径图像处理模块200,还用于:Furthermore, the aperture image processing module 200 is also used for:
通过TIS算法对图像序列中的多个子孔径图像按照参考时刻的子孔径图像的相对坐标进行拼接得到坐标拼接结果;Using the TIS algorithm, multiple sub-aperture images in the image sequence are stitched according to the relative coordinates of the sub-aperture images at the reference time to obtain a coordinate stitching result;
基于坐标拼接结果得到参考时刻的高空间采样率的子孔径图像。Based on the coordinate stitching results, a sub-aperture image with high spatial sampling rate at the reference time is obtained.
进一步地,波前像差转换模块300,还用于:Furthermore, the wavefront aberration conversion module 300 is also used for:
获取由粗粒度到细粒度的波前梯度估计算法得到的第二波前梯度;Obtaining a second wavefront gradient obtained by a coarse-grained to fine-grained wavefront gradient estimation algorithm;
利用二维积分算法将所述第二波前梯度转化为第二波前像差,以拟合第二波前像差对应的待拟合的Zernike多项式系数;The second wavefront gradient is converted into a second wavefront aberration by using a two-dimensional integration algorithm, so as to fit the Zernike polynomial coefficients to be fitted corresponding to the second wavefront aberration;
利用第二波前梯度训练MLP模型以得到训练好的MLP模型;其中,模型输入层对应第二波前梯度的维度,模型输出层的节点数等于待拟合的Zernike多项式系数的数量;The MLP model is trained using the second wavefront gradient to obtain a trained MLP model; wherein the model input layer corresponds to the dimension of the second wavefront gradient, and the number of nodes in the model output layer is equal to the number of Zernike polynomial coefficients to be fitted;
将第一波前梯度输入所述训练好的MLP模型输出相应的Zernike多项式系数,以计算所述第一波前像差。The first wavefront gradient is input into the trained MLP model and the corresponding Zernike polynomial coefficients are output to calculate the first wavefront aberration.
进一步地,在局部图像重建模块400之后,还包括:全局图像重建模块,用于:Furthermore, after the local image reconstruction module 400, it further includes: a global image reconstruction module, which is used to:
对每个局部图像进行重建得到所有局部图像的图像重建结果;Reconstruct each local image to obtain image reconstruction results of all local images;
对所有局部图像的图像重建结果进行拼接以得到全局图像的图像重建结果。The image reconstruction results of all local images are stitched together to obtain the image reconstruction result of the global image.
根据本发明实施例的数字自适应图像重建系统,通过即插即用的宽视场波前传感器进行高速宽视场波前检测,采用TIS算法实现了亚像素级的图像对齐和拼接,算法输入是多张低空间采样率的图片,输出是一张高空间采样率的图片。空间采样率较低时,快速变化的大气湍流导致的光学像差可视为空间非一致的快速扫描,TIS算法可以融合扫描信息,提高空间采样率。采用ISA算法实现空间非一致光学像差的去除工作,显著提高成像分辨率。According to the digital adaptive image reconstruction system of the embodiment of the present invention, high-speed wide-field-of-view wavefront detection is performed through a plug-and-play wide-field-of-view wavefront sensor, and sub-pixel image alignment and stitching are realized by using the TIS algorithm. The algorithm input is multiple pictures with low spatial sampling rate, and the output is a picture with high spatial sampling rate. When the spatial sampling rate is low, the optical aberration caused by the rapidly changing atmospheric turbulence can be regarded as a rapid scan of spatial inconsistency. The TIS algorithm can fuse the scanning information and improve the spatial sampling rate. The ISA algorithm is used to remove spatially inconsistent optical aberrations, which significantly improves the imaging resolution.
在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、 “示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
此外,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本发明的描述中,“多个”的含义是至少两个,例如两个,三个等,除非另有明确具体的限定。In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410573482.3A CN118154411B (en) | 2024-05-10 | 2024-05-10 | Digital adaptive optical architecture and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410573482.3A CN118154411B (en) | 2024-05-10 | 2024-05-10 | Digital adaptive optical architecture and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN118154411A true CN118154411A (en) | 2024-06-07 |
CN118154411B CN118154411B (en) | 2024-07-16 |
Family
ID=91290958
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202410573482.3A Active CN118154411B (en) | 2024-05-10 | 2024-05-10 | Digital adaptive optical architecture and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN118154411B (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113396440A (en) * | 2019-02-14 | 2021-09-14 | 卡尔蔡司医疗技术公司 | System for OCT image conversion and ophthalmic image denoising and neural network thereof |
CN115375586A (en) * | 2022-10-24 | 2022-11-22 | 中国人民解放军国防科技大学 | Method and device for sharpening high-turbulence image, computer equipment and medium |
CN117560105A (en) * | 2023-11-13 | 2024-02-13 | 北京理工大学 | Adaptive compensation method for large-capacity optical communications based on Swin-transformer |
-
2024
- 2024-05-10 CN CN202410573482.3A patent/CN118154411B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113396440A (en) * | 2019-02-14 | 2021-09-14 | 卡尔蔡司医疗技术公司 | System for OCT image conversion and ophthalmic image denoising and neural network thereof |
CN115375586A (en) * | 2022-10-24 | 2022-11-22 | 中国人民解放军国防科技大学 | Method and device for sharpening high-turbulence image, computer equipment and medium |
CN117560105A (en) * | 2023-11-13 | 2024-02-13 | 北京理工大学 | Adaptive compensation method for large-capacity optical communications based on Swin-transformer |
Non-Patent Citations (3)
Title |
---|
J.OSBORN 等: "First on-sky results of a neural network based tomographic reconstructor: Carmen on Canary", PROCEEDINGS OF SPIE, vol. 9148, 21 July 2014 (2014-07-21) * |
JIAMIN WU 等: "An integrated imaging sensor for aberration-corrected 3D photography", NATURE, vol. 612, 19 October 2022 (2022-10-19) * |
马辰昊;付跃刚;贺文俊;王加科;董亭亭;: "双光楔微扫描哈特曼-夏克波前探测技术", 红外与激光工程, no. 09, 25 September 2015 (2015-09-25) * |
Also Published As
Publication number | Publication date |
---|---|
CN118154411B (en) | 2024-07-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111539879B (en) | Blind video denoising method and device based on deep learning | |
US9792668B2 (en) | Photographic image acquistion device and method | |
RU2716843C1 (en) | Digital correction of optical system aberrations | |
EP2560375B1 (en) | Image processing device, image capture device, program, and image processing method | |
CN100481937C (en) | Equipment for reconstructing high dynamic image in high resolution | |
CN109118431A (en) | A kind of video super-resolution method for reconstructing based on more memories and losses by mixture | |
CN103310427B (en) | Image super-resolution and quality enhancement method | |
Wu et al. | Resolution enhancement of long-range imaging with sparse apertures | |
Liu et al. | Exploit camera raw data for video super-resolution via hidden markov model inference | |
JP2000188680A (en) | Method and system for producing high resolution image | |
CN115115516B (en) | Real world video super-resolution construction method based on Raw domain | |
JP2011193338A (en) | Image processing apparatus, imaging apparatus, and image processing program | |
CN114972022B (en) | A fusion hyperspectral super-resolution method and system based on non-aligned RGB images | |
CN115086550A (en) | Meta-imaging method and system | |
JP5566199B2 (en) | Image processing apparatus, control method therefor, and program | |
CN116740340A (en) | Calculation spectrum imaging error correction method based on deep learning | |
US11967096B2 (en) | Methods and apparatuses of depth estimation from focus information | |
Yu et al. | Color polarization image super-resolution reconstruction via a cross-branch supervised learning strategy | |
Li et al. | Imaging simulation and learning-based image restoration for remote sensing time delay and integration cameras | |
CN118154411B (en) | Digital adaptive optical architecture and system | |
Zhang et al. | Scene recovery: Combining visual enhancement and resolution improvement | |
Lu et al. | Event camera demosaicing via swin transformer and pixel-focus loss | |
Cao et al. | Practical real video denoising with realistic degradation model | |
JP2006285415A (en) | Image processing method, device and program | |
Bätz et al. | Multi-image super-resolution for fisheye video sequences using subpixel motion estimation based on calibrated re-projection |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |