WO2023029520A1 - 一种用于扫描光场成像系统的动态场景拍摄方法及装置 - Google Patents

一种用于扫描光场成像系统的动态场景拍摄方法及装置 Download PDF

Info

Publication number
WO2023029520A1
WO2023029520A1 PCT/CN2022/088564 CN2022088564W WO2023029520A1 WO 2023029520 A1 WO2023029520 A1 WO 2023029520A1 CN 2022088564 W CN2022088564 W CN 2022088564W WO 2023029520 A1 WO2023029520 A1 WO 2023029520A1
Authority
WO
WIPO (PCT)
Prior art keywords
light field
image
scanning light
images
dynamic scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/088564
Other languages
English (en)
French (fr)
Inventor
乔晖
戴琼海
吴嘉敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Publication of WO2023029520A1 publication Critical patent/WO2023029520A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10052—Images from lightfield camera
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing

Definitions

  • the present application relates to the technical field of scanning light field imaging systems, and in particular to a dynamic scene shooting method for scanning light field imaging systems.
  • the three limiting conditions that determine the imaging quality or imaging resolution of the imaging system are as follows: First, the sampling rate of the image sensor: the popular image sensors are mainly divided into two types: CCD and CMOS. Being smaller is conducive to the generation of higher-resolution images; the second is the optical diffraction limit of the system: according to the Rayleigh resolution criterion, the numerical aperture of each lens in the imaging system determines the resolution of the system; the third is aberration, image Differences generally exist in the natural environment, such as atmospheric scattering. At the same time, aberrations also exist in the lens. Since the lens is a manually polished device, there will be deviations from the ideal lens model in theoretical optics.
  • the diffraction limit that the optical system can achieve is enough to meet the needs of our macro scene shooting, and there is also enough technology to produce a high-resolution image sensor with a large area array and a small pixel size.
  • What hinders the further development of these processes is the existence of aberrations.
  • the physical model as the size of a single lens increases, the aberration will gradually increase. Therefore, as the number of pixels increases, the number of effective pixels will be limited to a limited scale. Under such conditions, no matter how the number of pixels of the image sensor or the numerical aperture of the lens is increased, it is difficult to further improve the resolution and clarity of the image.
  • the development of Gigapixel has gone through two stages.
  • the first stage by reducing the size of the optical aperture and reducing the existence of aberrations, it can break through the limitation of the number of effective pixels of the original ordinary camera.
  • this solution will lead to a decrease in the amount of light passing through, an increase in the exposure time, and a significant decrease in the signal-to-noise ratio.
  • multi-lens shooting is stitched together to form a billion-pixel imaging.
  • this solution reduces the corresponding increase in aberration with the increase in the size of a single lens.
  • the method of reducing the aperture it breaks through the limitation of aberrations and achieves better imaging effects.
  • neither of these two solutions actually solves the problem of aberration, and the best imaging effect that can be achieved is far less than the performance of the optical diffraction limit.
  • the scanning light field system can break the mutual constraints between spatial resolution and angular resolution by using microlenses for dense spatial moving sampling. Obtain multi-angle sampling while achieving high spatial resolution. However, the scanning light field imaging system sacrifices a certain time resolution to achieve high-resolution imaging through the fusion between multiple frames.
  • the scanning light field imaging technique mentioned above obtains high spatial resolution by sacrificing temporal resolution. Therefore, serious artifacts will appear when shooting high-speed dynamic scenes, making it impossible to perform high-resolution imaging with multi-frame fusion of scanning light fields.
  • This application aims to solve one of the technical problems in the related art at least to a certain extent.
  • the first purpose of the present application is to propose a dynamic scene shooting method for a scanning light field imaging system, which can use the scanning light field to perform artifact-free high-resolution shooting of dynamic scenes.
  • the second purpose of the present application is to propose a device for reconstructing geometric motion details by fitting a depth image.
  • the embodiment of the first aspect of the present application proposes a dynamic scene shooting method for a scanning light field imaging system, including:
  • the original image of the light field is acquired by scanning the light field system
  • the dynamic scene shooting method for the scanning light field imaging system may also have the following additional technical features:
  • the sent method also includes:
  • the images of each viewing angle in the three-dimensional image stack are registered to realize dynamic scene shooting in the scanning light field.
  • the processing of the images in the three-dimensional image stack separately and combining to obtain an image pair includes:
  • the fused single-view images are decoupled into low-resolution single-view time series images according to the shooting time distribution, and one sequence of images is acquired and combined with images in other time series.
  • the registering the image pair includes:
  • a certain time frame in a scanning period is used as a standard, and the registration estimation is performed on the standard with other time frames in the scanning period.
  • the scatter interpolation is performed according to the coordinate transformation relationship to obtain a high-resolution scanning light field single-view image, including:
  • Scatter interpolation is performed on the fused single-view image and the coordinate transformation relationship to obtain the normally distributed high-resolution scanning light field single-view image.
  • the original image of the light field is obtained by scanning the light field system, the original image of the light field is transformed to obtain a three-dimensional image stack, and the images in the three-dimensional image stack are processed. Separately process and combine to obtain image pairs, register the image pairs, calculate the coordinate transformation relationship of the image pairs, and perform scatter interpolation according to the coordinate transformation relationship to obtain high-resolution scanning light field single-view images.
  • the present application can use the scanning light field to perform artifact-free high-resolution shooting of dynamic scenes.
  • the embodiment of the second aspect of the present application proposes a dynamic scene shooting device for a scanning light field imaging system, including:
  • the acquisition module is used to acquire the original image of the light field through the scanning light field system
  • a transformation module configured to obtain a three-dimensional image stack by transforming the original image of the light field
  • a combining module configured to process the images in the three-dimensional image stack separately and combine them to obtain an image pair
  • a calculation module configured to register the image pair, and calculate the coordinate transformation relationship of the image pair
  • the single-view module is used to perform scatter interpolation according to the coordinate transformation relationship to obtain a high-resolution scanning light field single-view image.
  • the dynamic scene shooting device used in the scanning light field imaging system of the embodiment of the present application obtains the original image of the light field through the scanning light field system, converts the original image of the light field to obtain a three-dimensional image stack, and performs the image processing in the three-dimensional image stack Separately process and combine to obtain image pairs, register the image pairs, calculate the coordinate transformation relationship of the image pairs, and perform scatter interpolation according to the coordinate transformation relationship to obtain high-resolution scanning light field single-view images.
  • the present application can use the scanning light field to perform artifact-free high-resolution shooting of dynamic scenes.
  • Fig. 1 is a flowchart of a dynamic scene shooting method for a scanning light field imaging system according to an embodiment of the present application.
  • Fig. 2 is a schematic structural diagram of a dynamic scene shooting device used in a scanning light field imaging system according to an embodiment of the present application.
  • the dynamic scene shooting method used in the scanning light field imaging system of the embodiment of the present application solves the problem that the traditional scanning light field imaging technology obtains high spatial resolution by sacrificing time resolution, so serious artifacts will appear when shooting high-speed dynamic scenes , leading to the technical problem that high-resolution imaging of multi-frame fusion of scanning light field cannot be performed.
  • This application can use scanning light field to perform artifact-free high-resolution shooting of dynamic scenes.
  • FIG. 1 is a flow chart of a dynamic scene shooting method for a scanning light field imaging system provided by an embodiment of the present application.
  • the dynamic scene shooting method for the scanning light field imaging system includes:
  • step S1 the original image of the light field is acquired by scanning the light field system.
  • light field original images containing different angle information are collected through a light field system or a scanning light field system.
  • Step S2 transforming the original image of the light field to obtain a three-dimensional image stack.
  • the light field original image can be transformed into a three-dimensional image stack with different angle information by rearranging or other methods.
  • step S3 the images in the three-dimensional image stack are processed separately and combined to obtain an image pair.
  • step S2 the images of different viewing angles in step S2 are processed separately.
  • the processing method is as follows: decouple the fused single-view images into low-resolution single-view time series images according to the shooting time distribution, take one of the images as an example, and combine them with images in other time series in pairs.
  • Step S4 registering the image pair, and calculating the coordinate transformation relationship of the image pair.
  • Step S5 performing scatter interpolation according to the coordinate transformation relationship to obtain a high-resolution scanning light field single-view image.
  • step S3 After the image pairs in step S3 are registered or optical flow estimated, and the coordinate transformation relationship of the scene in the image pair is calculated, the fused single-view image and the calculated coordinate transformation are performed by the scatter interpolation method Scatter point interpolation is carried out according to the relation, and a high-resolution scanning light field single-view image with normal distribution is obtained.
  • scatter interpolation is performed on the rearranged high-resolution artifact single-view images in one scanning period according to the coordinate transformation relationship, and the optimal artifact-free high-resolution single-view images are estimated.
  • the relationship between images from different perspectives is decoupled from each other. Separate operations and processing are required for different perspectives.
  • the embodiment of the present application also includes: registering the images of each perspective in the three-dimensional image stack to realize dynamic scene shooting in the scanning light field.
  • step S4 and step S5 are performed for each viewing angle, and by analogy, scanning light field dynamic scene shooting can be realized.
  • Fig. 2 is a schematic structural diagram of a dynamic scene shooting device used in a scanning light field imaging system according to an embodiment of the present application.
  • the dynamic scene shooting device 10 for the scanning light field imaging system includes:
  • the acquisition module 100 the transformation module 200 , the combination module 300 , the calculation module 400 and the single view module 500 .
  • the acquisition module 100 is used to acquire the original image of the light field by scanning the light field system;
  • the addition module 200 is used for transforming the original image of the light field to obtain a three-dimensional image stack
  • a combination module 300 configured to process the images in the three-dimensional image stack separately, and combine them to obtain an image pair;
  • Calculation module 400 for registering the image pair, and calculating the coordinate transformation relationship of the image pair
  • the single-view module 500 is configured to perform scatter interpolation according to the coordinate transformation relationship to obtain a single-view image of a high-resolution scanning light field.
  • the device 10 also includes:
  • the dynamic scene module is used for registering the images of each perspective in the three-dimensional image stack to realize dynamic scene shooting in the scanning light field.
  • combination module 300 is specifically used for:
  • the fused single-view images are decoupled into low-resolution single-view time series images according to the shooting time distribution, and one sequence of images is acquired and combined with images in other time series.
  • the device 10 also includes:
  • the registration module is configured to use a certain time frame in a scanning cycle as a standard, and perform registration estimation on the standard and other time frames in the scanning cycle.
  • Scatter interpolation is performed on the fused single-view image and the coordinate transformation relationship to obtain a normal-distributed high-resolution scanning light field single-view image.
  • the original image of the light field is obtained by scanning the light field system, and the original image of the light field is transformed to obtain a three-dimensional image stack, and the three-dimensional image stack is obtained
  • the images are processed separately, combined to obtain image pairs, the image pairs are registered, and the coordinate transformation relationship of the image pairs is calculated, and the scatter interpolation is performed according to the coordinate transformation relationship to obtain a high-resolution scanning light field single-view image.
  • the present application can use the scanning light field to perform artifact-free high-resolution shooting of dynamic scenes.
  • first and second are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, features defined as “first” and “second” may explicitly or implicitly include at least one of these features. In the description of the present application, “plurality” means at least two, such as two, three, etc., unless otherwise specifically defined.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

本申请提出一种用于扫描光场成像系统的动态场景拍摄方法和装置，其中，方法包括：通过扫描光场系统采集得到光场原始图像，将光场原始图像通过变换得到三维图像堆栈，将三维图像堆栈中的图像进行单独处理，组合得到图像对，对图像对进行配准，计算得到图像对的坐标变换关系，根据坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。本申请通过采用上述方法，可以使用扫描光场进行动态场景的无伪影高分辨率拍摄。

Description

一种用于扫描光场成像系统的动态场景拍摄方法及装置
相关申请的交叉引用
本申请要求清华大学于2021年08月31日提交的、申请名称为“一种用于扫描光场成像系统的动态场景拍摄方法及装置”的、中国专利申请号“202111014061.X”的优先权。
技术领域
本申请涉及扫描光场成像系统技术领域，特别涉及一种用于扫描光场成像系统的动态场景拍摄方法。
背景技术
近年来，超高像素数目成像逐渐进入人们的视野。随着机器视觉、无人机、高清监视系统等行业或设备的普及与扩散，人们对于高分辨率、大像素数目成像的需求日益明显。
决定成像系统成像质量或成像分辨率的三方面限制条件如下：一是像感器的采样率：普遍流行的像感器时下主要分为CCD和CMOS两类，将像素数目做多，像元尺寸做小都有利于更高清的分辨率图像生成；二是系统的光学衍射极限：根据瑞利分辨率判据，成像系统中各透镜的数值孔径决定了系统的分辨能力；三是像差，像差普遍存在于自然环境中，比如大气的散射等。同时像差也存在于镜头之中，由于镜头属于人工磨制的器件，会存在与理论光学中的理想透镜模型的偏差。另一方面，随着镜头尺寸的逐渐增大，理想光学系统中的近轴光学理论不再适用，离轴光线的轨迹难以像近轴光线那样易于预测。以上三点都限制了系统的成像能力，进一步地，阻拦十亿像素成像的发展。
得益于工业的发展，光学系统可以达到的衍射极限已经足以满足我们宏观场景拍摄的需求，同时也有足够的工艺制作出大面阵小像元尺寸的高分辨率像感器。而阻拦这些工艺进一步发展的正是像差的存在。物理模型上，随着单镜头尺寸的增长，像差会随之逐步提升。因此随着像素数目的提升、有效像素数目会被限制在一个有限的尺度上。在这样的条件下，无论如何增加像感器的像素数目或是镜头的数值孔径，图像都很难进一步提升分辨率与清晰度。
在这样的条件下，十亿像素的发展经历了两个阶段。第一个阶段通过缩小光学孔径尺寸，减小像差的存在，可以突破原有普通相机的有效像素数目局限。但这种方案会导致通光量减小，曝光时间增大，信噪比大幅度降低。第二个阶段通过多镜头拍摄拼接，形成十亿像素成像。该种方案通过增多镜头数目的方式减小像差随单镜头尺寸增大而相应增大的 幅度，比减小孔径的方式更好的突破了像差限制，达到较好的成像效果。但该两种方案都没有实际上解决像差的问题，可以达到的最佳成像效果也远远小于光学衍射极限性能。
扫描光场系统通过使用微透镜进行稠密的空间上的移动采样，可以打破空间分辨率与角度分辨率之间的相互约束。在获得高空间分辨率的同时获得多角度采样。但是，扫描光场成像系统牺牲了一定的时间分辨率，通过多帧之间的融合实现高分辨率成像。
上述扫描光场成像技术通过牺牲时间分辨率获得高空间分辨率。因此在拍摄高速动态场景时会出现严重的伪影，导致无法进行扫描光场多帧融合的高分辨率成像。
发明内容
本申请旨在至少在一定程度上解决相关技术中的技术问题之一。
为此，本申请的第一个目的在于提出一种用于扫描光场成像系统的动态场景拍摄方法，可以使用扫描光场进行动态场景的无伪影高分辨率拍摄。
本申请的第二个目的在于提出一种拟合深度图像的几何运动细节重建装置。
为达上述目的，本申请第一方面实施例提出了一种用于扫描光场成像系统的动态场景拍摄方法，包括：
S1，通过扫描光场系统采集得到光场原始图像；
S2，将所述光场原始图像通过变换得到三维图像堆栈；
S3，将所述三维图像堆栈中的图像进行单独处理，组合得到图像对；
S4，对所述图像对进行配准，计算得到所述图像对的坐标变换关系；
S5，根据所述坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。
另外，根据本申请上述实施例的用于扫描光场成像系统的动态场景拍摄方法还可以具有以下附加的技术特征：
进一步地，在本申请的一个实施例中，所送方法还包括：
将所述三维图像堆栈中每一视角图像进行配准，以实现扫描光场动态场景拍摄。
进一步地，在本申请的一个实施例中，所述将所述三维图像堆栈中的图像进行单独处理，组合得到图像对，包括：
将融合单视角图像根据拍摄时间分布解耦成低分辨率的单视角时间序列图像，获取其中某一序列图像与其它时间序列内的图像并两两组合。
进一步地，在本申请的一个实施例中，所述对所述图像对进行配准，包括：
将一个扫描周期内的某一时间帧作为标准，将所述标准与所述扫描周期内的其它时间帧进行配准估算。
进一步地，在本申请的一个实施例中，所述根据所述坐标变换关系进行散点插值，获 得高分辨率扫描光场单视角图像，包括：
将融合单视角图与所述坐标变换关系进行散点插值，获得正常分布的所述高分辨率扫描光场单视角图像。
本申请实施例的用于扫描光场成像系统的动态场景拍摄方法，通过扫描光场系统采集得到光场原始图像，将光场原始图像通过变换得到三维图像堆栈，将三维图像堆栈中的图像进行单独处理，组合得到图像对，对图像对进行配准，计算得到图像对的坐标变换关系，根据坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。本申请通过采用上述方法，可以使用扫描光场进行动态场景的无伪影高分辨率拍摄。
为达上述目的，本申请第二方面实施例提出了一种用于扫描光场成像系统的动态场景拍摄装置，包括：
采集模块，用于通过扫描光场系统采集得到光场原始图像；
变换模块，用于将所述光场原始图像通过变换得到三维图像堆栈；
组合模块，用于将所述三维图像堆栈中的图像进行单独处理，组合得到图像对；
计算模块，用于对所述图像对进行配准，计算得到所述图像对的坐标变换关系；
单视角模块，用于根据所述坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。
本申请实施例的用于扫描光场成像系统的动态场景拍摄装置，通过扫描光场系统采集得到光场原始图像，将光场原始图像通过变换得到三维图像堆栈，将三维图像堆栈中的图像进行单独处理，组合得到图像对，对图像对进行配准，计算得到图像对的坐标变换关系，根据坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。本申请通过采用上述方法，可以使用扫描光场进行动态场景的无伪影高分辨率拍摄。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为根据本申请一个实施例的用于扫描光场成像系统的动态场景拍摄方法的流程图。
图2为根据本申请一个实施例的用于扫描光场成像系统的动态场景拍摄装置的结构示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
下面参考附图描述本申请实施例的用于扫描光场成像系统的动态场景拍摄方法和装置。
本申请实施例的用于扫描光场成像系统的动态场景拍摄方法，解决了传统扫描光场成像技术通过牺牲时间分辨率获得高空间分辨率，因此在拍摄高速动态场景时会出现严重的伪影，导致无法进行扫描光场多帧融合的高分辨率成像的技术问题，本申请可以使用扫描光场进行动态场景的无伪影高分辨率拍摄。
图1为本申请实施例所提供的一种用于扫描光场成像系统的动态场景拍摄方法的流程图。
如图1所示，该用于扫描光场成像系统的动态场景拍摄方法包括：
步骤S1，通过扫描光场系统采集得到光场原始图像。
具体的，通过光场系统或扫描光场系统采集包含不同角度信息的光场原始图像。
步骤S2，将光场原始图像通过变换得到三维图像堆栈。
可以理解的是，可以通过重排列等方法，将光场原始图像变换成具有不同角度信息的三维图像堆栈。
步骤S3，将三维图像堆栈中的图像进行单独处理，组合得到图像对。
可以理解的是，将步骤S2中不同视角图像各自进行单独处理。处理方法如下：将融合单视角图根据拍摄时间分布解耦成低分辨率的单视角时间序列图像，取其中某一图像作为范例，与其它时间序列内的图像两两组合。
步骤S4，对图像对进行配准，计算得到图像对的坐标变换关系。
可以理解的是，通过将一个扫描周期内的某一时间帧作为标准，将其与该扫描周期内的其它时间帧(如5*5周期扫描内即有24帧其它帧)进行配准估算，计算出两两图像间的坐标变换关系。
步骤S5，根据坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。
可以理解的是，将步骤S3中的图像对分别进行配准或光流估计，计算出图像对中场景的坐标变换关系之后，通过散点插值方法，将融合单视角图与其计算出来的坐标变换关系进行散点插值，获得正常分布的高分辨率扫描光场单视角图像。
也就是说，对一个扫描周期内重排列后的高分辨率伪影单视角图根据坐标变换关系进行散点插值，估算出最优的无伪影高分辨率单视角图像。需要注意不同视角图像间的关系是互相解耦的。需要对不同视角进行单独的运算与处理。
本申请实施例还包括：将三维图像堆栈中每一视角图像进行配准，以实现扫描光场动 态场景拍摄。
应当理解的是，对每一视角进行步骤S4和步骤S5，以此类推可以实现扫描光场动态场景拍摄。
图2为根据本申请一个实施例的用于扫描光场成像系统的动态场景拍摄装置的结构示意图。
如图2所示，该用于扫描光场成像系统的动态场景拍摄装置10包括：
采集模块100、变换模块200、组合模块300、计算模块400和单视角模块500。
采集模块100，用于通过扫描光场系统采集得到光场原始图像；
添加模块200，用于将光场原始图像通过变换得到三维图像堆栈；
组合模块300，用于将三维图像堆栈中的图像进行单独处理，组合得到图像对；
计算模块400，用于对图像对进行配准，计算得到图像对的坐标变换关系；
单视角模块500，用于根据坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。
进一步地，该装置10还包括：
动态场景模块，用于将三维图像堆栈中每一视角图像进行配准，以实现扫描光场动态场景拍摄。
进一步地，上述组合模块300，具体用于：
将融合单视角图像根据拍摄时间分布解耦成低分辨率的单视角时间序列图像，获取其中某一序列图像与其它时间序列内的图像并两两组合。
进一步地，该装置10还包括：
配准模块，用于将一个扫描周期内的某一时间帧作为标准，将标准与扫描周期内的其它时间帧进行配准估算。
进一步，上述单视角模块500，具体用于：
将融合单视角图与坐标变换关系进行散点插值，获得正常分布的高分辨率扫描光场单视角图像。
根据本申请实施例提出的用于扫描光场成像系统的动态场景拍摄装置，通过扫描光场系统采集得到光场原始图像，将光场原始图像通过变换得到三维图像堆栈，将三维图像堆栈中的图像进行单独处理，组合得到图像对，对图像对进行配准，计算得到图像对的坐标变换关系，根据坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。本申请通过采用上述方法，可以使用扫描光场进行动态场景的无伪影高分辨率拍摄。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示 或者隐含地包括至少一个该特征。在本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (10)

  1. 一种用于扫描光场成像系统的动态场景拍摄方法，其特征在于，所述方法包括以下步骤：
    通过扫描光场系统采集得到光场原始图像；
    将所述光场原始图像通过变换得到三维图像堆栈；
    将所述三维图像堆栈中的图像进行单独处理，组合得到图像对；
    对所述图像对进行配准，计算得到所述图像对的坐标变换关系；
    根据所述坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。
  2. 根据权利要求1所述的用于扫描光场成像系统的动态场景拍摄方法，其特征在于，所述方法还包括：
    将所述三维图像堆栈中每一视角图像进行配准，以实现扫描光场动态场景拍摄。
  3. 根据权利要求1所述的用于扫描光场成像系统的动态场景拍摄方法，其特征在于，所述将所述三维图像堆栈中的图像进行单独处理，组合得到图像对，包括：
    将融合单视角图像根据拍摄时间分布解耦成低分辨率的单视角时间序列图像，获取其中某一序列图像与其它时间序列内的图像并两两组合。
  4. 根据权利要求1所述的用于扫描光场成像系统的动态场景拍摄方法，其特征在于，所述对所述图像对进行配准，包括：
    将一个扫描周期内的某一时间帧作为标准，将所述标准与所述扫描周期内的其它时间帧进行配准估算。
  5. 根据权利要求1所述的用于扫描光场成像系统的动态场景拍摄方法，其特征在于，所述根据所述坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像，包括：
    将融合单视角图与所述坐标变换关系进行散点插值，获得正常分布的所述高分辨率扫描光场单视角图像。
  6. 一种用于扫描光场成像系统的动态场景拍摄装置，其特征在于，包括：
    采集模块，用于通过扫描光场系统采集得到光场原始图像；
    变换模块，用于将所述光场原始图像通过变换得到三维图像堆栈；
    组合模块，用于将所述三维图像堆栈中的图像进行单独处理，组合得到图像对；
    计算模块，用于对所述图像对进行配准，计算得到所述图像对的坐标变换关系；
    单视角模块，用于根据所述坐标变换关系进行散点插值，获得高分辨率扫描光场单视角图像。
  7. 根据权利要求6所述的用于扫描光场成像系统的动态场景拍摄装置，其特征在于， 还包括：
    动态场景模块，用于将所述三维图像堆栈中每一视角图像进行配准，以实现扫描光场动态场景拍摄。
  8. 根据权利要求6所述的用于扫描光场成像系统的动态场景拍摄装置，其特征在于，所述组合模块，用于：
    将融合单视角图像根据拍摄时间分布解耦成低分辨率的单视角时间序列图像，获取其中某一序列图像与其它时间序列内的图像并两两组合。
  9. 根据权利要求6所述的用于扫描光场成像系统的动态场景拍摄装置，其特征在于，还包括：
    配准模块，用于将一个扫描周期内的某一时间帧作为标准，将所述标准与所述扫描周期内的其它时间帧进行配准估算。
  10. 根据权利要求6所述的用于扫描光场成像系统的动态场景拍摄装置，其特征在于，所述单视角模块，用于：
    将融合单视角图与所述坐标变换关系进行散点插值，获得正常分布的所述高分辨率扫描光场单视角图像。
PCT/CN2022/088564 2021-08-31 2022-04-22 一种用于扫描光场成像系统的动态场景拍摄方法及装置 Ceased WO2023029520A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111014061.X 2021-08-31
CN202111014061.XA CN113487658B (zh) 2021-08-31 2021-08-31 一种用于扫描光场成像系统的动态场景拍摄方法及装置

Publications (1)

Publication Number Publication Date
WO2023029520A1 true WO2023029520A1 (zh) 2023-03-09

Family

ID=77947133

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/088564 Ceased WO2023029520A1 (zh) 2021-08-31 2022-04-22 一种用于扫描光场成像系统的动态场景拍摄方法及装置

Country Status (2)

Country Link
CN (1) CN113487658B (zh)
WO (1) WO2023029520A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117876279A (zh) * 2024-03-11 2024-04-12 浙江荷湖科技有限公司 基于扫描光场序列图像的去除运动伪影方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113487658B (zh) * 2021-08-31 2021-11-16 清华大学 一种用于扫描光场成像系统的动态场景拍摄方法及装置
CN115153422B (zh) * 2022-05-30 2025-06-13 清华大学 基于元成像的眼底成像方法及系统
CN115208992B (zh) * 2022-05-30 2025-05-27 元潼(北京)技术有限公司 元成像被动扫描系统与方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090022421A1 (en) * 2007-07-18 2009-01-22 Microsoft Corporation Generating gigapixel images
CN113256741A (zh) * 2021-07-14 2021-08-13 清华大学 用于扫描光场成像系统的镜头标定方法及装置
CN113487658A (zh) * 2021-08-31 2021-10-08 清华大学 一种用于扫描光场成像系统的动态场景拍摄方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102015121403A1 (de) * 2015-12-09 2017-06-14 Carl Zeiss Microscopy Gmbh Lichtfeld-bildgebung mit scanoptik
CN109923579B (zh) * 2016-09-28 2024-01-26 港大科桥有限公司 扫描成像中像素分辨率的恢复
CN110706346B (zh) * 2019-09-17 2022-11-15 浙江荷湖科技有限公司 时空联合优化重建方法及系统
CN110596885B (zh) * 2019-09-17 2022-11-18 浙江荷湖科技有限公司 扫描光场成像系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090022421A1 (en) * 2007-07-18 2009-01-22 Microsoft Corporation Generating gigapixel images
CN113256741A (zh) * 2021-07-14 2021-08-13 清华大学 用于扫描光场成像系统的镜头标定方法及装置
CN113487658A (zh) * 2021-08-31 2021-10-08 清华大学 一种用于扫描光场成像系统的动态场景拍摄方法及装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HUANG, JING LI; JINZONG, HUANG; JIANMING, CHEN FENG: "Image Super-Resolution Reconstruction Based on an Optimized Alignment Algorithm", JOURNAL OF COMPUTER APPLICATIONS, JISUANJI YINGYONG, CN, vol. 26, no. S2, 31 December 2006 (2006-12-31), CN , pages 109 - 112, XP009544209, ISSN: 1001-9081 *
LI, ZHAN; HAN, GUO-QIANG; CHEN, XIANG-JI; LIAO, XIU-XIU: "Fast Super-Resolution Image Reconstruction Based on Keren Registration and Interpolation", HUANAN LIGONG DAXUE XUEBAO - JOURNAL OF SOUTH CHINA UNIVERSITY OF TECHNOLOGY, GUANGZHOU, CH, vol. 39, no. 5, 31 May 2011 (2011-05-31), CH , pages 84 - 90, XP009544210, ISSN: 1000-565X *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117876279A (zh) * 2024-03-11 2024-04-12 浙江荷湖科技有限公司 基于扫描光场序列图像的去除运动伪影方法及系统
CN117876279B (zh) * 2024-03-11 2024-05-28 浙江荷湖科技有限公司 基于扫描光场序列图像的去除运动伪影方法及系统

Also Published As

Publication number Publication date
CN113487658B (zh) 2021-11-16
CN113487658A (zh) 2021-10-08

Similar Documents

Publication Publication Date Title
CN113487658B (zh) 一种用于扫描光场成像系统的动态场景拍摄方法及装置
JP6702323B2 (ja) カメラモジュール、固体撮像素子、電子機器、および撮像方法
CN101883215B (zh) 成像设备
US7567271B2 (en) Shared color sensors for high-resolution 3-D camera
JP5988790B2 (ja) 画像処理装置、撮像装置、画像処理方法、および、画像処理プログラム
EP2160018B1 (en) Image pickup apparatus
JP4706882B2 (ja) 撮像装置
CN101888487B (zh) 高动态范围视频成像系统和图像生成方法
TWI531852B (zh) 影像擷取裝置及數位變焦方法
WO2015081870A1 (zh) 一种图像处理方法、装置及终端
CN101720027B (zh) 可变焦阵列摄像机协同获取不同分辨率多目标视频方法
CN111182191A (zh) 基于计算像差补偿的宽视场高分辨率摄像设备及方法
CN113436130B (zh) 一种非结构光场智能感知系统与装置
WO2016045425A1 (zh) 一种两视点立体图像合成方法及系统
CN101072288A (zh) 取得鱼眼影像校正关系与鱼眼校正的方法
WO2023284792A1 (zh) 用于扫描光场成像系统的镜头标定方法及装置
CN108200324A (zh) 一种基于可变焦距镜头的成像系统及成像方法
JP2012191351A (ja) 撮像装置および画像処理方法
CN108805921B (zh) 图像获取系统及方法
TWI599809B (zh) 鏡頭模組陣列、影像感測裝置與數位縮放影像融合方法
CN108156383B (zh) 基于相机阵列的高动态十亿像素视频采集方法及装置
WO2024183688A1 (zh) 一种具有影像空间投影姿态矫正的数字影像立体拍摄系统
CN112258581A (zh) 一种多鱼眼镜头全景相机的现场标定方法
WO2020244273A1 (zh) 双摄像机三维立体成像系统和处理方法
JP4523538B2 (ja) 立体映像撮影表示装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22862666

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22862666

Country of ref document: EP

Kind code of ref document: A1