WO2021077944A1 - 基于频域的去除光场重建图像中的周期噪声的方法 - Google Patents

基于频域的去除光场重建图像中的周期噪声的方法 Download PDF

Info

Publication number
WO2021077944A1
WO2021077944A1 PCT/CN2020/115325 CN2020115325W WO2021077944A1 WO 2021077944 A1 WO2021077944 A1 WO 2021077944A1 CN 2020115325 W CN2020115325 W CN 2020115325W WO 2021077944 A1 WO2021077944 A1 WO 2021077944A1
Authority
WO
WIPO (PCT)
Prior art keywords
light field
spectrum
image
reconstructed image
imaging
Prior art date
Application number
PCT/CN2020/115325
Other languages
English (en)
French (fr)
Inventor
曹汛
李晓雯
华夏
Original Assignee
南京大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京大学 filed Critical 南京大学
Priority to US17/754,874 priority Critical patent/US20230306558A1/en
Publication of WO2021077944A1 publication Critical patent/WO2021077944A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/957—Light-field or plenoptic cameras or camera modules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10052—Images from lightfield camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details

Definitions

  • the invention belongs to the field of light field imaging, and in particular relates to a method for removing periodic noise in a light field reconstructed image based on the frequency domain.
  • the light field camera is realized by adding a micro lens array between the main lens and the sensor inside the traditional camera. Under the action of the microlens array, the light field camera can simultaneously capture the space and angle information of the light under a single exposure, that is, the complete four-dimensional light field, and adjust the film plane of the composite camera through digital refocusing technology after shooting. Thus, a light field image is rendered to render the reconstruction results focused on different depths. Light field cameras can achieve good digital refocusing effects for portraits, high-speed motions, and micro close-ups in traditional photography themes.
  • the I type light field camera places the micro lens array at one focal length in front of the sensor. The micro lens imaging is completely defocused relative to the main lens imaging.
  • the resolution of the final rendered image is limited by the number of micro lenses, which limits the I type light field.
  • the II type light field camera improves the spatial position and focus position of the microlens array.
  • Each microlens is used as the relay imaging system of the main lens.
  • the microlens imaging is focused on the imaging plane of the main lens, and the resolution of the final rendered image is not Restricted by the number of microlenses, the imaging resolution of the light field camera has been significantly improved.
  • All-focus method that is, the pixel blocks taken out of each microlens image are directly stitched to obtain a completely clear reconstructed image without changes in focus depth
  • Refocusing method that is, adjusting the synthetic aperture size of the imaging system by fusing the neighborhood microlens images, reducing the depth of field, and obtaining reconstructed images focused at different depths.
  • Both methods involve taking out pixel blocks of the same size from each microlens image and stitching them together. Taking into account the brightness difference of each microlens image and other factors, the stitching edge of the reconstruction result is obvious, and the reconstruction effect is reduced. Since the size of each pixel block is the same, the arrangement of the spliced edges is periodic, which can be optimized by the method of removing the periodic noise of the image.
  • the purpose of the present invention is to provide a frequency domain-based method for removing periodic noise in the light field reconstructed image, which can effectively remove the splicing edge noise in the light field reconstructed image while protecting the details of the image. Optimize the reconstruction effect.
  • step S3 using the optical center position icon to determine the imaging center of each microlens, and then rendering and reconstructing the light field imaging map collected in step S1 to obtain a light field reconstruction image;
  • step S5 preprocessing the spectrum of the reconstructed image of the light field transformed in step S4, that is, taking the amplitude and compressing the value range;
  • S6 Generate a low-pass filter to protect the low-frequency component part of the spectrum of the reconstructed image of the light field in a circular area with a zero frequency as the center and a radius of r;
  • step S7 Multiply the low-pass filter by the light field preprocessed in step S5 to reconstruct the image spectrum, and set the spectrum value of the low-frequency component to zero;
  • step S9 using the image mask to filter the original spectrum of the light field reconstructed image obtained in step S4 to filter out high-frequency periodic noise components in the spectrum;
  • the specific process of collecting the optical center position map of each microlens imaging corresponding to the light field imaging map is: taking the light source as the sample to directly illuminate the imaging system, after ensuring that the light source is imaged by the main lens, it is enough to cover the entire Under the premise of the microlens array, each microlens can record a sufficient angle of light to form a circle on the sensor, and its center is the center of the microlens imaging, so the light field imaging map of the light source is used as the calculation center of each microlens imaging The position map of the optical center; adjust the distance between the light source and the imaging system so that the imaging of each microlens does not overlap.
  • the specific method of using the optical center position icon to determine the imaging center of each microlens is as follows: first initialize two two-dimensional matrices C x and Cy with the same size as the microlens array, and record each microlens respectively. The horizontal and vertical coordinates of the lens imaging center; retrieve the circle that appears in the optical center position map, and fill in the center coordinates of the identified circle into the corresponding positions of the matrix C x and Cy as the center position coordinates of the micro lens imaging at that position , Interpolate and fill the matrix value corresponding to the position where the circle is not recognized, as the estimated value of the center position coordinate of the micro lens imaging at that position.
  • the specific method for rendering and reconstructing the light field imaging map is: taking out a rectangular pixel with the imaging center of each microlens as the center and a side length of (2l+1) in the light field imaging map.
  • the blocks are arranged according to the position of the microlens array, and the pixel blocks taken out from the imaging of each microlens are flipped horizontally and then sequentially spliced to obtain a light field reconstruction image; among them, the value of the length l should be less than the radius of the microlens .
  • step S6 initializing a two-dimensional matrix with the same size as the spectrum of the light field reconstructed image, the two-dimensional matrix is divided into two parts by a circle with a radius of r at its center: matrix value within the circle Set to 0, indicating that the component corresponding to the area in the object to be processed is suppressed and filtered out; the value of the matrix outside the circle is set to 1, indicating that the component corresponding to the area in the object to be processed is not affected and passes normally.
  • step S8 the specific method for generating an image mask to: a low pass filter after the pre-matrices Lp and the optical field spectrum of a reconstructed image I 'F multiplied by the low-frequency component of the optical nulling spectral values Field reconstruction image spectrum I′′ F :
  • the spectral value of the corresponding position of the component in the light field reconstructed image spectrum I F is set to zero, and the high frequency is filtered out.
  • the light field reconstructed image spectrum I Fd of the periodic noise component is
  • the present invention transforms the light field reconstructed image to the frequency domain, utilizes the spectral characteristics of periodic noise to generate an image mask, filters out high frequency periodic noise components in the spectrum of the light field reconstructed image, and obtains a light field reconstructed image with periodic noise removed.
  • this method is an image denoising method based on the frequency domain, which can effectively remove periodic splicing edge noise in the light field reconstructed image while protecting the details of the image, avoiding image blur, and optimizing the reconstruction effect.
  • Figure 1 is a flow chart of the method of the present invention.
  • Fig. 2 is a schematic diagram of the spectrum of the reconstructed image of the light field after preprocessing of the present invention.
  • Figure 3 is a schematic diagram of the image mask of the present invention.
  • the present invention is based on the frequency domain method for removing periodic noise in the light field reconstructed image.
  • the steps are as follows: S1: Place the sample in front of the light field imaging system, illuminate the sample with a light source, and adjust the system imaging plane to a clear image of the sample Plane, the light field imaging map of the collected sample.
  • the principle of the light field imaging system of the present invention is based on a type II light field camera.
  • the light field imaging system includes a main lens, a micro lens array and a sensor.
  • the specific imaging process is that the sample A is imaged in front of the micro lens array through the main lens, that is, image A'; Different sub-parts like A'are imaged on the sensor, and the light field imaging image I f of the sample is obtained.
  • the non-overlapping parts of the microlenses on the sensor can be spliced together to reconstruct a complete scene.
  • step S2 Keep the light field imaging system consistent with step S1. When no sample is placed, turn on the light source to directly illuminate the light field imaging system, and collect the optical center position map of each microlens image corresponding to the light field imaging map.
  • each microlens In order to integrate the imaging of each microlens in the light field reconstruction image I f to reconstruct a complete scene, it is necessary to determine the center position of the imaging of each microlens in I f.
  • the light source is used as the sample to directly illuminate the imaging system.
  • each microlens can record a sufficient angle of light, and the image is formed as a circle on the sensor.
  • the center of the circle is the microlens.
  • the light field imaging map of the light source can be used as the optical center position map I c for calculating the imaging center of each microlens.
  • step S3 Use the optical center position map collected in step S2 to calculate the imaging center of each microlens, and then render and reconstruct the light field imaging map of the sample collected in S1.
  • the method for calculating the imaging center of each microlens is to first initialize two two-dimensional matrices C x and Cy with the same size as the microlens array, and record the horizontal and vertical coordinates of the imaging center of each microlens. Retrieve the circle appearing in the optical center position map I c , and fill in the center coordinates of the recognized circle into the corresponding positions of the matrices C x and Cy as the center position coordinates of the microlens imaging at that position. For those that have not recognized the circle The matrix value corresponding to the position is interpolated and filled as the estimated value of the center position coordinate of the micro lens imaging at that position.
  • the method of rendering and reconstructing the light field imaging map is: select a suitable length l, take out a rectangular pixel block with the imaging center of each microlens as the center and a side length of (2l+1) size, and arrange them according to the position of the microlens array ,
  • the pixel blocks taken out from the imaging of each microlens are spliced in an orderly manner, the image content of adjacent pixel blocks can be connected, and the light field reconstructed image Ir is obtained .
  • Adjust the size of the length l to make the details of the reconstructed image clear and sharp, and the value of the length l should be smaller than the radius of the microlens. It should be noted during reconstruction that since the imaging of each microlens is inverted, each pixel block needs to be flipped horizontally before splicing.
  • step S4 Transform the reconstructed image of the light field obtained in step S3 to the frequency domain, and arrange the low-frequency components to the center of the spectrum;
  • F() represents Fourier transform.
  • the high-frequency components are distributed in the center of the spectrum, and the low-frequency components are distributed in the periphery of the spectrum. It is difficult to focus on the low-frequency components. Therefore, they are rearranged, the half-space of the spectrum is exchanged along each dimension, and the zero frequency is moved to the center of the spectrum. . Since the image can be regarded as a two-dimensional matrix and the frequency spectrum is distributed in four quadrants, only one three-quadrant exchange and two-four-quadrant exchange of the original spectrum are needed to obtain the light field reconstructed image spectrum with low frequency components arranged in the center of the spectrum. Denoted as I F.
  • step S5 Preprocess the spectrum of the reconstructed image of the light field transformed in step S4, take the amplitude, and compress the value range.
  • the spectrum I F of the light field reconstructed image obtained by the Fourier transform is a complex number, and the amplitude is taken to obtain the amplitude spectrum
  • S6 Generate a low-pass filter to protect the low-frequency component part of the spectrum of the reconstructed image of the light field in a circular area with a zero frequency as the center and a radius of r.
  • the low-pass filter is a two-dimensional matrix Lp with the same size as the spectrum I F of the light field reconstructed image. It is divided into two parts by a circle with a radius of r in the center area: the value of the matrix in the circle is set to 0, which means the area in the object to be processed The corresponding component is suppressed and filtered out; the matrix value outside the circle is set to 1, which means that the component corresponding to the area in the object to be processed is not affected and passes normally.
  • the image spectrum obtained by the matrix Lp processing is used as the initial state of the image mask in subsequent operations to extract the light field reconstruction image spectrum I F , see step S9, and after the image mask is filtered, the spectrum I F is The low-frequency components in the circular area with the radius r are preserved, so the final processing effect of the matrix Lp on the frequency spectrum I F appears as a low-pass filter. During processing, you can control the range of low frequency components that you want to retain by adjusting the size of the radius r.
  • step S7 Multiply the low-pass filter generated in step S6 by the light field preprocessed in step S5 to reconstruct the image spectrum, and set the spectrum value of the low frequency component to zero.
  • I′′ F is used as the initial state of the image mask in subsequent operations to protect the low-frequency components of the light field reconstructed image spectrum I F.
  • the low-frequency components are the useful signal part of the light field reconstructed image Ir .
  • the zero-set low-frequency components can be seen in the center of Figure 3 Black circular area (black means that the spectral value of this position is zero after binarization).
  • step S8 Binarize the spectrum of the light field reconstruction image obtained in step S7 with the low-frequency component spectrum value zeroed to obtain an image mask.
  • the part with a spectrum value higher than Tr is the frequency component that needs to be filtered out for the light field reconstruction image spectrum I F to obtain the image mask M , See Figure 3, where black means the binarization value is 0, and white means the binarization value is 1.
  • the corresponding value of the low-frequency component of the image mask M obtained by binarization must be zero regardless of the threshold Tr.
  • the corresponding appearance on the spectrogram is that the spectral value of some high-frequency components is relatively large.
  • the value of the image mask M corresponding to this part of the high-frequency component is 1.
  • step S9 Use the image mask generated in step S8 to filter the original spectrum of the light field reconstructed image obtained in step S4 to filter out high-frequency periodic noise components in the spectrum.
  • step S8 Use the image mask M generated in step S8 to filter the light field reconstruction image spectrum I F obtained in step S4. If the value of the image mask M corresponding to a certain frequency component is 1, then this component is included in the image spectrum I F The spectrum value of the corresponding position is set to zero, and the spectrum I Fd of the reconstructed image of the light field with the high-frequency periodic noise components filtered out is obtained:
  • step S10 Transform the spectrum of the light field reconstructed image after filtering the high frequency periodic noise components in step S9 back to the space domain to obtain the light field reconstructed image with the periodic noise removed.
  • F -1 () represents the inverse Fourier transform

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

本发明公开了一种基于频域的去除光场重建图像中的周期噪声的方法。具体步骤为：S1采集样本的光场成像图；S2采集无样本光心位置图；S3标定各微透镜成像的中心并对光场成像图进行渲染重建；S4将光场重建图像变换至频域，生成图像频谱；S5对图像频谱进行预处理；S6生成低通滤波器；S7将低通滤波器与预处理后的图像频谱相乘，低频分量的频谱值置零；S8对光场重建图像频谱作二值化处理，得到图像掩膜；S9滤除光场重建图像原始频谱中的高频周期噪声分量；S10将过滤后的光场重建图像频谱变换回空间域，得到去除周期噪声的光场重建图像。本发明能够有效的去除光场重建图像中的周期噪声，消除拼接边缘痕迹，优化重建结果。

Description

基于频域的去除光场重建图像中的周期噪声的方法 技术领域
本发明属于光场成像领域，尤其涉及一种基于频域的去除光场重建图像中的周期噪声的方法。
背景技术
光场相机的出现为人们提供了一种关于图像拍摄机制的新思考。传统相机在拍摄时，必须先选定聚焦位置，一旦图像拍摄完成，聚焦位置即不可更改。然而在一些高速摄影场景下，如体育运动、动物摄影等，能够实现“先拍摄后聚焦”则显得意义重大。
光场相机通过在传统相机内部的主透镜和传感器之间加入微透镜阵列来实现。在微透镜阵列的作用下，光场相机能够在单次拍摄曝光下同时捕获光线的空间和角度信息，即完整的四维光场，并在拍摄后通过数字重聚焦技术调整合成相机的胶片平面，从而由一张光场图像渲染出分别聚焦在不同深度的重建结果。光场相机对传统摄影题材中的人像、高速运动、微观特写等都能实现好的数字重聚焦效果。I型光场相机将微透镜阵列放置在传感器前一倍焦距处，微透镜成像相对于主透镜成像完全散焦，最终渲染图像的分辨率受限于微透镜的数目，限制了I型光场相机的发展。Ⅱ型光场相机改进了微透镜阵列的空间位置及聚焦位置，将每个微透镜作为主透镜的中继成像系统，微透镜成像聚焦在主透镜的成像平面上，最终渲染图像的分辨率不再受微透镜数目的制约，光场相机的成像分辨率得到显著提升。
对光场相机拍摄得到的图像进行渲染重建有两种方式：(1)全聚焦方法，即对每个微透镜图像取出的像素块直接拼接，得到无聚焦深度变化的完全清晰的重建图像；(2)重聚焦方法，即通过融合邻域微透镜图像来调节成像系统合成孔径大小，减小景深，得到聚焦在不同深度的重建图像。两种方式都涉及到从每个微透镜图像中取出大小相同的像素块并拼接，考虑到各微透镜图像的亮度差异等因素，重建结果的拼接边缘明显，重建效果下降。由于各像素块大小相同，拼接边缘的排列具有周期性，可利用去除图像周期噪声的方法加以优化。
现有常用的基于空间域的图像去噪方法，如均值滤波，中值滤波，高斯滤波等，对于颗粒噪声、高斯噪声、椒盐噪声、乘性噪声等有较好的去除效果，但是对于光场重建图像中拼接边缘带来的周期噪声，上述方法则会在平滑图像噪声的 同时使图像模糊，细节丢失，抑制噪声的同时也抑制了信号。
发明内容
针对以上现有技术缺陷，本发明的目的在于提供一种基于频域的去除光场重建图像中的周期噪声的方法，能够在保护图像细节的同时有效去除光场重建图像中的拼接边缘噪声，优化重建效果。
实现本发明目的的技术解决方案为：
基于频域的去除光场重建图像中的周期噪声的方法，包括如下步骤：
S1，利用光场成像系统采集样本的光场成像图；
S2，移除样本，将光源直接照射光场成像系统，采集光场成像图对应的各微透镜成像的光心位置图；
S3，利用所述光心位置图标定各微透镜成像的中心，进而对步骤S1采集的光场成像图进行渲染重建，得到光场重建图像；
S4，将所述光场重建图像变换到频域，并将低频分量排列至频谱中心；
S5，对步骤S4变换后的光场重建图像频谱进行预处理，即取幅值并压缩值域；
S6，生成低通滤波器，保护以零频为中心、半径为r的圆形区域内的光场重建图像频谱的低频分量部分；
S7，将所述低通滤波器乘以步骤S5预处理后的光场重建图像频谱，低频分量的频谱值置零；
S8，对经步骤S7处理后得到的光场重建图像频谱作二值化处理，得到图像掩膜；
S9，利用所述图像掩膜，对步骤S4变换得到的光场重建图像原始频谱进行过滤，滤除频谱中的高频周期噪声分量；
S10，将过滤后的光场重建图像频谱变换回空间域，得到去除周期噪声的光场重建图像。
进一步地，所述步骤S2中，采集光场成像图对应的各微透镜成像的光心位置图的具体过程为：以光源为样本直接照射成像系统，在确保光源经主透镜成像后足以覆盖整个微透镜阵列的前提下，各微透镜能够记录充足角度的光线，在传感器上成像为圆形，其圆心即为微透镜成像的中心，因此将光源的光场成像图作为计算各微透镜成像中心的光心位置图；调整光源与成像系统之间的距离，使各 微透镜成像之间不重叠。
进一步地，所述步骤S3中，利用光心位置图标定各微透镜成像的中心的具体方法为：首先初始化与微透镜阵列大小相同的两个二维矩阵C x、C y，分别记录各微透镜成像中心的横、纵坐标；检索光心位置图中出现的圆，将识别到的圆的圆心坐标填入矩阵C x、C y的相应位置，作为该位置的微透镜成像的中心位置坐标，对未识别到圆的位置对应的矩阵值进行插值填充，作为该位置微透镜成像的中心位置坐标的估计值。
进一步地，所述步骤S3中，对光场成像图进行渲染重建的具体方法为：在光场成像图中取出以各微透镜成像中心为中心、边长为(2l+1)大小的矩形像素块，按照微透镜阵列的位置排布，将从各微透镜成像中取出的像素块先水平翻转后再进行有序拼接，得到光场重建图像；其中，长度l的取值应小于微透镜半径。
进一步地，所述步骤S6的具体方法为：初始化一个与光场重建图像频谱大小相同的二维矩阵，该二维矩阵由其中心位置一个半径为r的圆划分为两部分：圆内矩阵值置0，表示待处理对象中该区域对应的分量受到抑制并被滤除；圆外矩阵值置1，表示待处理对象中该区域对应的分量不受影响并正常通过。
进一步地，所述步骤S8中，图像掩膜的具体生成方法为：将低通滤波器矩阵Lp与预处理后的光场重建图像频谱I′ F相乘，得到低频分量频谱值置零的光场重建图像频谱I″ F：
I″ F＝Lp×I′ F
设置二值化阈值Tr，对低频分量频谱值置零的光场重建图像频谱I″ F作二值化处理，得到图像掩膜M，M值为1的部分即对应光场重建图像频谱I F需要滤除的频率分量；通过改变阈值Tr的大小，调节图像掩膜M中值为1的频率分量的占比，进而调节对噪声的抑制程度。
进一步地，所述步骤S9中，若某频率分量对应的图像掩膜M的值为1，则将该分量在光场重建图像频谱I F中对应位置的频谱值置零，得到滤除高频周期噪声分量的光场重建图像频谱I Fd：
Figure PCTCN2020115325-appb-000001
其中，i＝1…r，j＝1…c，r、c分别为光场重建图像I r的行数和列数。
本发明将光场重建图像变换至频域，利用周期噪声的频谱特征，生成图像掩 膜，滤除光场重建图像频谱中的高频周期噪声分量，得到去除周期噪声的光场重建图像。与现有技术相比，本方法是基于频域的图像去噪方法，能够在保护图像细节特征的同时，有效去除光场重建图像中周期性的拼接边缘噪声，避免图像模糊，优化重建效果。
附图说明
图1是本发明方法的流程图。
图2是本发明预处理之后的光场重建图像频谱的示意图。
图3是本发明图像掩膜的示意图。
具体实施方式
为使本发明的目的，技术方案和优点更加清楚，下面将结合附图对本发明实施方法作进一步地详细描述。
参见图1，本发明基于频域的去除光场重建图像中的周期噪声的方法，步骤如下：S1：将样本放置于光场成像系统前，光源照射样本，调整系统成像平面至样本清晰成像的平面，采集样本的光场成像图。
本发明的光场成像系统的原理是基于Ⅱ型光场相机。该光场成像系统包括主透镜、微透镜阵列和传感器，具体成像过程为，样本A通过主透镜成像在微透镜阵列的前方，即像A’；每个微透镜再以中继系统的形式将像A’的不同子部分成像在传感器上，得到样本的光场成像图I f。后期处理时，传感器上各微透镜成像的不重叠部分拼接起来即可重建成完整的场景。
S2：保持光场成像系统与步骤S1的一致，在无样本放置的情况下，打开光源，直接照射光场成像系统，采集光场成像图对应的各微透镜成像的光心位置图。
为将光场重建图像I f中各微透镜成像进行整合以重建出完整的场景，需要确定I f中各微透镜成像的中心位置。以光源为样本直接照射成像系统，在确保光源经主透镜成像后足以覆盖整个微透镜阵列的前提下，各微透镜能够记录充足角度的光线，在传感器上成像为圆形，其圆心即为微透镜成像的中心，因此可以将光源的光场成像图作为计算各微透镜成像中心的光心位置图I c。
在拍摄时，要注意调整光源的入射方向，使各微透镜成像趋近于完整的圆形，此时的圆心才可认为是微透镜成像的中心；注意调整光源与成像系统之间的距离，使各微透镜成像之间不重叠。
S3：利用步骤S2采集到的光心位置图计算各微透镜成像的中心，进而对S1 采集到的样本的光场成像图进行渲染重建。
计算各微透镜成像中心的方法为，首先初始化与微透镜阵列大小相同的两个二维矩阵C x、C y，分别记录各微透镜成像中心的横、纵坐标。检索光心位置图I c中出现的圆，将识别到的圆的圆心坐标填入矩阵C x、C y的相应位置，作为该位置的微透镜成像的中心位置坐标，对未识别到圆的位置对应的矩阵值进行插值填充，作为该位置微透镜成像的中心位置坐标的估计值。
对光场成像图进行渲染重建的方法为：选择合适的长度l，取出以各微透镜成像中心为中心、边长为(2l+1)大小的矩形像素块，按照微透镜阵列的位置排布，将从各微透镜成像中取出的像素块有序拼接，相邻像素块的图像内容得以衔接，得到光场重建图像I r。调整长度l的大小，使重建图像细节明晰、锐利，长度l的取值应小于微透镜半径。重建时需注意，由于各微透镜成像是倒立的，因此在拼接之前需要对各像素块进行水平翻转。
S4：将步骤S3得到的光场重建图像变换到频域，并将低频分量排列至频谱中心；
利用傅里叶变换将光场重建图像I r由空间域变换至频域，得到光场重建图像频谱I F：
I F＝F(I r)
其中，F( )表示傅里叶变换。原始频谱中，高频分量分布在频谱中心，低频分量分布在频谱外围，难以对低频分量进行集中处理，因此对其重新排列，沿每个维度交换频谱的半空间，将零频移动至频谱中心。由于图像可以视作二维矩阵，频谱分布于四象限中，因此只需将原始频谱的一三象限交换，二四象限交换，即可得到低频分量排列于频谱中心的光场重建图像频谱，仍记为I F。
S5：对步骤S4变换后的光场重建图像频谱进行预处理，取幅值，并压缩值域。
经傅里叶变换得到的光场重建图像频谱I F为复数，取幅值，获得光场重建图像傅里叶变换的幅度谱|I F|。由于低频分量的频谱值过高，在二值化操作时会掩盖其他频谱分量，因此对光场重建图像傅里叶变换的幅度谱|I F|取对数，压缩值域。可见图2，得到预处理后的光场重建图像频谱I′ F：
I′ F＝log(|I F|+1)
S6：生成低通滤波器，保护以零频为中心，半径为r的圆形区域内的光场重建图像频谱的低频分量部分。
低通滤波器，是一个与光场重建图像频谱I F大小相同的二维矩阵Lp，由中心区域半径为r的圆划分为两部分：圆内矩阵值置0，表示待处理对象中该区域对应的分量受到抑制、被滤除；圆外矩阵值置1，表示待处理对象中该区域对应的分量不受影响、正常通过。由于经矩阵Lp处理得到的图像频谱在后续操作中被当作图像掩膜的初始态来对光场重建图像频谱I F进行提取，见步骤S9，且经图像掩膜过滤后，频谱I F中半径为r的圆形区域内的低频分量得以保留，因此矩阵Lp最终对频谱I F的处理作用表现为低通滤波器。处理时，可通过调节半径r的大小，控制希望保留的低频分量的范围。
S7：将步骤S6生成的低通滤波器乘以步骤S5预处理后的光场重建图像频谱，低频分量的频谱值置零。
将低通滤波器矩阵Lp与预处理后的光场重建图像频谱I′ F相乘，I′ F中心半径为r的圆形区域内的频谱值被置零，得到滤除低频分量的图像频谱I″ F：
I″ F＝Lp×I′ F
I″ F在后续操作中作为图像掩膜的初始态保护光场重建图像频谱I F的低频分量，低频分量为光场重建图像I r的有用信号部分。置零的低频分量可见图3中心的黑色圆形区域(黑色表示二值化后该位置频谱值为零)。
S8：对步骤S7得到的低频分量频谱值置零的光场重建图像频谱作二值化处理，得到图像掩膜。
设置二值化阈值Tr，对S7得到的图像频谱I″ F作二值化处理，频谱值高于Tr的部分即对应光场重建图像频谱I F需要滤除的频率分量，得到图像掩膜M，可见图3，其中，黑色表示二值化值为0，白色表示二值化值为1。
考虑到经低通滤波器处理后的图像频谱I″ F的低频分量的频谱值被置零，因此无论阈值Tr大小如何，二值化得到的图像掩膜M的低频分量对应值一定为0。对于上述低频分量之外的其他频率分量，由于拼接边缘噪声具有较强的周期性，对应在频谱图上表现为，部分高频分量处频谱值较大，在选择合适大小的Tr的前提下，二值化后该部分高频分量对应的图像掩膜M的值为1。处理时，可通过改变阈值Tr的大小，调节图像掩膜M中值为1的频率分量的占比，进而调节对噪声的抑制程度。
S9：利用步骤S8生成的图像掩膜，对步骤S4得到的光场重建图像原始频谱进行过滤，滤除频谱中的高频周期噪声分量。
利用步骤S8生成的图像掩膜M，对步骤S4得到的光场重建图像频谱I F进行过滤，若某频率分量对应的图像掩膜M的值为1，则将该分量在图像频谱I F中对应位置的频谱值置零，得到滤除高频周期噪声分量的光场重建图像频谱I Fd：
Figure PCTCN2020115325-appb-000002
其中，i＝1…r，j＝1…c，r、c分别为光场重建图像I r的行数和列数。
S10：将步骤S9滤除高频周期噪声分量后的光场重建图像频谱变换回空间域，得到去除周期噪声的光场重建图像。
对滤除了高频周期噪声分量的光场重建图像频谱I Fd重新排列，利用傅里叶逆变换从频域变换回空间域，得到去除了周期噪声的光场重建图像I rd：
I rd＝F -1(I Fd)
其中F -1( )表示傅里叶逆变换。

Claims (7)

  1. 基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，包括如下步骤：
    S1，利用光场成像系统采集样本的光场成像图；
    S2，移除样本，将光源直接照射光场成像系统，采集光场成像图对应的各微透镜成像的光心位置图；
    S3，利用所述光心位置图标定各微透镜成像的中心，进而对步骤S1采集的光场成像图进行渲染重建，得到光场重建图像；
    S4，将所述光场重建图像变换到频域，并将低频分量排列至频谱中心；
    S5，对步骤S4变换后的光场重建图像频谱进行预处理，即取幅值并压缩值域；
    S6，生成低通滤波器，保护以零频为中心、半径为r的圆形区域内的光场重建图像频谱的低频分量部分；
    S7，将所述低通滤波器乘以步骤S5预处理后的光场重建图像频谱，低频分量的频谱值置零；
    S8，对经步骤S7处理后得到的光场重建图像频谱作二值化处理，得到图像掩膜；
    S9，利用所述图像掩膜，对步骤S4变换得到的光场重建图像原始频谱进行过滤，滤除频谱中的高频周期噪声分量；
    S10，将过滤后的光场重建图像频谱变换回空间域，得到去除周期噪声的光场重建图像。
  2. 根据权利要求1所述的基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，所述步骤S2中，采集光场成像图对应的各微透镜成像的光心位置图的具体过程为：
    以光源为样本直接照射成像系统，在确保光源经主透镜成像后足以覆盖整个微透镜阵列的前提下，各微透镜能够记录充足角度的光线，在传感器上成像为圆形，其圆心即为微透镜成像的中心，因此将光源的光场成像图作为计算各微透镜成像中心的光心位置图；调整光源与成像系统之间的距离，使各微透镜成像之间不重叠。
  3. 根据权利要求1所述的基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，所述步骤S3中，利用光心位置图标定各微透镜成像的中心的 具体方法为：
    首先初始化与微透镜阵列大小相同的两个二维矩阵C x、C y，分别记录各微透镜成像中心的横、纵坐标；检索光心位置图中出现的圆，将识别到的圆的圆心坐标填入矩阵C x、C y的相应位置，作为该位置的微透镜成像的中心位置坐标，对未识别到圆的位置对应的矩阵值进行插值填充，作为该位置微透镜成像的中心位置坐标的估计值。
  4. 根据权利要求1所述的基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，所述步骤S3中，对光场成像图进行渲染重建的具体方法为：
    在光场成像图中取出以各微透镜成像中心为中心、边长为(2l+1)大小的矩形像素块，按照微透镜阵列的位置排布，将从各微透镜成像中取出的像素块先水平翻转后再进行有序拼接，得到光场重建图像；其中，长度l的取值应小于微透镜半径。
  5. 根据权利要求1所述的基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，所述步骤S6的具体方法为：
    初始化一个与光场重建图像频谱大小相同的二维矩阵，该二维矩阵由其中心位置一个半径为r的圆划分为两部分：圆内矩阵值置0，表示待处理对象中该区域对应的分量受到抑制并被滤除；圆外矩阵值置1，表示待处理对象中该区域对应的分量不受影响并正常通过。
  6. 根据权利要求1所述的基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，所述步骤S8中，图像掩膜的具体生成方法为：
    将低通滤波器矩阵Lp与预处理后的光场重建图像频谱I′ F相乘，得到低频分量频谱值置零的光场重建图像频谱I″ F：
    I″ F＝Lp×I′ F
    设置二值化阈值Tr，对低频分量频谱值置零的光场重建图像频谱I″ F作二值化处理，得到图像掩膜M，M值为1的部分即对应光场重建图像频谱I F需要滤除的频率分量；通过改变阈值Tr的大小，调节图像掩膜M中值为1的频率分量的占比，进而调节对噪声的抑制程度。
  7. 根据权利要求1所述的基于频域的去除光场重建图像中的周期噪声的方法，其特征在于，所述步骤S9中，若某频率分量对应的图像掩膜M的值为1，则将该分量在光场重建图像频谱I F中对应位置的频谱值置零，得到滤除高频周期 噪声分量的光场重建图像频谱I Fd：
    Figure PCTCN2020115325-appb-100001
    其中，i＝1…r，j＝1…c，r、c分别为光场重建图像I r的行数和列数。
PCT/CN2020/115325 2019-10-21 2020-09-15 基于频域的去除光场重建图像中的周期噪声的方法 WO2021077944A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/754,874 US20230306558A1 (en) 2019-10-21 2020-09-15 Frequency domain-based method for removing periodic noise from reconstructed light field image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910998260.5 2019-10-21
CN201910998260.5A CN110866874B (zh) 2019-10-21 2019-10-21 基于频域的去除光场重建图像中的周期噪声的方法

Publications (1)

Publication Number Publication Date
WO2021077944A1 true WO2021077944A1 (zh) 2021-04-29

Family

ID=69652691

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/115325 WO2021077944A1 (zh) 2019-10-21 2020-09-15 基于频域的去除光场重建图像中的周期噪声的方法

Country Status (3)

Country Link
US (1) US20230306558A1 (zh)
CN (1) CN110866874B (zh)
WO (1) WO2021077944A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092509A (zh) * 2021-11-12 2022-02-25 湖南大学 一种医药高光谱数据的图像重构方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110866874B (zh) * 2019-10-21 2021-07-30 南京大学 基于频域的去除光场重建图像中的周期噪声的方法
CN112184578A (zh) * 2020-09-23 2021-01-05 南京大学 一种标定光场成像中各微透镜成像的中心坐标的方法
CN113689361B (zh) * 2021-10-27 2022-02-22 深圳市慧鲤科技有限公司 图像处理方法及装置、电子设备和存储介质
CN116777750B (zh) * 2023-07-31 2023-12-26 山东黄河三角洲国家级自然保护区管理委员会 一种基于视差维滤波的聚焦堆栈视差维超分辨方法和装置
CN117314798B (zh) * 2023-11-29 2024-05-03 国药新光医疗科技有限公司 一种去除纤维镜图像网格的方法
CN118365626A (zh) * 2024-05-10 2024-07-19 中国科学院过程工程研究所 一种基于sam神经网络的多相流分散相识别和补全方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103679643A (zh) * 2013-06-03 2014-03-26 哈尔滨工程大学 一种多条纹噪声定位滤除方法
US20170078558A1 (en) * 2015-09-16 2017-03-16 Canon Kabushiki Kaisha Image capturing apparatus, method for controlling an image capturing apparatus, and storage medium
CN108337434A (zh) * 2018-03-27 2018-07-27 中国人民解放军国防科技大学 一种针对光场阵列相机的焦外虚化重聚焦方法
CN109712107A (zh) * 2017-10-26 2019-05-03 北京航天长峰科技工业集团有限公司 一种监控视频条纹快速检测方法
CN110866874A (zh) * 2019-10-21 2020-03-06 南京大学 基于频域的去除光场重建图像中的周期噪声的方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6995387B2 (en) * 2004-05-18 2006-02-07 Agfa-Gevaert Suppression of periodic variations in a digital signal
US8831377B2 (en) * 2012-02-28 2014-09-09 Lytro, Inc. Compensating for variation in microlens position during light-field image processing
CN104871210B (zh) * 2013-01-18 2018-11-13 英特尔公司 用于散焦模糊的方差估计光场重构
WO2018067212A2 (en) * 2016-06-20 2018-04-12 Massachusetts Institute Of Technology Methods and systems for time-encoded multiplexed imaging
CN107481205B (zh) * 2017-08-23 2020-06-09 电子科技大学 一种太赫兹图像条纹噪声处理方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103679643A (zh) * 2013-06-03 2014-03-26 哈尔滨工程大学 一种多条纹噪声定位滤除方法
US20170078558A1 (en) * 2015-09-16 2017-03-16 Canon Kabushiki Kaisha Image capturing apparatus, method for controlling an image capturing apparatus, and storage medium
CN109712107A (zh) * 2017-10-26 2019-05-03 北京航天长峰科技工业集团有限公司 一种监控视频条纹快速检测方法
CN108337434A (zh) * 2018-03-27 2018-07-27 中国人民解放军国防科技大学 一种针对光场阵列相机的焦外虚化重聚焦方法
CN110866874A (zh) * 2019-10-21 2020-03-06 南京大学 基于频域的去除光场重建图像中的周期噪声的方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092509A (zh) * 2021-11-12 2022-02-25 湖南大学 一种医药高光谱数据的图像重构方法

Also Published As

Publication number Publication date
CN110866874A (zh) 2020-03-06
CN110866874B (zh) 2021-07-30
US20230306558A1 (en) 2023-09-28

Similar Documents

Publication Publication Date Title
WO2021077944A1 (zh) 基于频域的去除光场重建图像中的周期噪声的方法
RU2431889C1 (ru) Способ суперразрешения изображений и нелинейный цифровой фильтр для его осуществления
CN110023810B (zh) 光学系统像差的数字校正
Liang et al. Programmable aperture photography: multiplexed light field acquisition
Raskar et al. Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses
CN111145134B (zh) 基于块效应的微透镜光场相机全聚焦图像生成算法
JP6545670B2 (ja) デジタルカメラに対する単一フレームベースのスーパー解像度補間のためのシステムおよび方法
CN110517206B (zh) 彩色摩尔纹消除方法及装置
Akyüz Deep joint deinterlacing and denoising for single shot dual-ISO HDR reconstruction
CN111080774B (zh) 一种应用深度采样进行光场重构的方法及系统
WO2016175044A1 (ja) 画像処理装置及び画像処理方法
Chen et al. Weighted sparse representation and gradient domain guided filter pyramid image fusion based on low-light-level dual-channel camera
US9258460B2 (en) Image processing method and apparatus for correcting an image
CN112651911A (zh) 一种基于偏振图像的高动态范围成像生成方法
Liang et al. Guidance network with staged learning for image enhancement
Alam et al. Reduction of glare in images with saturated pixels
Li et al. Overview of multi-exposure image fusion
CN108307122A (zh) 多摄像头的降噪拍摄方法、拍摄装置和移动终端及可读存储介质
Singh et al. Detail Enhanced Multi-Exposer Image Fusion Based on Edge Perserving Filters
CN111738933A (zh) 一种红外数字全息重建中的噪声抑制方法及装置
CN118138902B (zh) 基于偏振特性结合光场成像去除镜面反射的成像方法和系统
Soulez et al. Joint deconvolution and demosaicing
Moon et al. A fast low-light multi-image fusion with online image restoration
LU102214B1 (en) Method for digital image processing
CN110031972B (zh) 一种基于菲涅尔光圈的无镜头红外图像处理方法、存储介质和终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20879481

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20879481

Country of ref document: EP

Kind code of ref document: A1