WO2023231140A1 - 用于元成像的cis系统 - Google Patents

用于元成像的cis系统 Download PDF

Info

Publication number
WO2023231140A1
WO2023231140A1 PCT/CN2022/103915 CN2022103915W WO2023231140A1 WO 2023231140 A1 WO2023231140 A1 WO 2023231140A1 CN 2022103915 W CN2022103915 W CN 2022103915W WO 2023231140 A1 WO2023231140 A1 WO 2023231140A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
mask
light field
imaging
microlens array
Prior art date
Application number
PCT/CN2022/103915
Other languages
English (en)
French (fr)
Inventor
蔡娅雯
郭泽群
Original Assignee
元潼(北京)技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 元潼(北京)技术有限公司 filed Critical 元潼(北京)技术有限公司
Publication of WO2023231140A1 publication Critical patent/WO2023231140A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B3/00—Simple or compound lenses
    • G02B3/0006—Arrays

Definitions

  • This application relates to the field of computational imaging technology, and in particular to a CIS (CMOS Image Sensor) system for meta-imaging.
  • CIS CMOS Image Sensor
  • light field imaging technology In recent years, with the improvement of computer computing power, computational light field imaging technology has developed rapidly. Among related technologies, light field mainly focuses on the distribution function of light as position and angle. Benefiting from the space-viewing dimension assumption and advances in software and hardware technology, light field imaging can now be realized by portable commercial equipment and can even be integrated into mobile phones. .
  • the collection of 4D light field information mainly involves encoding the 4D light field into the 2D sensor plane by multiplexing the angle domain into the space (or frequency) domain.
  • the light field collection device is called a "light field chip” and is mainly installed using This is achieved with a microlens array or lenslet array on the image sensor.
  • Light field cameras can obtain spatial and angular 4D light field information through a single exposure, but light field chips face an inherent contradiction between spatial resolution and angular resolution.
  • light field acquisition equipment using the microlens array principle such as Lytro Illume and Raytrix
  • Signal sampling in related technologies follows the Nyquist sampling theorem.
  • the amount of data collected in this sampling method is large and requires a large amount of transmission bandwidth and storage space.
  • the scanning light field imaging method improves the spatial resolution of light field imaging while ensuring angular resolution by sacrificing time resolution.
  • the existing scanning imaging method has many scanning times and low imaging efficiency.
  • problems such as spectrum aliasing leading to information loss and imaging quality degradation.
  • compressed sensing technology can greatly improve the shortcomings of low spatial resolution of light fields captured in non-compressed light fields.
  • a layer of random mask is added between the camera lens and the sensor to realize the compressed collection of the target light field.
  • the compressed sensing theory and the over-complete light field dictionary are combined to perform nonlinear optimization of the compressed coded image. Dense light fields can be recovered offline.
  • One way compression coding is achieved is by adding mask modulation to the imaging process to achieve better sampling.
  • This application provides a CIS system for meta-imaging, which introduces more uniform spectrum aliasing through mask modulation to solve the problem of reduced light field reconstruction quality caused by information aliasing in the existing microlens array light field acquisition system.
  • a first embodiment of the present application provides a CIS system for meta-imaging, including: a main imaging system, a microlens array, an image acquisition unit, and a first mask disposed before and after the microlens array, including on the microlens array. ,in,
  • the main imaging system is used to obtain the first image of the target object
  • the microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object;
  • the first mask is used to perform mask modulation on the second image through a preset first mask modulation function, introduce preset spectrum aliasing, and obtain the light field information of the target object, so that the The image acquisition unit generates a final image based on the light field information.
  • the above-mentioned CIS system for meta-imaging also includes:
  • Mask modulation to obtain the first image for phase modulation.
  • the wave function of the first image is:
  • x, y, z are the three-dimensional space distance coordinates of the object point
  • z means depth
  • ⁇ is the wavelength
  • r is the radial distance of the aperture plane
  • J 0 ( ⁇ ) is the first kind of zero-order Bessel function
  • s is the distance between the main imaging system and the microlens array
  • d is the distance between the object and the main imaging system
  • D(r, ⁇ , z) models the changes in imaging response at a distance z from the main imaging system.
  • the preset phase modulation function is:
  • x 0 and y 0 are the center coordinates of the microlens array
  • f is the focal length
  • n is the refractive index
  • rext(.) is the rectangular window function
  • i is the imaginary part
  • exp(.) is the exponential function.
  • the preset first mask modulation function is:
  • m 2 (x, y) is the mask modulation function provided before and after the microlens array, including on the microlens array, and before the image acquisition unit.
  • the light field information is:
  • ⁇ x , ⁇ y are the frequency domain sampling of space (x, y)
  • F ⁇ (.) is the Fourier transform operation
  • m 1 (r) is set near the main imaging system including the main imaging lens
  • the microlens array is preceded by a second mask modulation function.
  • generating a final image using the light field information on the image acquisition unit includes:
  • the light field information is used to generate a final image on the image acquisition unit, where the imaging response is:
  • ⁇ u is the spatial frequency position corresponding to angle u
  • ⁇ v is the spatial frequency position corresponding to angle v
  • s ( ⁇ u , ⁇ v ) is the acquisition process of specific frequency components by camera pixels.
  • the image acquisition unit is a CMOS Image Sensor (CIS).
  • CIS CMOS Image Sensor
  • the main imaging system is used to obtain the first image of the target object, and the microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function,
  • the second image of the target object is obtained.
  • the first mask is used to perform mask modulation on the second image through the preset first mask modulation function, and introduces the preset spectrum aliasing to obtain the light field information of the target object, such that
  • the image acquisition unit generates the final image based on the light field information.
  • the light field collection method of adding a mask based on two microlens arrays improves the sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problem of adding a mask between the camera lens and the sensor to achieve light field collection in the camera array. Problems such as high cost and large size.
  • a second embodiment of the present application provides an imaging method of a CIS system for meta-imaging, using the CIS system for meta-imaging as described in the embodiment of the first aspect.
  • the method includes the following steps:
  • the first mask modulation is performed on the second image through the first mask modulation function, and preset spectrum aliasing is introduced to obtain the light field information of the target object, and the light field information is
  • the image acquisition unit forms a final image.
  • the above-mentioned imaging method of the CIS system for meta-imaging also includes:
  • Mask modulation is performed on the first image through the second mask using a preset second mask modulation function to obtain a first image for phase modulation.
  • the main imaging system is used to obtain the first image of the target object
  • the microlens array is used to use the preset phase modulation function to perform the optical path of the first image.
  • Phase modulation is used to obtain the second image of the target object.
  • the first mask is used to perform mask modulation on the second image through the preset first mask modulation function to obtain the light field information of the target object, so that the image acquisition unit can field information generates the final image.
  • the light field collection method of adding a mask based on two microlens arrays improves the sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problem of adding a mask between the camera lens and the sensor to achieve light field collection in the camera array. Problems such as high cost and large size.
  • Figure 1 is a schematic diagram of the design of a series of light field masks for traditional camera arrays in related technologies
  • Figure 2 is a block schematic diagram of an imaging system of a CIS system for meta-imaging provided according to an embodiment of the present application
  • Figure 3 is a schematic diagram of the design of a microlens array mask CMOS image sensor system according to an embodiment of the present application
  • Figure 4 is a schematic diagram of the design of a microlens array-optical lens combination mask CMOS image sensor system according to an embodiment of the present application
  • FIG. 5 is a flow chart of an imaging method of a CIS system for meta-imaging according to an embodiment of the present application.
  • this application provides a CIS system for meta-imaging, which uses the main imaging
  • the system is used to obtain a first image of the target object.
  • the microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object.
  • the first mask is used to pass the preset phase modulation function.
  • the first mask modulation function performs mask modulation on the second image, introduces preset spectrum aliasing, and obtains the light field information of the target object, so that the image acquisition unit generates a final image based on the light field information.
  • the light field collection method of adding a mask based on two microlens arrays improves the sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problem of adding a mask between the camera lens and the sensor to achieve light field collection in the camera array. Problems such as high cost and large size.
  • FIG. 2 is a block diagram of a CIS system for meta-imaging provided by an embodiment of the present application.
  • the CIS system 10 for meta-imaging includes: a main imaging system 100 , a microlens array 200 , an image acquisition unit 400 , and a first image sensor disposed before and after the microlens array 200 .
  • Mask 300 As shown in FIG. 2 , the CIS system 10 for meta-imaging includes: a main imaging system 100 , a microlens array 200 , an image acquisition unit 400 , and a first image sensor disposed before and after the microlens array 200 .
  • the main imaging system 100 is used to obtain the first image of the target object; the microlens array 200 is used to use a preset phase modulation function to phase modulate the optical path of the first image, introduce preset spectrum aliasing, and obtain the target object. the second image; the first mask 300 is used to perform mask modulation on the second image through the preset first mask modulation function, introduce preset spectrum aliasing, and obtain the light field information of the target object, so that image acquisition Unit 400 generates a final image based on the light field information.
  • the image acquisition unit 400 in the embodiment of the present application can use a CMOS image sensor.
  • other image acquisition devices with image acquisition functions can also be used. Here, No specific restrictions are made.
  • the wave function of the first image is:
  • x, y, z are the three-dimensional space distance coordinates of the object point
  • z means depth
  • ⁇ is the wavelength
  • r is the radial distance of the aperture plane
  • J 0 ( ⁇ ) is the first kind of zero-order Bessel function
  • s is the distance between the main imaging system and the microlens array
  • d is the distance between the object and the main imaging system
  • D(r, ⁇ , z) models the changes in imaging response at a distance z from the main imaging system.
  • the preset phase modulation function is:
  • x 0 and y 0 are the center coordinates of the microlens array
  • f is the focal length
  • n is the refractive index
  • rect(.) is the rectangular window function
  • i is the imaginary part
  • exp(.) is the exponential function.
  • the preset first mask modulation function is:
  • m 2 (x, y) is the mask modulation function provided before and after the microlens array 200 , including on the microlens array 200 , and before the image acquisition unit 400 .
  • the resolution of each camera is changed by using a camera array light field mask to control the size of the light field spatial resolution.
  • a camera array light field mask to control the size of the light field spatial resolution.
  • the embodiment of the present application adopts two light field collection methods by adding masks based on the microlens array 200 to improve the sampling efficiency and light field imaging quality.
  • spectral information can be moved and reduced. Frequency domain response zero point improves light field imaging quality and imaging resolution.
  • a mask that is, the first mask 300 , before and after the microlens array 200 , including on the microlens array 200 and before the image acquisition unit 400 , and in the main imaging system 100
  • a mask that is, a second mask, is added nearby including the main imaging lens and before the microlens array 200, thereby obtaining comprehensive light source modulation information and angle domain modulation information, optimizing the response of the imaging system, and improving the light field imaging quality and imaging resolution.
  • the first image of the target object is obtained through the main imaging system; secondly, the microlens array 200 uses a preset phase modulation function to phase modulate the optical path of the first image, thereby obtaining Second image of the target object.
  • the wave function of the first image passing through the main imaging system can be expressed as:
  • x, y, z are the three-dimensional space distance coordinates of the object point, z means depth, ⁇ is the wavelength, r is the radial distance of the aperture plane, and J 0 ( ⁇ ) is the first kind of zero-order Bessel function.
  • D(r, ⁇ ,z) can be expressed as:
  • s is the distance between the main imaging system and the microlens array
  • d is the distance between the object and the main imaging system.
  • the preset phase modulation function can be expressed as:
  • x 0 and y 0 are the center coordinates of the microlens array
  • f is the focal length
  • n is the refractive index
  • rect(.) is the rectangular window function
  • i is the imaginary part
  • exp(.) is the exponential function.
  • the preset first mask modulation function is:
  • m 2 (x, y) is the mask modulation function provided before and after the microlens array, including on the microlens array, and before the image acquisition unit 400 .
  • the above-mentioned CIS system 10 for primary imaging also includes: a second mask disposed near the main imaging system 100 including on the main imaging lens and in front of the microlens array 200.
  • the film is used to perform mask modulation on the first image using a preset second mask modulation function to obtain a first image for phase modulation.
  • the light field information is:
  • ⁇ x , ⁇ y are the frequency domain sampling of the space (x, y)
  • F ⁇ (.) is the Fourier transform operation
  • m 1 (r) is set near the main imaging system 100 including the main imaging lens.
  • the second mask modulation function before the microlens array 200 is set near the main imaging system 100 including the main imaging lens.
  • generating a final image with the light field information on the image acquisition unit 400 includes: generating a final image with the light field information on the image acquisition unit 400 based on the imaging response, where the imaging response is :
  • ⁇ u is the spatial frequency position corresponding to angle u
  • ⁇ v is the spatial frequency position corresponding to angle v
  • s ( ⁇ u , ⁇ v ) is the acquisition process of specific frequency components by camera pixels.
  • the embodiment of the present application first uses the first mask 300 disposed before and after the microlens array 200 including the microlens array 200, and uses the preset first mask modulation function to perform correlation calculation on the target object obtained above.
  • Mask modulation is performed on the second image, and a more uniform spectrum aliasing is introduced to avoid the degradation of light field reconstruction quality caused by information aliasing in the light field acquisition system of the microlens array 200, thereby obtaining the light of the target object.
  • Field information This light field information contains the four-dimensional position and direction information of light radiation during the propagation process. The digital refocusing technology solves the problem of image defocus in special occasions.
  • a final image is generated on the image acquisition unit 400.
  • ⁇ x , ⁇ y are the frequency domain sampling of the space (x, y)
  • F ⁇ (.) is the Fourier transform operation
  • m 1 (r) is set near the main imaging system 100 including the main imaging lens.
  • the second mask modulation function before the microlens array 200 is set near the main imaging system 100 including the main imaging lens.
  • the imaging response can be expressed as:
  • ⁇ u is the spatial frequency position corresponding to angle u
  • ⁇ v is the spatial frequency position corresponding to angle v
  • s ( ⁇ u , ⁇ v ) is the acquisition process of specific frequency components by camera pixels.
  • the above-mentioned mask placement positions are only exemplary, and those skilled in the art can set the mask positions according to actual conditions.
  • the second mask is placed on the left side of the main imaging system, and immediately The first mask is attached to the main imaging system, and is disposed between the main imaging system and the microlens array, and is close to the microlens array.
  • the main imaging system is used to obtain the first image of the target object, and the microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function,
  • the second image of the target object is obtained.
  • the first mask is used to perform mask modulation on the second image through the preset first mask modulation function, and introduces the preset spectrum aliasing to obtain the light field information of the target object, such that
  • the image acquisition unit generates the final image based on the light field information.
  • the light field collection method of adding a mask based on two microlens arrays improves the sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problem of adding a mask between the camera lens and the sensor to achieve light field collection in the camera array. Problems such as high cost and large size.
  • FIG. 5 is a flow chart of an imaging method of a CIS system for meta-imaging according to an embodiment of the present application.
  • the imaging method of the CIS system for meta-imaging adopts the CIS system for meta-imaging as mentioned above.
  • the method includes the following steps:
  • step S501 the first image of the target object is obtained through the main imaging system
  • step S502 the microlens array is used to phase modulate the optical path of the first image through a phase modulation function to obtain a second image of the target object;
  • step S503 perform first mask modulation on the second image through the first mask modulation function, introduce preset spectrum aliasing, obtain the light field information of the target object, and generate the light field information on the image acquisition unit. A final image.
  • the above-mentioned imaging method of the CIS system for meta-imaging also includes:
  • Mask modulation is performed on the first image through the second mask using a preset second mask modulation function to obtain a first image for phase modulation.
  • the main imaging system is used to obtain the first image of the target object
  • the microlens array is used to use the preset phase modulation function to perform the optical path of the first image.
  • Phase modulation is used to obtain the second image of the target object.
  • the first mask is used to perform mask modulation on the second image through the preset first mask modulation function, and introduces preset spectrum aliasing to obtain the light field of the target object. information, allowing the image acquisition unit to generate the final image based on the light field information.
  • the light field collection method of adding a mask based on two microlens arrays improves the sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problem of adding a mask between the camera lens and the sensor to achieve light field collection in the camera array. Problems such as high cost and large size.
  • references to the terms “one embodiment,” “some embodiments,” “an example,” “specific examples,” or “some examples” or the like means that specific features are described in connection with the embodiment or example. , structures, materials or features are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification unless they are inconsistent with each other.
  • first and second are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as “first” and “second” may explicitly or implicitly include at least one of these features. In the description of this application, “N” means at least two, such as two, three, etc., unless otherwise clearly and specifically limited.
  • a "computer-readable medium” may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
  • Non-exhaustive list of computer readable media include the following: electrical connections with one or N wires (electronic device), portable computer disk cartridge (magnetic device), random access memory (RAM), Read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM).
  • the computer-readable medium may even be paper or other suitable medium on which the program may be printed, as the program may be printed, for example, by optical scanning of the paper or other medium, followed by editing, interpretation, or in other suitable manner if necessary Processing to obtain a program electronically and then store it in computer memory.
  • N steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: discrete logic gate circuits with logic functions for implementing data signals; Logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.
  • each functional unit in various embodiments of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules. Integrated modules can also be stored in a computer-readable storage medium if they are implemented in the form of software function modules and sold or used as independent products.
  • the storage media mentioned above can be read-only memory, magnetic disks or optical disks, etc.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Studio Devices (AREA)

Abstract

一种涉及计算成像技术领域的用于元成像的CIS系统（10），包括：主成像系统（100）、微透镜阵列（200）、图像采集单元（400）、设置于微透镜阵列（200）前后包括微透镜阵列（200）上的第一掩膜（300）和设置于主成像系统（100）附近包括主成像透镜上、微透镜阵列（200）之前的第二掩膜，其中，主成像系统（100）用于得到目标物体的第一图像；第一掩膜（300）用于对第一图像进行幅度调制，微透镜阵列（200）用于利用预设的相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像；第一掩膜（300）用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，引入更均一的频谱混叠，得到目标物体的光场信息，使得图像采集单元（400）根据光场信息生成最终图像。由此，基于两种微透镜阵列（200）增加掩膜的光场采集方式，提升了采样效率和光场成像与重建质量，同时也解决了相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题。

Description

用于元成像的CIS系统
相关申请的交叉引用
本申请基于申请号为202210602153.8，申请日为2022年05月30日申请的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及计算成像技术领域，特别涉及一种用于元成像的CIS(CMOS Image Sensor，图像传感器)系统。
背景技术
近几年来，随着计算机计算能力的提高，计算光场成像技术得到了快速地发展。相关技术中，光场主要关注于光线作为位置和角度的分布函数，受益于空间-视角维度假设以及软硬件技术的进步，如今光场成像可由便携化的商用设备实现，甚至可以集成到手机中。
光场相机的广泛使用可以使一些新的应用成为可能，包括最初的基于真实感图像的光场渲染以及现在利用光场编码信息的计算机视觉应用，例如3D重建、分割、显著性检测、物体检测与识别、跟踪和视频增稳等。鉴于光场巨大的使用潜力以及待攻克的技术难题，光场技术受到了国内外研究团队的广泛关注。
4D光场信息的采集主要是通过角域复用到空间(或频率)域的方式，将4D光场编码到2D传感器平面，其光场采集设备称为“光场芯片”，主要是使用安装在图像传感器上的微透镜阵列或小透镜阵列来实现。光场相机能够通过单次曝光获得空间和角度的4D光场信息，但光场芯片面临空间分辨率与角度分辨率的固有矛盾。一方面，采用微透镜阵列原理的光场采集设备(如Lytro Illume与Raytrix)通过将各个角度的光线直接分散到成像单元上，来获取高角度分辨率的光场图像，但受到成像单元分辨率的限制，其空间分辨率往往远小于如今比较通用的成像设备(单反相机、手机等)；另一方面，如图1所示，相机阵列系统虽能够通过改变每个相机的分辨率来控制光场空间分辨率的大小，但受到相机物理空间以及系统复杂度、整体成本等因素的限制，其角度分辨率远远无法满足实时光场渲染的要求。
相关技术中的信号采样遵循奈奎斯特采样定理，这种采样方式下的采集数据量较大，需要占用较多的传输带宽和存储空间。扫描光场成像方式通过牺牲时间分辨率的扫描方式，在保证角度分辨率的同时提高光场成像的空间分辨率。然而现有扫描成像方式扫描次数多， 成像效率低，在采集过程中存在频谱混叠导致信息缺失，成像质量下降的问题。在传感器分辨率不变的前提下，压缩感知技术能够很大程度地改善非压缩光场拍摄的光场空间分辨率低的不足。基于压缩光场的拍摄技术，通过在相机镜头与传感器之间增加一层随机掩膜实现对目标光场的压缩采集，结合压缩感知理论与过完备光场字典对压缩编码图像进行非线性优化，可离线地恢复出稠密光场。压缩编码实现的一种方式即为在成像过程中增加掩膜调制以获得更优采样。
发明内容
本申请提供一种用于元成像的CIS系统，通过掩膜调制引入更均一的频谱混叠，以解决现有微透镜阵列光场采集系统中信息混叠导致的光场重建质量下降的问题。
本申请第一方面实施例提供一种用于元成像的CIS系统，包括：主成像系统、微透镜阵列、图像采集单元和设置于所述微透镜阵列前后包括微透镜阵列上的第一掩膜，其中，
所述主成像系统用于得到目标物体的第一图像；
所述微透镜阵列用于利用预设的相位调制函数对所述第一图像的光路进行相位调制，得到所述目标物体的第二图像；
所述第一掩膜用于通过预设的第一掩膜调制函数对所述第二图像进行掩膜调制，引入预设的频谱混叠，得到所述目标物体的光场信息，使得所述图像采集单元根据所述光场信息生成最终图像。
可选地，上述的用于元成像的CIS系统，还包括：
设置于所述主成像系统中主成像透镜上、所述微透镜阵列之前的第二掩膜，所述第二掩膜用于利用预设的第二掩膜调制函数对所述第一图像进行掩膜调制，得到用于相位调制的第一图像。
可选地，所述第一图像的波函数为：
Figure PCTCN2022103915-appb-000001
其中，x,y,z为物点的三维空间距离坐标，z含义为深度，λ为波长，r为孔径平面径向距离，J 0(·)为第一类零阶贝塞尔函数，s为主成像系统与微透镜阵列之间距离，d为物体与主成像系统距离，D(r,λ,z)为建模了距离主成像系统距离z处的成像响应变化情况，表达式如下式：
Figure PCTCN2022103915-appb-000002
可选地，所述预设的相位调制函数为：
Figure PCTCN2022103915-appb-000003
其中，x 0，y 0为微透镜阵列的中心坐标，f为焦距，n为折射率，rext(.)为矩形窗函数，i为虚部，exp(.)为指数函数。
可选地，所述预设的第一掩膜调制函数为：
Figure PCTCN2022103915-appb-000004
其中，m 2(x,y)为设置于微透镜阵列前后包括微透镜阵列上、图像采集单元之前的掩膜调制函数。
可选地，所述光场信息为：
U′(ω x,ω y)＝F ω(U 0(x,y,z,λ)m 1(r)·t(x,y,x 0,y 0))；
其中，ω x,ω y为对空间(x,y)的频域采样，F ω(.)为傅里叶变换操作，m 1(r)为设置于主成像系统附近包括主成像透镜上、所述微透镜阵列之前的第二掩膜调制函数。
可选地，所述将所述光场信息在所述图像采集单元上生成一个最终图像，包括：
基于成像响应，将所述光场信息在所述图像采集单元上生成一个最终图像，其中，所述成像响应为：
Figure PCTCN2022103915-appb-000005
其中，ω u为角度u对应的空间频率位置，ω v为角度v对应的空间频率位置，s(ω u,ω v)为相机像素对特定频率分量的采集过程。
可选地，所述图像采集单元为CMOS图像传感器(CMOS Image Sensor，CIS)。
根据本申请实施例的用于元成像的CIS系统，通过主成像系统用于得到目标物体的第一图像，微透镜阵列用于利用预设的相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像，第一掩膜用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，引入预设的频谱混叠，得到目标物体的光场信息，使得图像采集单元根据光场信息生成最终图像。由此，基于两种微透镜阵列增加掩膜的光场采集方式，提升了采样效率和光场成像与重建质量，同时也解决了相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题。
本申请第二方面实施例提供一种用于元成像的CIS系统的成像方法，采用如第一方面实施例所述的用于元成像的CIS系统，所述方法包括以下步骤：
通过所述主成像系统得到所述目标物体的第一图像；
利用所述微透镜阵列，通过所述相位调制函数对所述第一图像的光路进行相位调制， 得到所述目标物体的第二图像；
通过所述第一掩膜调制函数对所述第二图像进行所述第一掩膜调制，引入预设的频谱混叠，得到所述目标物体的光场信息，并将所述光场信息在所述图像采集单元上成一个最终图像。
可选地，上述的用于元成像的CIS系统的成像方法，还包括：
通过所述第二掩膜利用预设的第二掩膜调制函数对所述第一图像进行掩膜调制，得到用于相位调制的第一图像。
根据本申请实施例的用于元成像的CIS系统的成像方法，通过主成像系统用于得到目标物体的第一图像，微透镜阵列用于利用预设的相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像，第一掩膜用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，得到目标物体的光场信息，使得图像采集单元根据光场信息生成最终图像。由此，基于两种微透镜阵列增加掩膜的光场采集方式，提升了采样效率和光场成像与重建质量，同时也解决了相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为相关技术中的传统相机阵列光场掩膜系列设计示意图；
图2为根据本申请实施例提供的一种用于元成像的CIS系统的成像系统的方框示意图；
图3为根据本申请一个实施例提供的微透镜阵列掩膜CMOS图像传感器系统设计示意图；
图4为根据本申请一个实施例提供的微透镜阵列-光学透镜组合掩膜CMOS图像传感器系统设计示意图；
图5为根据本申请实施例的用于元成像的CIS系统的成像方法的流程图。
具体实施方式
下面详细描述本申请的实施例，实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
下面参考附图描述本申请实施例的用于元成像的CIS系统。针对上述背景技术中提到的相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题，本申请提供了一种用于元成像的CIS系统，通过主成像系统用于得到目标物体的第一图像，微透镜阵列用于利用预设的相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像，第一掩膜用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，引入预设的频谱混叠，得到目标物体的光场信息，使得图像采集单元根据光场信息生成最终图像。由此，基于两种微透镜阵列增加掩膜的光场采集方式，提升了采样效率和光场成像与重建质量，同时也解决了相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题。
具体而言，图2为本申请实施例所提供的一种用于元成像的CIS系统的方框示意图。
如图2所示，该用于元成像的CIS系统10，包括：主成像系统100、微透镜阵列200、图像采集单元400和设置于微透镜阵列200前后包括微透镜阵列200上的的第一掩膜300。
其中，主成像系统100用于得到目标物体的第一图像；微透镜阵列200用于利用预设的相位调制函数对第一图像的光路进行相位调制，引入预设的频谱混叠，得到目标物体的第二图像；第一掩膜300用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，引入预设的频谱混叠，得到目标物体的光场信息，使得图像采集单元400根据光场信息生成最终图像。优选地，为了提高图像质量以及可靠性，本申请实施例的图像采集单元400可以选用CMOS图像传感器，为满足更多图像的实际采集需求也可以采用其他具有图像采集功能的图像采集设备，在此不做具体限定。
进一步地，在一些实施例中，第一图像的波函数为：
Figure PCTCN2022103915-appb-000006
其中，x,y,z为物点的三维空间距离坐标，z含义为深度，λ为波长，r为孔径平面径向距离，J 0(·)为第一类零阶贝塞尔函数，s为主成像系统与微透镜阵列之间距离，d为物体与主成像系统距离，D(r,λ,z)为建模了距离主成像系统距离z处的成像响应变化情况，表达式如下式：
Figure PCTCN2022103915-appb-000007
进一步地，在一些实施例中，预设的相位调制函数为：
Figure PCTCN2022103915-appb-000008
其中，x 0，y 0为微透镜阵列的中心坐标，f为焦距，n为折射率，rect(.)为矩形窗函数，i为虚部，exp(.)为指数函数。
进一步地，在一些实施例中，预设的第一掩膜调制函数为：
Figure PCTCN2022103915-appb-000009
其中，m 2(x,y)为设置于微透镜阵列200前后包括微透镜阵列200上、图像采集单元400之前的掩膜调制函数。
具体地，相对于相关技术中通过采用相机阵列光场掩膜来改变每个相机的分辨率以控制光场空间分辨率的大小，但是存在相机物理空间以及系统复杂度、整体成本等因素的限制问题，本申请实施例采用通过基于微透镜阵列200增加掩膜的两种光场采集方式来提升采样效率和光场成像质量。
作为一种可实现的方式，如图3所示，通过在微透镜阵列200前后包括微透镜阵列200上和图像采集单元400之前增加掩膜即第一掩膜300，以实现频谱信息搬移，减少频域响应零点，提高光场成像质量与成像分辨率。
作为另一种可实现的方式，如图4所示，通过在微透镜阵列200前后包括微透镜阵列200上和图像采集单元400之前增加掩膜即第一掩膜300，以及在主成像系统100附近包括主成像透镜上、微透镜阵列200之前增加掩膜即第二掩膜，从而得到综合光源调制信息与角度域调制信息，优化了成像系统响应，提高了光场成像质量与成像分辨率。
具体而言，在本申请实施例中，首先，通过主成像系统得到目标物体的第一图像；其次，微透镜阵列200利用预设的相位调制函数对第一图像的光路进行相位调制，从而得到目标物体的第二图像。
其中，第一图像经过主成像系统的波函数可以表示为：
Figure PCTCN2022103915-appb-000010
其中，x,y,z为物点的三维空间距离坐标，z含义为深度，λ为波长，r为孔径平面径向距离，J 0(·)为第一类零阶贝塞尔函数。
在上式中，D(r,λ,z)可以表示为：
Figure PCTCN2022103915-appb-000011
其中，s为主成像系统与微透镜阵列之间距离，d为物体与主成像系统距离。
预设的相位调制函数可以表示为：
Figure PCTCN2022103915-appb-000012
其中，x 0，y 0为微透镜阵列的中心坐标，f为焦距，n为折射率，rect(.)为矩形窗函数，i为虚部，exp(.)为指数函数。
预设的第一掩膜调制函数为：
Figure PCTCN2022103915-appb-000013
其中，m 2(x,y)为设置于微透镜阵列前后包括微透镜阵列上、图像采集单元400之前的掩膜调制函数。
进一步地，在一些实施例中，上述的用于元成像的CIS系统10，还包括：设置于主成像系统100附近包括主成像透镜上、微透镜阵列200之前的第二掩膜，第二掩膜用于利用预设的第二掩膜调制函数对第一图像进行掩膜调制，得到用于相位调制的第一图像。
进一步地，在一些实施例中，光场信息为：
U′(ω x,ω y)＝F ω(U 0(x,y,z,λ)m 1(r)·t(x,y,x 0,y 0))；
其中，ω x,ω y为对空间(x,y)的频域采样，F ω(.)为傅里叶变换操作，m 1(r)为设置于主成像系统100附近包括主成像透镜上、微透镜阵列200之前的第二掩膜调制函数。
进一步地，在一些实施例中，将光场信息在图像采集单元400上生成一个最终图像，包括：基于成像响应，将光场信息在图像采集单元400上生成一个最终图像，其中，成像响应为：
Figure PCTCN2022103915-appb-000014
其中，ω u为角度u对应的空间频率位置，ω v为角度v对应的空间频率位置，s(ω u,ω v)为相机像素对特定频率分量的采集过程。
具体地，本申请实施例首先利用设置于微透镜阵列200前后包括微透镜阵列200上的第一掩膜300，将其通过预设的第一掩膜调制函数经相关计算对上述得到的目标物体的第二图像进行掩膜调制，并引入更均一的频谱混叠，以避免微透镜阵列200光场采集系统中信息混叠时，导致的光场重建质量下降的情况，进而得到目标物体的光场信息，该光场信息包含了光辐射在传播过程中的四维位置和方向信息，通过数字重聚焦技术解决了特殊场合图像的失焦等问题；其次，基于成像响应，通过获得的光场信息在图像采集单元400上生成一个最终的图像。
光场信息可以表示为：
U′(ω x,ω y)＝F ω(U 0(x,y,z,λ)m 1(r)·t(x,y,x 0,y 0))；(5)
其中，ω x,ω y为对空间(x,y)的频域采样，F ω(.)为傅里叶变换操作，m 1(r)为设置于主成像系统100附近包括主成像透镜上、微透镜阵列200之前的第二掩膜调制函数。
成像响应可以表示为：
Figure PCTCN2022103915-appb-000015
其中，ω u为角度u对应的空间频率位置，ω v为角度v对应的空间频率位置，s(ω u,ω v)为相机像素对特定频率分量的采集过程。
综上，通过上述的基于微透镜阵列增加掩膜的两种光场采集方式，不仅减少了频域零点，同时获得了最优成像系统，最终得以提升成像系统的灵活性和分辨率。需要说明的是，上述掩膜的设置位置仅是示例性的，本领域技术人员可以根据实际情况对掩膜位置进行设定，例如，第二掩膜设置在主成像系统的左侧，且紧贴主成像系统，再如第一掩膜设置在主成像系统和微透镜阵列之间，且紧靠微透镜阵列。
根据本申请实施例的用于元成像的CIS系统，通过主成像系统用于得到目标物体的第一图像，微透镜阵列用于利用预设的相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像，第一掩膜用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，引入预设的频谱混叠，得到目标物体的光场信息，使得图像采集单元根据光场信息生成最终图像。由此，基于两种微透镜阵列增加掩膜的光场采集方式，提升了采样效率和光场成像与重建质量，同时也解决了相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题。
其次参照附图描述根据本申请实施例提出的微透镜阵列成像方法。
图5是本申请实施例的用于元成像的CIS系统的成像方法的流程图。
如图5所示，该用于元成像的CIS系统的成像方法采用如上述的用于元成像的CIS系统，方法包括以下步骤：
在步骤S501中，通过主成像系统得到目标物体的第一图像；
在步骤S502中，利用微透镜阵列，通过相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像；
在步骤S503中，通过第一掩膜调制函数对第二图像进行第一掩膜调制，引入预设的频谱混叠，得到目标物体的光场信息，并将光场信息在图像采集单元上成一个最终图像。
进一步地，在一些实施例中，上述的用于元成像的CIS系统的成像方法，还包括：
通过第二掩膜利用预设的第二掩膜调制函数对第一图像进行掩膜调制，得到用于相位调制的第一图像。
根据本申请实施例的用于元成像的CIS系统的成像方法，通过主成像系统用于得到目标物体的第一图像，微透镜阵列用于利用预设的相位调制函数对第一图像的光路进行相位调制，得到目标物体的第二图像，第一掩膜用于通过预设的第一掩膜调制函数对第二图像进行掩膜调制，引入预设的频谱混叠，得到目标物体的光场信息，使得图像采集单元根据 光场信息生成最终图像。由此，基于两种微透镜阵列增加掩膜的光场采集方式，提升了采样效率和光场成像与重建质量，同时也解决了相机阵列在相机镜头和传感器之间增加掩膜实现光场采集时成本高、体积大等问题。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或N个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“N个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更N个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或N个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得程序，然后将其存储在计算机存 储器中。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，N个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。如，如果用硬件来实现和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (10)

  1. 一种用于元成像的CIS系统，其特征在于，包括：主成像系统、微透镜阵列、图像采集单元、设置于所述微透镜阵列前后包括微透镜阵列上的第一掩膜，其中，
    所述主成像系统用于得到目标物体的第一图像；
    所述微透镜阵列用于利用预设的相位调制函数对所述第一图像的光路进行相位调制，得到所述目标物体的第二图像；
    所述第一掩膜用于通过预设的第一掩膜调制函数对所述第二图像进行掩膜调制，引入预设的频谱混叠，得到所述目标物体的光场信息，使得所述图像采集单元根据所述光场信息生成最终图像。
  2. 根据权利要求1所述的系统，其特征在于，还包括：
    设置于所述主成像系统中主成像透镜上、所述微透镜阵列之前的第二掩膜，所述第二掩膜用于利用预设的第二掩膜调制函数对所述第一图像进行掩膜调制，得到用于相位调制的第一图像。
  3. 根据权利要求1所述的系统，其特征在于，所述第一图像的波函数为：
    Figure PCTCN2022103915-appb-100001
    其中，x,y,z为物点的三维空间距离坐标，z含义为深度，λ为波长，r为孔径平面径向距离，J 0(·)为第一类零阶贝塞尔函数，s为所述主成像系统与所述微透镜阵列之间距离，d为所述目标物体与所述主成像系统距离，D(r,λ,z)为建模了距离主成像系统距离z处的成像响应变化情况，表达式如下式：
    Figure PCTCN2022103915-appb-100002
  4. 根据权利要求3所述的系统，其特征在于，所述预设的相位调制函数为：
    Figure PCTCN2022103915-appb-100003
    其中，x 0，y 0为微透镜阵列的中心坐标，f为焦距，n为折射率，rect(.)为矩形窗函数，i为虚部，exp(.)为指数函数。
  5. 根据权利要求4所述的系统，其特征在于，所述预设的第一掩膜调制函数为：
    Figure PCTCN2022103915-appb-100004
    其中，m 2(x,y)为设置于微透镜阵列前后包括微透镜阵列上、图像采集单元之前的掩膜调制函数。
  6. 根据权利要求5所述的系统，其特征在于，所述光场信息为：
    U′(ω x,ω y)＝F ω(U 0(x,y,z,λ)m 1(r)·t(x,y,x 0,y 0))；
    其中，ω x,ω y为对空间(x,y)的频域采样，F ω(.)为傅里叶变换操作，m 1(r)为设置于主成像系统附近包括主成像透镜上、所述微透镜阵列之前的第二掩膜调制函数。
  7. 根据权利要求6所述的系统，其特征在于，所述将所述光场信息在所述图像采集单元上生成一个最终图像，包括：
    基于成像响应，将所述光场信息在所述图像采集单元上生成一个最终图像，其中，所述成像响应为：
    Figure PCTCN2022103915-appb-100005
    其中，ω u为角度u对应的空间频率位置，ω v为角度v对应的空间频率位置，s(ω u,ω v)为相机像素对特定频率分量的采集过程。
  8. 根据权利要求1-7任一项所述的系统，其特征在于，所述图像采集单元为CMOS图像传感器。
  9. 一种用于元成像的CIS系统的成像方法，其特征在于，采用如权利要求1-8任一项所述的用于元成像的CIS系统，所述方法包括以下步骤：
    通过所述主成像系统得到所述目标物体的第一图像；
    利用所述微透镜阵列，通过所述相位调制函数对所述第一图像的光路进行相位调制，得到所述目标物体的第二图像；
    通过所述第一掩膜调制函数对所述第二图像进行所述第一掩膜调制，引入预设的频谱混叠，得到所述目标物体的光场信息，并将所述光场信息在所述图像采集单元上成一个最终图像。
  10. 根据权利要求9所述的方法，其特征在于，还包括：
    通过所述第二掩膜利用预设的第二掩膜调制函数对所述第一图像进行掩膜调制，得到用于相位调制的第一图像。
PCT/CN2022/103915 2022-05-30 2022-07-05 用于元成像的cis系统 WO2023231140A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210602153.8 2022-05-30
CN202210602153.8A CN115190257A (zh) 2022-05-30 2022-05-30 用于元成像的cis系统

Publications (1)

Publication Number Publication Date
WO2023231140A1 true WO2023231140A1 (zh) 2023-12-07

Family

ID=83514311

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/103915 WO2023231140A1 (zh) 2022-05-30 2022-07-05 用于元成像的cis系统

Country Status (2)

Country Link
CN (1) CN115190257A (zh)
WO (1) WO2023231140A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090316014A1 (en) * 2008-06-18 2009-12-24 Samsung Electronics Co., Ltd. Apparatus and method for capturing digital images
CN103237161A (zh) * 2013-04-10 2013-08-07 中国科学院自动化研究所 基于数字编码控制的光场成像装置及方法
CN105931190A (zh) * 2016-06-14 2016-09-07 西北工业大学 高角度分辨率光场获取装置与图像生成方法
CN106067162A (zh) * 2016-06-03 2016-11-02 西安电子科技大学 集成成像超分辨率微单元图像阵列采集与重构方法
WO2016175568A1 (ko) * 2015-04-29 2016-11-03 주식회사 큐티아이 플레놉틱 카메라 원리를 이용한 디지털 촬상 장치
CN112945141A (zh) * 2021-01-29 2021-06-11 中北大学 基于微透镜阵列的结构光快速成像方法及系统
CN113504547A (zh) * 2021-09-07 2021-10-15 清华大学 基于扫描光场的视觉雷达成像系统和方法
CN114511605A (zh) * 2022-04-18 2022-05-17 清华大学 光场深度估计方法、装置、电子设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090316014A1 (en) * 2008-06-18 2009-12-24 Samsung Electronics Co., Ltd. Apparatus and method for capturing digital images
CN103237161A (zh) * 2013-04-10 2013-08-07 中国科学院自动化研究所 基于数字编码控制的光场成像装置及方法
WO2016175568A1 (ko) * 2015-04-29 2016-11-03 주식회사 큐티아이 플레놉틱 카메라 원리를 이용한 디지털 촬상 장치
CN106067162A (zh) * 2016-06-03 2016-11-02 西安电子科技大学 集成成像超分辨率微单元图像阵列采集与重构方法
CN105931190A (zh) * 2016-06-14 2016-09-07 西北工业大学 高角度分辨率光场获取装置与图像生成方法
CN112945141A (zh) * 2021-01-29 2021-06-11 中北大学 基于微透镜阵列的结构光快速成像方法及系统
CN113504547A (zh) * 2021-09-07 2021-10-15 清华大学 基于扫描光场的视觉雷达成像系统和方法
CN114511605A (zh) * 2022-04-18 2022-05-17 清华大学 光场深度估计方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN115190257A (zh) 2022-10-14

Similar Documents

Publication Publication Date Title
Pozo et al. An integrated 6DoF video camera and system design
WO2019105214A1 (zh) 图像虚化方法、装置、移动终端和存储介质
Wang et al. The light field attachment: Turning a DSLR into a light field camera using a low budget camera ring
Lei et al. Depth map super-resolution considering view synthesis quality
KR20200030603A (ko) 심 없는 이미지 스티칭(Seamless image stitching)
CN111344746A (zh) 一种利用可重构混合成像系统的动态场景的三维3d重建方法
US20180014003A1 (en) Measuring Accuracy of Image Based Depth Sensing Systems
WO2010028559A1 (zh) 图像拼接方法及装置
CN106256124B (zh) 结构化立体
WO2011014421A2 (en) Methods, systems, and computer-readable storage media for generating stereoscopic content via depth map creation
TW201518847A (zh) 拍攝立體影像之方法以及電子裝置以及儲存其之電腦可讀取記錄媒體
JP2018538709A (ja) ピクセルビームを表すデータを生成する方法及び装置
Liu et al. High quality depth map estimation of object surface from light-field images
JP2019511851A (ja) ピクセルビームを表すデータを生成する方法及び装置
Zhang et al. Synthetic aperture based on plenoptic camera for seeing through occlusions
Zhang et al. MobiDepth: real-time depth estimation using on-device dual cameras
Ma et al. Effective video stabilization via joint trajectory smoothing and frame warping
CN113807160B (zh) 图像采集设备劫持的检测方法、装置及计算机设备
Popovic et al. Image blending in a high frame rate FPGA-based multi-camera system
WO2023231140A1 (zh) 用于元成像的cis系统
CN108352061B (zh) 用于产生表示像素光束的数据的装置和方法
CN112749610A (zh) 深度图像、参考结构光图像生成方法、装置及电子设备
Cai et al. Towards 6DoF live video streaming system for immersive media
CN110198391A (zh) 摄像装置、摄像方法和图像处理装置
US20220321859A1 (en) Real-time omnidirectional stereo matching method using multi-view fisheye lenses and system thereof