WO2021190674A1 - 一种可实现彩色成像的多光谱成像芯片及彩色成像方法 - Google Patents

一种可实现彩色成像的多光谱成像芯片及彩色成像方法 Download PDF

Info

Publication number
WO2021190674A1
WO2021190674A1 PCT/CN2021/095490 CN2021095490W WO2021190674A1 WO 2021190674 A1 WO2021190674 A1 WO 2021190674A1 CN 2021095490 W CN2021095490 W CN 2021095490W WO 2021190674 A1 WO2021190674 A1 WO 2021190674A1
Authority
WO
WIPO (PCT)
Prior art keywords
spectrum
color
filter film
photoelectric conversion
spectral
Prior art date
Application number
PCT/CN2021/095490
Other languages
English (en)
French (fr)
Inventor
蔡红星
石晶
姚治海
宋晨智
任玉
韩颖
田野
蔡鹏程
Original Assignee
吉林求是光谱数据科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 吉林求是光谱数据科技有限公司 filed Critical 吉林求是光谱数据科技有限公司
Publication of WO2021190674A1 publication Critical patent/WO2021190674A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G01J2003/2826—Multispectral imaging, e.g. filter imaging

Definitions

  • the invention belongs to the field of multi-spectral chips, and relates to a multi-spectral imaging chip capable of realizing color imaging and a color imaging method.
  • liquid crystal adjustable color filter [J].Optical Technology, 2012,38(05):593-597];
  • the liquid crystal tunable color filter is large in size, it takes time to tune to obtain the image, and cannot be integrated into the chip; at the same time, the liquid crystal tunable color filter can only collect For a single-wavelength image, it takes a period of scanning to obtain the information of the full spectrum, and then to reconstruct the color, which takes a long time.
  • the first object of the present invention is to provide a multi-spectral imaging chip that can realize color imaging, so as to solve the problem that the existing multi-spectral imaging chip cannot realize the function of color imaging.
  • a multi-spectral imaging chip that can realize color imaging.
  • the chip includes a multi-spectral chip body and a color restoration module; wherein, the multi-spectral chip body includes a spectrum modulation module and a spectrum inversion module;
  • a filter film made of N materials with different light transmittances is known to achieve spectral spectroscopy; the photoelectric conversion substrate under the filter film converts the optical signal split by the filter film into an electrical signal, which is amplified and modulated Converted to digital signal or coded output after conversion;
  • the spectrum inversion module is electrically connected to the spectrum modulation module, and is used to invert the incident spectrum information according to the optical signal intensity information output by the spectrum modulation module and corresponding pixel position information;
  • the color restoration module establishes a spectrum-to-color conversion matrix according to the visual response curve of the human eye, and converts the spectrum information output by the multi-spectrum chip body into an RGB image.
  • the conversion method is to multiply the spectrum information by the spectrum to the color change matrix.
  • the method for determining the spectrum-to-color change matrix is:
  • Step S1 Obtain the spectral data of color patches or samples with known three primary color values: use the multi-spectral chip body to measure the spectral information of multiple color patches or single color samples respectively, which is represented by I j (N), that is, I j (N ) Is the measured value of the spectrum, which is a known number, N represents N kinds of filter film materials with different spectral transmittances, and j represents the jth color patch;
  • Step S2 Calculate the spectrum to the color change matrix: Since the three primary color values of each color block are known, that is, I j (R, G, B) is known; assuming that there is a transformation matrix [M], the following formula is established, Then the above process is described as:
  • the matrix [M] is the spectrum to color change matrix.
  • the spectral modulation module includes a photoelectric conversion substrate, and a filter film disposed on the photoelectric conversion substrate; wherein, the photoelectric conversion substrate is used to convert optical signals into electrical signals and use digital signals or codes. Output;
  • the filter film is used to distinguish the incident light spectrum.
  • the filter film is a single-layer structure, which is made of N materials with known and different light transmittances through coating and etching one by one. ,
  • the filter film includes N periods, each period includes T 1 , T 2 ...
  • each unit covers M pixels on the photoelectric conversion substrate, where M is greater than or equal to 1, all units A periodic structure is formed to cover all pixels on the photoelectric conversion substrate, and the filter film corresponding to each pixel has the same or different spectral transmittance to realize spectral light splitting.
  • the color reproduction module is integrated in the body of the multispectral chip, or independently installed on a computing platform such as a computer, cloud, mobile phone, etc.
  • a microlens array is provided on the filter film, and each microlens on the microlens array corresponds to a photoelectric conversion substrate pixel of the photoelectric conversion substrate in a one-to-one correspondence, and is used to transform the incident light beam Converge.
  • the photoelectric conversion substrate is a silicon-based image sensor, specifically a CMOS image sensor or a CCD image sensor.
  • the photoelectric conversion substrate converts the optical signal filtered by the filter film into an electrical signal, and is amplified And converted to digital signal or coded output after analog-to-digital conversion.
  • the preparation method of the spectrum modulation module specifically includes the following steps:
  • Step S1 select a suitable photoelectric conversion substrate according to the usage scenario
  • Step S2 Select N kinds of filter film materials with different spectral transmittances, and first coat the first filter film material on the photoelectric conversion substrate, and then coat an etching layer, according to the correspondence with the pixels of the photoelectric conversion substrate Retain the necessary places and etch away the unneeded places; then apply the second filter film material, and then coat an etching layer, according to the corresponding relationship with the photoelectric conversion substrate pixels, the required Keep the place, and etch away the unneeded places; cycle in turn until all N kinds of filter film materials are coated on the photoelectric conversion substrate.
  • each period includes T 1 , T 2 ??T n units, and each unit covers M pixels on the photoelectric conversion substrate, and M is greater than Equal to 1, the filter film corresponding to each pixel has the same or different spectral transmittance;
  • Step S3. A microlens array is processed on the filter film.
  • Each microlens on the microlens array corresponds to the photoelectric conversion substrate pixel of the photoelectric conversion substrate one-to-one.
  • the second object of the present invention is to provide a color imaging method of a multispectral imaging chip that can realize color imaging, which specifically includes the following steps:
  • Step S1 The optical signal passing through the filter film enters the underlying photoelectric conversion substrate.
  • the photoelectric conversion substrate converts the incident optical signal into an electrical signal, and after amplification and analog-to-digital conversion, it is converted into a digital signal or coded output.
  • the output is each The light intensity information at the pixel position, that is, there is a one-to-one correspondence between the output light intensity information and the pixel position information;
  • Step S2 Spectral inversion: For the signal output in step S1, since the spectral transmittance of the pixel is known, in a periodic structure composed of N pixels, according to the spectral transmittance curve, N pixels are combined, The inversion calculates the incident spectral values of N pixels, the calculation method is as shown in formula (3),
  • S is the intensity value of the optical signal output by the photoelectric conversion substrate
  • I is the incident spectrum, which is the signal to be solved
  • T is the spectral transmittance of the filter film
  • ⁇ is the quantum efficiency of the photoelectric conversion substrate
  • ⁇ is the incident wavelength
  • Step S3 Multiply the spectrum information output by the multispectral chip body by the spectrum to color change matrix to obtain an RGB image; the spectrum to color change matrix is pre-stored in the color restoration module.
  • the multi-spectral imaging chip that can realize color imaging provided by the present invention can convert the spectral information output by the multi-spectral imaging chip into RGB images, making full use of the advantages of the large amount of spectral image information, and realizing high-fidelity color imaging technology and imaging effects Better than general color imaging technology.
  • the multi-spectral imaging chip that can realize color imaging provided by the present invention has the advantages of good imaging color reproduction and high signal-to-noise ratio under dark light conditions.
  • the dimensionality reduction transformation is performed from the 8 dimensions of the multi-spectrum to the 3 dimensions of the color.
  • This strongly constrained dimensionality reduction mathematical operation has the advantage of improving the signal-to-noise ratio of the original image.
  • the multispectral imaging chip used in the present invention has simple structure, small size, thin thickness, light weight, wide spectral range, high energy utilization, high spectral resolution, high spatial resolution, high accuracy, and fast detection speed. It is easy to use and other features. It can recognize objects through imaging and perform color inversion for different areas of objects. Especially for small objects, the advantage is obvious, which can overcome the small proportion of small objects in the field of view and difficult to shoot. The problem.
  • Fig. 1 is a schematic diagram of a multi-spectral imaging chip capable of realizing color imaging according to the present invention.
  • FIG. 2 is a schematic diagram of the spectrum modulation module of the present invention.
  • Figure 3 is a unit structure diagram of the spectrum modulation module of the present invention.
  • Figure 4 is a structural diagram of the spectrum modulation module of the present invention.
  • Embodiment 1 Multispectral imaging chip capable of realizing color imaging
  • the present invention provides a multispectral imaging chip that can realize color imaging.
  • the chip includes a multispectral chip body A and a color restoration module B; wherein the multispectral chip body A includes a spectrum modulation module 1.
  • the base image sensor specifically a CMOS image sensor or a CCD image sensor, is used to convert light signals into electrical signals and output them as digital signals or codes.
  • the output is the spectral intensity information at each pixel position (spectral intensity information and pixel position Information one-to-one correspondence);
  • the filter film is used to distinguish the incident light spectrum.
  • the filter film is a single-layer structure, which is made of N materials with known and different light transmittances through coating and etching one by one After splicing, the filter film includes N periods, each period includes T 1 , T 2 ??T n units, and each unit covers M pixels on the photoelectric conversion substrate, where M is greater than or equal to 1.
  • All units constitute a periodic structure, covering all pixels on the photoelectric conversion substrate, and the filter film corresponding to each pixel has the same or different spectral transmittance to achieve spectral splitting; each of the microlens arrays
  • the microlens corresponds to the pixels of the photoelectric conversion substrate of the photoelectric conversion substrate one-to-one, and is used to converge the incident light beam to improve the utilization rate of the incident light energy and the aperture ratio of the chip;
  • the spectrum inversion module is electrically connected to the spectrum modulation module, and is used to invert the incident spectrum information according to the optical signal intensity information output by the spectrum modulation module and corresponding pixel position information;
  • the color restoration module establishes a spectrum-to-color conversion matrix according to the visual response curve of the human eye, and converts the spectrum information output by the multispectral chip body into an RGB image, and the conversion method is to multiply the spectrum information by the spectrum to the color change matrix;
  • the method of determining the spectrum to the color change matrix is:
  • Step S1 Obtain the spectral data of color patches or samples with known three primary color values: use the multi-spectral chip body to measure the spectral information of multiple color patches or single color samples respectively, which is represented by I j (N), that is, I j (N ) Is the measured value of the spectrum, which is a known number, N represents N kinds of filter film materials with different spectral transmittances, and j represents the jth color patch;
  • Step S2 Calculate the spectrum to the color change matrix: Since the three primary color values of each color block are known, that is, I j (R, G, B) is known; assuming that there is a transformation matrix [M], the following formula is established, Then the above process is described as:
  • Matrix [M] is the spectrum to color change matrix
  • step S1 of the present invention When measuring samples or color plates in step S1 of the present invention, it is better to select 24 standard color plates or 48 standard color plates, and save them separately after measurement.
  • the color restoration module of the present invention can be integrated in the body of the multispectral chip, or can be independently installed on computing platforms such as computers, clouds, and mobile phones.
  • Embodiment 2 Preparation method of spectrum modulation module in multi-spectral chip body
  • the preparation method of the spectrum modulation module in the multi-spectral chip body of the present invention specifically includes the following steps:
  • Step S1 select a suitable photoelectric conversion substrate according to the usage scenario; when selecting the substrate, parameters such as spatial resolution, pixel size, signal-to-noise ratio, dynamic range, etc. need to be considered;
  • Step S2 Select N kinds of filter film materials with different spectral transmittances, and first apply a standard spraying or spin coating method on the photoelectric conversion substrate to coat the first filter film material, and then coat an etching layer According to the corresponding relationship with the pixels of the photoelectric conversion substrate, the necessary places are reserved and the unneeded places are etched away; then the second filter film material is coated, and then an etching layer is coated, according to the photoelectric conversion
  • the corresponding relationship of the pixels on the substrate is to keep the places that are needed and etch away the places that are not needed.
  • each period includes T 1 , T 2 ??T n units, and each unit covers the photoelectric conversion M pixels on the substrate, M is greater than or equal to 1, and the filter film corresponding to each pixel has the same or different spectral transmittance;
  • Step S3. A microlens array is processed on the filter film.
  • Each microlens on the microlens array corresponds to the photoelectric conversion substrate pixel of the photoelectric conversion substrate one-to-one.
  • etching in step S2 a laser direct writing etching method, a mask photolithography etching method, an ion beam etching method, an electron beam etching method, etc. are used; when a mask photolithography etching method is used, A layer of photoresist is coated on each filter film material; then the etching is completed by standard photolithography processes such as exposure, development, drying, etching, and post-drying; when the laser direct writing etching method is used, In the ion beam etching method and the electron beam etching method, the preparation process is similar to the mask photolithography etching method, and the existing method is used for etching.
  • the filter film material used in the present invention is a polyimide material; the transparent photoresist is made of epoxy resin.
  • Embodiment 3 Color imaging method of multispectral imaging chip capable of realizing color imaging
  • the color imaging method using the multi-spectral imaging chip capable of color imaging of the present invention specifically includes the following steps:
  • Step S1 The optical signal passing through the filter film enters the underlying photoelectric conversion substrate.
  • the photoelectric conversion substrate converts the incident optical signal into an electrical signal, and after amplification and analog-to-digital conversion, it is converted into a digital signal or coded output.
  • the output is each The light intensity information at the pixel position, that is, there is a one-to-one correspondence between the output light intensity information and the pixel position information;
  • Step S2 Spectral inversion: For the signal output in step S1, since the spectral transmittance of the pixel is known, in a periodic structure composed of N pixels, according to the spectral transmittance curve, N pixels are combined, The inversion calculates the incident spectral values of N pixels, the calculation method is as shown in formula (3),
  • S is the intensity value of the optical signal output by the photoelectric conversion substrate
  • I is the incident spectrum, which is the signal to be solved
  • T is the spectral transmittance of the filter film
  • ⁇ is the quantum efficiency of the photoelectric conversion substrate
  • ⁇ is the incident wavelength
  • Step S3 Multiply the spectrum information output by the multispectral chip body by the spectrum to a color change matrix to obtain an RGB image; the spectrum to color change matrix is pre-stored in the color restoration module. See Embodiment 1 for the specific determination method.

Landscapes

  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Color Television Image Signal Generators (AREA)
  • Solid State Image Pick-Up Elements (AREA)

Abstract

一种可实现彩色成像的多光谱成像芯片及彩色成像方法，芯片包括多光谱芯片本体（A）、色彩还原模块（B）；多光谱芯片本体（A）包括光谱调制模块（1）、光谱反演模块（2）；光谱调制模块（1），通过由已知且透光率不同的N种材料形成的滤光薄膜来实现光谱分光；通过滤光薄膜下的光电转换基底将分光后的光信号转换成为电信号，经过放大及模数转换后转为数字信号或者编码输出；光谱反演模块（2），根据光谱调制模块输出的光信号强度信息和对应像素位置信息，反演出入射光谱信息；色彩还原模块（B），将多光谱芯片本体（A）输出的光谱信息转换为RGB图像，转换方法为光谱信息乘以光谱到颜色变化矩阵。芯片将光谱信息转换为RGB图像，充分利用光谱图像信息量大的优点，实现高保真的彩色成像功能。

Description

一种可实现彩色成像的多光谱成像芯片及彩色成像方法 技术领域
本发明属于多光谱芯片领域，涉及可实现彩色成像的多光谱成像芯片及彩色成像方法。
背景技术
虽然成像技术和光谱技术均有着广阔的应用前景，然而从目前可以公开检索到论文和专利等资料来分析，尚未检索到可实现彩色成像的多光谱成像芯片及其色彩还原技术。与本专利有部分相关的技术有基于液晶可调变滤色器的光谱图像获取及颜色重建方法。在颜色重建方面的资料有：[田立勋,廖宁放,王佳佳,柴阿丽,谭博能,廉玉生.基于液晶可调变滤色器的光谱图像获取及颜色重建方法[J].光学技术,2012,38(05):593-597]；然而，该液晶可调谐滤色器体积大，需要时间调谐才能获得图像，无法集成到芯片上；同时液晶可调谐滤色器每个时刻只能采集单一波长的图像，需要经过一段时间的扫描才能获得全谱段的信息，进而进行颜色的重建，耗时长。
在光谱芯片方面，可以检索到的相关资料有：一是以欧洲IMEC为代表的采用法布里-珀罗干涉滤光的方法，其专利包括[国外专利:EP2746740,2014-06-25.Gonzalez,Pilar；Jayapala,Murali；Lambrechts,Andy；Tack,Nicolaas.Spectral imaging device and method to calibrate the same[P]]；二是以中国科学院西安光机所等单位为代表的基于表面超材料的技术方案，其专利和论文包括[CN 106847849 B,一种基于超表面窄带滤光的多光谱芯片及其制备方法[P]]，[Zhu Wang，Soongyu Yi etc，Single-shot on-chip spectral sensors based on photonic crystal slab，nature，doi：10.1038/s41467-019-08994-5[J]]；三是以清华大学为代表基于量子点和浙江大学基于碳纳米线等新型材料的技术方案，其论文包括[Jie Bao&Moungi G.Bawendi，nature--A colloidal quantum dot spectrometer doi:10.1038/nature14576[J]]，[Yang et al.,Single-nanowire spectrometers，Science 365,1017–1020(2019)，DOI:10.1126/science.aax8814[J]]。从上述公开的材料来看，上述光谱芯片均未报导可以实现彩色图像的功能，主要原因是色散单元结构尺寸较大，难以做到与每个像素对应，降低了空间分辨率，难以实现较好的彩色成像 效果。
发明内容
鉴于上述问题，本发明的第一个目的在于提供一种可实现彩色成像的多光谱成像芯片，以解决现有多光谱成像芯片无法实现彩色成像的功能。
为实现上述目的，本发明具体是采用如下技术方案实现的：
一种可实现彩色成像的多光谱成像芯片，该芯片包括多光谱芯片本体、色彩还原模块；其中，所述多光谱芯片本体包括光谱调制模块、光谱反演模块；所述光谱调制模块，通过由已知且透光率不同的N种材料形成的滤光薄膜来实现光谱分光；通过滤光薄膜下的光电转换基底将经过滤光薄膜分光后的光信号转换成为电信号，经过放大及模数转换后转为数字信号或者编码输出；
所述光谱反演模块与光谱调制模块电连接，用于根据光谱调制模块输出的光信号强度信息和对应像素位置信息，反演出入射光谱信息；
所述色彩还原模块，根据人眼的视觉响应曲线建立由光谱到颜色变换矩阵，将多光谱芯片本体输出的光谱信息转换为RGB图像，转换方法为光谱信息乘以光谱到颜色变化矩阵。
作为本发明的优选，所述光谱到颜色变化矩阵的确定方法为：
步骤S1、获取已知三基色值色块或样品的光谱数据：利用多光谱芯片本体分别测量多种色块或单一颜色样品的光谱信息，其用I j(N)表示，即I j(N)为光谱测量值，是已知数，N表示N种光谱透过率不同的滤光薄膜材料，j表示第j个色块；
步骤S2、计算光谱到颜色变化矩阵：由于每个色块的三基色值是已知的，即I j(R、G、B)已知；假设存在变换矩阵[M]，使得下式成立，则上述过程描述为：
I j(R、G、B)＝[M]·I j(N)        (1)
其中，I j(R、G、B)和I j(N)均为已知，所以矩阵[M]由下式给出；
Figure PCTCN2021095490-appb-000001
矩阵[M]即为光谱到颜色变化矩阵。
作为本发明的优选，所述光谱调制模块包括光电转换基底、设置在光电转换基底上面的滤光薄膜；其中，所述光电转换基底，用于将光信号转化为电信号并 以数字信号或者编码输出；所述滤光薄膜，用于将入射光光谱进行区分，滤光薄膜为单层结构，其是由已知且透光率不同的N种材料通过逐一涂覆、刻蚀后拼接而成，滤光薄膜包括N个周期，每个周期包括T 1、T 2......T n个单元，每个单元覆盖光电转换基底上的M个像素，其中M大于等于1，所有单元构成周期性结构，覆盖光电转换基底上的所有像素，与每个像素对应的滤光薄膜具有相同或者不同的光谱透过率，实现光谱分光。
作为本发明的优选，所述色彩还原模块集成于多光谱芯片本体内部，或独立安装在计算机、云端、手机等计算平台上。
作为本发明的进一步优选，在所述滤光薄膜上设置有微透镜阵列，所述微透镜阵列上的每个微透镜与光电转换基底的光电转换基底像素一一对应，用于将入射光光束进行汇聚。
作为本发明的进一步优选，所述光电转换基底为硅基图像传感器，具体为CMOS图像传感器或CCD图像传感器，该光电转换基底将经过滤光薄膜滤光的光信号转换为电信号，并经过放大及模数转换后转为数字信号或者编码输出。
作为本发明的更进一步优选，所述光谱调制模块的制备方法，具体包括以下步骤：
步骤S1、根据使用场景情况，选择合适的光电转换基底；
步骤S2、选择N种光谱透过率不同的滤光薄膜材料，先在光电转换基底上涂覆第一种滤光薄膜材料，再涂覆一层刻蚀层，根据与光电转换基底像素的对应关系，将需要的地方保留，将不需要的地方刻蚀掉；之后涂覆第二种滤光薄膜材料，再涂覆一层刻蚀层，根据与光电转换基底像素的对应关系，将需要的地方保留，将不需要的地方刻蚀掉；依次循环，直至将N种滤光薄膜材料全部涂覆到光电转换基底上，上述N种滤光薄膜材料经过逐一的涂覆和刻蚀后，最后形成一层完整的具有N个周期性的滤光薄膜，每个周期包括T 1、T 2......T n个单元，每个单元覆盖光电转换基底上的M个像素，M大于等于1，与每个像素对应的滤光薄膜具有相同或者不同的光谱透过率；
步骤S3、在所述滤光薄膜上加工有微透镜阵列，制备微透镜阵列时，首先在滤光薄膜上面涂覆一层用于制备微透镜阵列的透明光刻胶，然后采用激光直写或者掩膜光刻的方法刻蚀不需要的部分，保留下来的部分便构成了微透镜阵列， 微透镜阵列上的每个微透镜与光电转换基底的光电转换基底像素一一对应。
本发明的第二个目的在于提供一种可实现彩色成像的多光谱成像芯片的彩色成像方法，具体包括以下步骤：
步骤S1、经过滤光薄膜的光信号进入底层的光电转换基底，光电转换基底将入射光信号转换为电信号，并经过放大及模数转换后转为数字信号或者编码输出，输出的为每个像素位置上的光强度信息，即输出的光强度信息与像素位置信息有一一对应关系；
步骤S2：光谱反演：对于步骤S1输出的信号，由于该像素上的光谱透过率已知，在由N个像素组成的周期性结构中，根据光谱透过率曲线，N个像素组合，反演计算出N个像素的入射光谱值，计算方法如公式(3)所示，
S i＝∫I(λ)T i(λ)η(λ)dλ，    (3)
其中，S为光电转换基底输出的光信号强度值，I为入射光谱，是待求解信号，T为滤光薄膜的光谱透过率，η为光电转换基底的量子效率，λ为入射波长；
步骤S3、将多光谱芯片本体输出的光谱信息乘以光谱到颜色变化矩阵，从而获得RGB图像；所述光谱到颜色变化矩阵预存在所述色彩还原模块内。
本发明的优点及积极效果是：
1、本发明提供的可实现彩色成像的多光谱成像芯片可以将多光谱成像芯片输出的光谱信息转换为RGB图像，充分利用光谱图像信息量大的优点，实现高保真的彩色成像技术，成像效果优于一般彩色成像技术。
2、本发明提供的可实现彩色成像的多光谱成像芯片，具有在暗光条件下成像色彩还原度好和信噪比高的优点。在暗光下拍摄时，一般的彩色成像技术均会出现噪点，色差等现象，本发明由于采用多光谱技术，从多光谱的8个维度向色彩的3个维度进行降维变换，本身是一种强约束的降维数学运算，具有提高原始图像信噪比的优势。
3、本发明采用的多光谱成像芯片具有结构简单、体积小、厚度薄、重量轻、光谱范围广、能量利用率高、光谱分辨率高，空间分辨率高、精准度高、检测速度快、使用便捷等特点，即可以通过成像识别物体，又可以对物体的不同区域分别进行颜色反演，尤其是对于微小物体时，优势很明显，可以克服微小物体在视 场中占比小，难以拍摄的问题。
附图说明
通过参考以下结合附图的说明，并且随着对本发明的更全面理解，本发明的其它目的及结果将更加明白及易于理解。在附图中：
图1为本发明可实现彩色成像的多光谱成像芯片的原理图。
图2为本发明光谱调制模块的原理图；
图3为本发明光谱调制模块的单元结构图；
图4为本发明光谱调制模块的结构图。
具体实施方式
在下面的描述中，出于说明的目的，为了提供对一个或多个实施例的全面理解，阐述了许多具体细节。然而，很明显，也可以在没有这些具体细节的情况下实现这些实施例。在其它例子中，为了便于描述一个或多个实施例，公知的结构和设备以方框图的形式示出。
实施例1可实现彩色成像的多光谱成像芯片
参阅图1至图4，本发明提供的一种可实现彩色成像的多光谱成像芯片，该芯片包括多光谱芯片本体A、色彩还原模块B；其中，所述多光谱芯片本体A包括光谱调制模块1、光谱反演模块2；所述光谱调制模块1包括光电转换基底、设置在光电转换基底上面的滤光薄膜、设置在滤光薄膜上的微透镜阵列；其中，所述光电转换基底为硅基图像传感器，具体为CMOS图像传感器或CCD图像传感器，用于将光信号转化为电信号并以数字信号或者编码输出，输出的为每个像素位置上的光谱强度信息(光谱强度信息与像素位置信息一一对应)；所述滤光薄膜，用于将入射光光谱进行区分，滤光薄膜为单层结构，其是由已知且透光率不同的N种材料通过逐一涂覆、刻蚀后拼接而成，滤光薄膜包括N个周期，每个周期包括T 1、T 2......T n个单元，每个单元覆盖光电转换基底上的M个像素，其中M大于等于1，所有单元构成周期性结构，覆盖光电转换基底上的所有像素，与每个像素对应的滤光薄膜具有相同或者不同的光谱透过率，实现光谱分光；所述微透镜阵列上的每个微透镜与光电转换基底的光电转换基底像素一一对应，用于将入射光光束进行汇聚，提高入射光能量的利用率以及芯片的开口率；
所述光谱反演模块与光谱调制模块电连接，用于根据光谱调制模块输出的光 信号强度信息和对应像素位置信息，反演出入射光谱信息；
所述色彩还原模块，根据人眼的视觉响应曲线建立由光谱到颜色变换矩阵，将多光谱芯片本体输出的光谱信息转换为RGB图像，转换方法为光谱信息乘以光谱到颜色变化矩阵；所述光谱到颜色变化矩阵的确定方法为：
步骤S1、获取已知三基色值色块或样品的光谱数据：利用多光谱芯片本体分别测量多种色块或单一颜色样品的光谱信息，其用I j(N)表示，即I j(N)为光谱测量值，是已知数，N表示N种光谱透过率不同的滤光薄膜材料，j表示第j个色块；
步骤S2、计算光谱到颜色变化矩阵：由于每个色块的三基色值是已知的，即I j(R、G、B)已知；假设存在变换矩阵[M]，使得下式成立，则上述过程描述为：
I j(R、G、B)＝[M]·I j(N)              (1)
其中，I j(R、G、B)和I j(N)均为已知，所以矩阵[M]由下式给出；
Figure PCTCN2021095490-appb-000002
矩阵[M]即为光谱到颜色变化矩阵；
本发明步骤S1测量样品或色板时，最好选择24标准色板或48标准色板，测量后分别进行保存。
本发明所述的色彩还原模块可以集成于多光谱芯片本体内部，也可以独立安装在计算机、云端、手机等计算平台上。
实施例2多光谱芯片本体中光谱调制模块的制备方法
本发明多光谱芯片本体中光谱调制模块的制备方法，具体包括以下步骤：
步骤S1、根据使用场景情况，选择合适的光电转换基底；基底选取时需考虑空间分辨率，像素大小，信噪比，动态范围等参数；
步骤S2、选择N种光谱透过率不同的滤光薄膜材料，先在光电转换基底上采用标准的喷涂或者旋涂方法，涂覆第一种滤光薄膜材料，再涂覆一层刻蚀层，根据与光电转换基底像素的对应关系，将需要的地方保留，将不需要的地方刻蚀掉；之后涂覆第二种滤光薄膜材料，再涂覆一层刻蚀层，根据与光电转换基底像素的对应关系，将需要的地方保留，将不需要的地方刻蚀掉；依次循环，直至将N种滤光薄膜材料全部涂覆到光电转换基底上，上述N种滤光薄膜材料经过逐 一的涂覆和刻蚀后，最后形成一层完整的具有N个周期性的滤光薄膜，每个周期包括T 1、T 2......T n个单元，每个单元覆盖光电转换基底上的M个像素，M大于等于1，与每个像素对应的滤光薄膜具有相同或者不同的光谱透过率；
步骤S3、在所述滤光薄膜上加工有微透镜阵列，制备微透镜阵列时，首先在滤光薄膜上面涂覆一层用于制备微透镜阵列的透明光刻胶，然后采用激光直写或者掩膜光刻的方法刻蚀不需要的部分，保留下来的部分便构成了微透镜阵列，微透镜阵列上的每个微透镜与光电转换基底的光电转换基底像素一一对应。
进一步，所述步骤S2进行刻蚀时，采用激光直写刻蚀方法、掩膜光刻刻蚀方法、离子束刻蚀方法、电子束刻蚀方法等；当采用掩膜光刻刻蚀时，在每种滤光薄膜材料上均涂覆一层光刻胶；之后经过曝光、显影、烘干、刻蚀、后烘干等标准光刻工艺完成刻蚀；当采用激光直写刻蚀方法、离子束刻蚀方法、电子束刻蚀方法时，制备过程与掩膜光刻刻蚀方法类似，均是采用现有方法进行刻蚀。
另外，本发明所用的滤光薄膜材料为聚酰亚胺类材料；所述透明光刻胶采用环氧树脂材料。
实施例3可实现彩色成像的多光谱成像芯片的彩色成像方法
利用本发明的可实现彩色成像的多光谱成像芯片的彩色成像方法，具体包括以下步骤：
步骤S1、经过滤光薄膜的光信号进入底层的光电转换基底，光电转换基底将入射光信号转换为电信号，并经过放大及模数转换后转为数字信号或者编码输出，输出的为每个像素位置上的光强度信息，即输出的光强度信息与像素位置信息有一一对应关系；
步骤S2：光谱反演：对于步骤S1输出的信号，由于该像素上的光谱透过率已知，在由N个像素组成的周期性结构中，根据光谱透过率曲线，N个像素组合，反演计算出N个像素的入射光谱值，计算方法如公式(3)所示，
S i＝∫I(λ)T i(λ)η(λ)dλ，     (3)
其中，S为光电转换基底输出的光信号强度值，I为入射光谱，是待求解信号，T为滤光薄膜的光谱透过率，η为光电转换基底的量子效率，λ为入射波长；
步骤S3、将多光谱芯片本体输出的光谱信息乘以光谱到颜色变化矩阵，从 而获得RGB图像；所述光谱到颜色变化矩阵预存在所述色彩还原模块内，具体确定方法见实施例1。
以上，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (8)

  1. 一种可实现彩色成像的多光谱成像芯片，其特征在于，该芯片包括多光谱芯片本体、色彩还原模块；其中，所述多光谱芯片本体包括光谱调制模块、光谱反演模块；所述光谱调制模块，通过由已知且透光率不同的N种材料形成的滤光薄膜来实现光谱分光；通过滤光薄膜下的光电转换基底将经过滤光薄膜分光后的光信号转换成为电信号，经过放大及模数转换后转为数字信号或者编码输出；
    所述光谱反演模块与光谱调制模块电连接，用于根据光谱调制模块输出的光信号强度信息和对应像素位置信息，反演出入射光谱信息；
    所述色彩还原模块，根据人眼的视觉响应曲线建立由光谱到颜色变换矩阵，将多光谱芯片本体输出的光谱信息转换为RGB图像，转换方法为光谱信息乘以光谱到颜色变化矩阵。
  2. 根据权利要求1所述的可实现彩色成像的多光谱成像芯片，其特征在于，所述光谱到颜色变化矩阵的确定方法为：
    步骤S1、获取已知三基色值色块或样品的光谱数据：利用多光谱芯片本体分别测量多种色块或单一颜色样品的光谱信息，其用I j(N)表示，即I j(N)为光谱测量值，是已知数，N表示N种光谱透过率不同的滤光薄膜材料，j表示第j个色块；
    步骤S2、计算光谱到颜色变化矩阵：由于每个色块的三基色值是已知的，即I j(R、G、B)已知；假设存在变换矩阵[M]，使得下式成立，则上述过程描述为：
    I j(R、G、B)＝[M]·I j(N)  (1)
    其中，I j(R、G、B)和I j(N)均为已知，所以[M]由下式给出；
    Figure PCTCN2021095490-appb-100001
    矩阵[M]即为光谱到颜色变化矩阵。
  3. 根据权利要求1所述的可实现彩色成像的多光谱成像芯片，其特征在于，所述光谱调制模块包括光电转换基底、设置在光电转换基底上面的滤光薄膜；其中，所述光电转换基底，用于将光信号转化为电信号并以数字信号或者编码输出；所述滤光薄膜，用于将入射光光谱进行区分，滤光薄膜为单层结构，其是由已知且透光率不同的N种材料通过逐一涂覆、刻蚀后拼接而成，滤光薄膜包括N个 周期，每个周期包括T 1、T 2......T n个单元，每个单元覆盖光电转换基底上的M个像素，其中M大于等于1，所有单元构成周期性结构，覆盖光电转换基底上的所有像素，与每个像素对应的滤光薄膜具有相同或者不同的光谱透过率，实现光谱分光。
  4. 根据权利要求1所述的可实现彩色成像的多光谱成像芯片，其特征在于，所述色彩还原模块集成于多光谱芯片本体内部，或独立安装在计算机、云端、手机上。
  5. 根据权利要求3所述的可实现彩色成像的多光谱成像芯片，其特征在于，在所述滤光薄膜上设置有微透镜阵列，所述微透镜阵列上的每个微透镜与光电转换基底的光电转换基底像素一一对应，用于将入射光光束进行汇聚。
  6. 根据权利要求3所述的可实现彩色成像的多光谱成像芯片，其特征在于，所述光电转换基底为硅基图像传感器，具体为CMOS图像传感器或CCD图像传感器。
  7. 根据权利要求5所述的可实现彩色成像的多光谱成像芯片，其特征在于，所述光谱调制模块的制备方法，具体包括以下步骤：
    步骤S1、根据使用场景情况，选择合适的光电转换基底；
    步骤S2、选择N种光谱透过率不同的滤光薄膜材料，先在光电转换基底上涂覆第一种滤光薄膜材料，再涂覆一层刻蚀层，根据与光电转换基底像素的对应关系，将需要的地方保留，将不需要的地方刻蚀掉；之后涂覆第二种滤光薄膜材料，再涂覆一层刻蚀层，根据与光电转换基底像素的对应关系，将需要的地方保留，将不需要的地方刻蚀掉；依次循环，直至将N种滤光薄膜材料全部涂覆到光电转换基底上，上述N种滤光薄膜材料经过逐一的涂覆和刻蚀后，最后形成一层完整的具有N个周期性的滤光薄膜，每个周期包括T 1、T 2......T n个单元，每个单元覆盖光电转换基底上的M个像素，M大于等于1，与每个像素对应的滤光薄膜具有相同或者不同的光谱透过率；
    步骤S3、在所述滤光薄膜上加工有微透镜阵列，制备微透镜阵列时，首先在滤光薄膜上面涂覆一层用于制备微透镜阵列的透明光刻胶，然后采用激光直写或者掩膜光刻的方法刻蚀不需要的部分，保留下来的部分便构成了微透镜阵列，微透镜阵列上的每个微透镜与光电转换基底的光电转换基底像素一一对应。
  8. 权利要求1所述可实现彩色成像的多光谱成像芯片的彩色成像方法，其特征在于，具体包括以下步骤：
    步骤S1、经过滤光薄膜的光信号进入底层的光电转换基底，光电转换基底将入射光信号转换为电信号，并经过放大及模数转换后转为数字信号或者编码输出，输出的为每个像素位置上的光强度信息，即输出的光强度信息与像素位置信息有一一对应关系；
    步骤S2：光谱反演：对于步骤S1输出的信号，由于该像素上的光谱透过率已知，在由N个像素组成的周期性结构中，根据光谱透过率曲线，N个像素组合，反演计算出N个像素的入射光谱值，计算方法如公式(3)所示，
    S i＝∫I(λ)T i(λ)η(λ)dλ，  (3)
    其中，S为光电转换基底输出的光信号强度值，I为入射光谱，是待求解信号，T为滤光薄膜的光谱透过率，η为光电转换基底的量子效率，λ为入射波长；
    步骤S3、将多光谱芯片本体输出的光谱信息乘以光谱到颜色变化矩阵，从而获得RGB图像；所述光谱到颜色变化矩阵预存在所述色彩还原模块内。
PCT/CN2021/095490 2020-03-24 2021-05-24 一种可实现彩色成像的多光谱成像芯片及彩色成像方法 WO2021190674A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010211115.0 2020-03-24
CN202010211115.0A CN113447118B (zh) 2020-03-24 2020-03-24 一种可实现彩色成像的多光谱成像芯片及彩色成像方法

Publications (1)

Publication Number Publication Date
WO2021190674A1 true WO2021190674A1 (zh) 2021-09-30

Family

ID=77806329

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/095490 WO2021190674A1 (zh) 2020-03-24 2021-05-24 一种可实现彩色成像的多光谱成像芯片及彩色成像方法

Country Status (2)

Country Link
CN (1) CN113447118B (zh)
WO (1) WO2021190674A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115315040A (zh) * 2022-07-25 2022-11-08 上海芯龙光电科技股份有限公司 一种基于rgbwcla七色合一的led全光谱色彩管理系统
WO2023088186A1 (zh) * 2021-11-18 2023-05-25 北京与光科技有限公司 基于光谱成像的图像处理方法、装置和电子设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113418864B (zh) * 2021-06-03 2022-09-16 奥比中光科技集团股份有限公司 一种多光谱图像传感器及其制造方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102768069A (zh) * 2012-07-27 2012-11-07 中国科学院空间科学与应用研究中心 一种互补测量的单光子光谱计数成像系统及方法
WO2016172173A1 (en) * 2015-04-20 2016-10-27 Stc.Unm Method for simultaneous spectrally resolved detection or imaging of items in multiple flowing streams
CN106847849A (zh) * 2016-12-30 2017-06-13 中国科学院西安光学精密机械研究所 一种基于超表面窄带滤光的多光谱芯片及其制备方法
CN107328475A (zh) * 2017-07-28 2017-11-07 中国科学院西安光学精密机械研究所 宽带滤光光谱成像仪以及自适应直接光谱分类方法
CN108416745A (zh) * 2018-02-02 2018-08-17 中国科学院西安光学精密机械研究所 一种具有色彩恒常性的图像自适应去雾增强方法
CN110108358A (zh) * 2019-03-28 2019-08-09 浙江大学 一种高光谱成像方法及装置
CN110375852A (zh) * 2019-06-28 2019-10-25 中国科学院西安光学精密机械研究所 多条带微窄带滤光片阵列光谱成像芯片及其实现方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7433042B1 (en) * 2003-12-05 2008-10-07 Surface Optics Corporation Spatially corrected full-cubed hyperspectral imager
CN106840398B (zh) * 2017-01-12 2018-02-02 南京大学 一种多光谱光场成像方法
CN108088561A (zh) * 2017-12-15 2018-05-29 哈尔滨工业大学 一种快照式光场-光谱成像仪及成像方法
CN109813450A (zh) * 2019-03-27 2019-05-28 中国航空工业集团公司北京长城计量测试技术研究所 一种瞬态成像式红外多光谱温度场测量装置
CN111866318A (zh) * 2019-04-29 2020-10-30 北京小米移动软件有限公司 多光谱成像方法、多光谱成像装置、移动终端和存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102768069A (zh) * 2012-07-27 2012-11-07 中国科学院空间科学与应用研究中心 一种互补测量的单光子光谱计数成像系统及方法
WO2016172173A1 (en) * 2015-04-20 2016-10-27 Stc.Unm Method for simultaneous spectrally resolved detection or imaging of items in multiple flowing streams
CN106847849A (zh) * 2016-12-30 2017-06-13 中国科学院西安光学精密机械研究所 一种基于超表面窄带滤光的多光谱芯片及其制备方法
CN107328475A (zh) * 2017-07-28 2017-11-07 中国科学院西安光学精密机械研究所 宽带滤光光谱成像仪以及自适应直接光谱分类方法
CN108416745A (zh) * 2018-02-02 2018-08-17 中国科学院西安光学精密机械研究所 一种具有色彩恒常性的图像自适应去雾增强方法
CN110108358A (zh) * 2019-03-28 2019-08-09 浙江大学 一种高光谱成像方法及装置
CN110375852A (zh) * 2019-06-28 2019-10-25 中国科学院西安光学精密机械研究所 多条带微窄带滤光片阵列光谱成像芯片及其实现方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023088186A1 (zh) * 2021-11-18 2023-05-25 北京与光科技有限公司 基于光谱成像的图像处理方法、装置和电子设备
CN115315040A (zh) * 2022-07-25 2022-11-08 上海芯龙光电科技股份有限公司 一种基于rgbwcla七色合一的led全光谱色彩管理系统
CN115315040B (zh) * 2022-07-25 2024-04-09 上海芯龙光电科技股份有限公司 一种基于rgbwcla七色合一的led全光谱色彩管理系统

Also Published As

Publication number Publication date
CN113447118A (zh) 2021-09-28
CN113447118B (zh) 2023-05-16

Similar Documents

Publication Publication Date Title
WO2021190674A1 (zh) 一种可实现彩色成像的多光谱成像芯片及彩色成像方法
Chu et al. Integrated polarization dependent photodetector and its application for polarization navigation
CN113497065B (zh) 兼具光谱和成像功能的成像光谱芯片及其制备方法
CN103733340A (zh) 固体摄像元件和摄像系统
CN111024229B (zh) 单芯片集成型光谱成像微系统光谱数据校正方法
JP2015012128A (ja) 撮像装置及び電子機器
Nordin et al. Diffractive optical element for Stokes vector measurement with a focal plane array
Wang et al. Single-layer nanowire polarizer integrated with photodetector and its application for polarization navigation
Zhou et al. A compact polarization-integrated long wavelength infrared focal plane array based on InAs/GaSb superlattice
Garcia et al. A 1300× 800, 700 mW, 30 fps spectral polarization imager
CN113884234A (zh) 一种互补单像素质心探测系统及方法
US20240147032A1 (en) Imaging device and optical element
CN110095189A (zh) 一种双目的八谱段多光谱相机设计方法
Sun et al. Fabrication and performance analysis of infrared InGaAs polarimetric detector with complete coverage of superpixel-structured grating
Jian et al. Research progress and applications of polarization integrated infrared photodetector
CN109781260B (zh) 超紧凑型快照式偏振光谱成像探测装置及探测方法
York et al. Calibration method for division of focal plane polarimeters in the optical and near-infrared regime
CN111998962A (zh) 一种基于阵列型二元相位调制的哈特曼波前传感器
CN115000107B (zh) 多光谱成像芯片、多光谱成像组件、制备方法及移动终端
Wen et al. On-chip Real-time Hyperspectral Imager with Full CMOS Resolution Enabled by Massively Parallel Neural Network
CN116417483A (zh) 一种光谱成像芯片及光谱成像芯片的制备方法
CN112304904B (zh) 基于滤波阵列的硅片反射率检测方法
Wang et al. Computational snapshot angular-spectral lensless imaging
CN113916798B (zh) 一种适用于典型偏振多谱段探测系统的信噪比估算方法
CN116600212B (zh) 一种基于大气层外环境光谱的白平衡定标方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21776901

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21776901

Country of ref document: EP

Kind code of ref document: A1