CN112504454A - Super-resolution spectral imaging filter structure based on pixel-level spectral modulation - Google Patents

Super-resolution spectral imaging filter structure based on pixel-level spectral modulation Download PDF

Info

Publication number
CN112504454A
CN112504454A CN202011341571.3A CN202011341571A CN112504454A CN 112504454 A CN112504454 A CN 112504454A CN 202011341571 A CN202011341571 A CN 202011341571A CN 112504454 A CN112504454 A CN 112504454A
Authority
CN
China
Prior art keywords
spectral
pixel
pixels
modulation
resolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011341571.3A
Other languages
Chinese (zh)
Other versions
CN112504454B (en
Inventor
赵安娜
刘舒扬
潘建旋
张晨
王天鹤
张云昊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Jinhang Institute of Technical Physics
Original Assignee
Tianjin Jinhang Institute of Technical Physics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Jinhang Institute of Technical Physics filed Critical Tianjin Jinhang Institute of Technical Physics
Priority to CN202011341571.3A priority Critical patent/CN112504454B/en
Publication of CN112504454A publication Critical patent/CN112504454A/en
Application granted granted Critical
Publication of CN112504454B publication Critical patent/CN112504454B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28Investigating the spectrum
    • G01J3/2823Imaging spectrometer
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28Investigating the spectrum
    • G01J3/2823Imaging spectrometer
    • G01J2003/2826Multispectral imaging, e.g. filter imaging

Landscapes

  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Solid State Image Pick-Up Elements (AREA)

Abstract

The invention belongs to the technical field of filters, and particularly relates to a super-resolution spectral imaging filter structure based on pixel-level spectral modulation, which is composed of 4 different pixel-level interference films and full-transmission spectral bands, wherein full-transmission pixels are arranged in 4 neighborhoods of all spectral modulation pixels, and 4 different channels of narrow-band spectral modulation pixels are arranged in 4 neighborhoods of all full-transmission pixels. The novel structure comprises 5 different optical channels, can form a snapshot type spectral imaging sensor based on a CMOS (complementary metal oxide semiconductor) process and having pixel-level spectral modulation capability and high-resolution and super-resolution imaging capabilities, and effectively avoids crosstalk between pixels.

Description

Super-resolution spectral imaging filter structure based on pixel-level spectral modulation
Technical Field
The invention belongs to the technical field of filters, and particularly relates to a super-resolution spectral imaging filter structure based on pixel-level spectral modulation.
Background
The spectral imaging technology is a non-contact nondestructive detection technology and can be used for detecting multiple information of the appearance and the components of a substance in real time. Two-dimensional space and one-dimensional spectral information of a substance can be simultaneously obtained through a spectral imaging technology. The traditional spectral imaging equipment usually adopts core light splitting elements with larger volume, such as prisms, gratings and the like, and along with the rising of detection requirements and the promotion of detection environment diversity and complexity, a microminiature spectral imaging chip based on thin film light splitting with small volume, low quality, high imaging efficiency and low cost is disclosed.
The spectral imaging microsystem based on film beam split mainly carries out the process processing of an interference film on a CMOS sensor based on a semiconductor process according to an interference beam split principle, spectral imaging sensor chips with different structures are designed according to different imaging modes, and the spectral imaging microsystem based on film beam split can be divided into a line scanning type chip, a tile type chip and a mosaic type chip according to chip structures. The mosaic type spectral imaging sensing chip can realize visible-near infrared imaging of pixel-level spectral modulation by changing the interference film structures corresponding to different pixels on the CMOS sensor.
The existing mosaic type visible light range or visible-near infrared range spectral imaging sensor chip is composed of a CMOS sensor substrate and interference films with pixel-level spectral modulation structures grown on photosensitive surface pixels of the CMOS sensor substrate. For an n × n channel spectral imaging chip, an n × n filter unit group arrangement format is generally adopted, filter unit groups composed of n × n channels are periodically arranged in the range of the sensor photosurface, and a spectral modulation filter structure is configured as shown in the 2 × 2 channel example of fig. 1: the spectral modulation filter structure comprises 4 channels, wherein each channel is arranged according to a Chinese character 'tian' to form a filtering unit group, and the filtering unit groups are periodically arranged to form a complete spectral modulation filter covering a photosensitive surface within the range of the photosensitive surface. For a sensor comprising 2 hx 2r picture elements, hx r groups of filter cells may be formed. In the data acquisition process, the image information and the spectrum information of the object in the target scene can be obtained by shooting once according to snapshot acquisition.
However, the prior art has the following disadvantages:
disadvantage 1: inter-pixel crosstalk
The traditional mosaic type spectrum modulation structure adopts an interference film to perform pixel-level narrow-band spectrum modulation, adjacent pixels are narrow-band modulation of light waves with different central wavelengths, photoelectric crosstalk exists between the adjacent pixels, and the accuracy of acquired spectrum data is influenced.
And (2) disadvantage: loss of image resolution
The existing nxn channel spectral imaging sensor chip can acquire image information and scene information of a target object by shooting once in the data acquisition process. In the data processing process, the two-dimensional mixed information needs to be restored to a two-dimensional image information and one-dimensional spectrum information form, and in the process, the original data needs to be down-sampled. Take a 4-spectrum snapshot spectrum imaging chip with a spatial resolution of 2h × 2r pixels (the number of row pixels is 2h, and the number of column pixels is 2r) as an example: in a single filter unit group, the spectral bands are arranged according to 2 × 2 channels, so that h × r filter unit groups can be formed. In the down-sampling process, image information in each filtering unit group is combined to form h multiplied by r multiplied by 4 spectral imaging data, wherein the first two dimensions are the spatial resolution of the data, and the third dimension is the number of spectral channels of the data. That is, in the process, the image dimension resolution needs to be sacrificed for extracting the spectral dimension data, and when the number of spectral dimension channels is increased, the image resolution of the data is further reduced. Data acquisition is carried out under the mosaic structure, and the image resolution performance of the sensor is difficult to improve.
Disclosure of Invention
Technical problem to be solved
The technical problem to be solved by the invention is as follows: how to put forward a pixel level spectrum modulation new structure that can be used for super-resolution spectral imaging, this structure still arranges in the form of mosaic mainly, but require it on the basis of obtaining the dimensional information of spectrum, realize the pixel of image dimension is lossless to be imaged, super-resolution is imaged.
(II) technical scheme
In order to solve the technical problem, the invention provides a super-resolution spectral imaging filter structure based on pixel-level spectral modulation, the filter structure adopts a mosaic type spectral imaging sensor chip structure, each filter unit is composed of a filter structure with 16 pixels arranged according to 4 × 4, each filter unit group contains 8 spectral modulation spectral bands of 4 × 2, 4 spectral modulation channels are contained, and the rest 8 pixels are full-transmission spectral bands, and 5 channels are counted.
Wherein, each filtering unit group in the filter structure traverses from left to right and from top to bottom in sequence, which is:
a spectral band 4, a full transmission spectral band, a spectral band 1 and a full transmission spectral band;
a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3;
a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band;
full transmittance spectrum, spectrum 3, full transmittance spectrum, spectrum 2.
The sensor edge pixels are not considered, all the 4 neighborhoods of all the spectrum modulation pixels are full-transmission pixels, and all the 4 neighborhoods of all the full-transmission pixels are 4 different channels of the narrow-band spectrum modulation pixels.
In the global range of the spectral imaging filter chip, the filter unit group which is arranged by 4 multiplied by 4 and comprises 8 spectral modulation spectral bands and 8 full transmission spectral bands is formed by two-dimensional periodic arrangement, and a high-resolution imaging filter structure modulated by a pixel level is formed.
(III) advantageous effects
The invention provides a novel structure of a spectrum modulation filter of an interference film based on CMOS (complementary metal oxide semiconductor) sensor integrated pixel level modulation, which is formed by arranging 4 different pixel level interference films and a total transmission spectrum section according to the following sequence: each filtering unit group consists of 4 multiplied by 4 pixels, and the pixel level modulation structure is traversed from left to right and from top to bottom in sequence, namely a spectrum section 4, a full transmittance spectrum section, a spectrum section 1 and a full transmittance spectrum section; a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3; a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band; full transmittance spectrum, spectrum 3, full transmittance spectrum, spectrum 2. The edge pixels of the sensor are not considered, all the 4 neighborhoods of all the spectrum modulation pixels are all full-transmission pixels, and 4 different channels of the narrow-band spectrum modulation pixels are all arranged in the 4 neighborhoods of all the full-transmission pixels.
The novel structure comprises 5 different optical channels, and can form a snapshot type spectral imaging sensor based on a CMOS (complementary metal oxide semiconductor) process and having pixel-level spectral modulation capability and high-resolution and super-resolution imaging capabilities.
Compared with the prior art, the invention has the following beneficial effects:
(1) avoiding inter-pixel crosstalk
The invention adopts a novel pixel-level modulation spectral imaging filter structure, can realize that all pixels (with edge being not counted) in 4 neighborhoods of pixels of all the spectral modulation structures in a snapshot mode are all 4 different channels of narrow-band spectral modulation pixels in 4 neighborhoods of all the spectral modulation structures, and effectively avoids crosstalk among pixels.
(2) The invention adopts a novel pixel-level modulation spectral imaging filter structure, and can acquire more spatial information while acquiring spectral information. Calculating image information in a certain spectral modulation pixel by using image information of a full-transmission pixel around the spectral modulation pixel through numerical modeling; meanwhile, the spectral information of a certain full-transmission spectrum band can be resolved by an interpolation method according to the spectral information around the full-transmission spectrum band, the spatial resolution is improved without sacrificing the spectral resolution, and finally high-resolution and super-resolution imaging of 5-channel pixel-level spectral modulation is realized.
In other words, for a sensor with 2h × 2r pixel spatial resolution, edge pixels are removed, and spectral imaging data with dimensions of (2h-2) × (2r-2) × 4 can be acquired by using the structure, in a practical situation, h >1, r >1 is greatly improved compared with the h × r × 4 data information amount of the existing structure, and the image resolution is improved by nearly 4 times.
Drawings
Fig. 1 is a schematic diagram of a 2 × 2 mosaic type spectral imaging sensor chip filter unit group structure.
Fig. 2 is a schematic diagram of a filter unit group structure of a pixel-level spectrally modulated 5-channel high-resolution imaging filter.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
In order to solve the problems in the prior art, the invention provides a super-resolution spectral imaging filter structure based on pixel-level spectral modulation, the filter structure adopts a mosaic type spectral imaging sensor chip structure, each filter unit is composed of a filter structure with 16 pixels arranged according to 4 multiplied by 4, each filter unit group comprises 8 spectral modulation spectral bands of 4 multiplied by 2, 4 spectral modulation channels are included, and the rest 8 pixels are full transmission spectral bands, and 5 channels are counted.
As shown in fig. 2, each filtering unit group in the filter structure traverses from left to right and from top to bottom sequentially as follows:
a spectral band 4, a full transmission spectral band, a spectral band 1 and a full transmission spectral band;
a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3;
a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band;
full transmittance spectrum, spectrum 3, full transmittance spectrum, spectrum 2.
The sensor edge pixels are not considered, all the 4 neighborhoods of all the spectrum modulation pixels are full-transmission pixels, and all the 4 neighborhoods of all the full-transmission pixels are 4 different channels of the narrow-band spectrum modulation pixels.
Therefore, in the global range of the spectral imaging filter chip, the filter unit groups which are arranged by 4 multiplied by 4 and comprise 8 spectral modulation spectral bands and 8 full transmission spectral bands are arranged according to two-dimensional periodicity, and a high-resolution imaging filter structure with pixel level modulation is formed.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (5)

1. A super-resolution spectral imaging filter structure based on pixel-level spectral modulation is characterized in that the filter structure adopts a mosaic type spectral imaging sensor chip structure, each filter unit is composed of filter structures with 16 pixels arranged according to 4 x 4, each filter unit group comprises 8 spectral modulation spectral bands of 4 x 2, 4 spectral modulation channels are included, and the rest 8 pixels are full transmission spectral bands and 5 channels are included;
the filter structure is characterized in that each filtering unit group in the filter structure traverses from left to right and from top to bottom sequentially, and the sequence is as follows:
a spectral band 4, a full transmission spectral band, a spectral band 1 and a full transmission spectral band;
a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3;
a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band;
full transmittance spectrum, spectrum 3, full transmittance spectrum, spectrum 2.
2. The pixel-level spectral modulation-based super-resolution spectral imaging filter structure of claim 1, wherein all 4 neighborhoods of each spectral modulation pixel are all transflective pixels, and all 4 different channels of narrow-band spectral modulation pixels are all within 4 neighborhoods of each transflective pixel, regardless of sensor edge pixels.
3. The super-resolution spectral imaging filter structure based on pixel-level spectral modulation according to claim 2, wherein the spectral imaging filter chip is formed by the 4 x 4 arrangement of filter unit groups including 8 spectral modulation bands and 8 full transmission bands in a two-dimensional periodic arrangement in a global range, so as to form a pixel-level modulated high-resolution imaging filter structure.
4. The pixel-level spectral modulation-based super-resolution spectral imaging filter structure according to claim 1, wherein the filter structure implements 4 different channels of a fully transparent pixel in 4 neighborhoods of pixels of snapshot-type spectral modulation structures, and a narrowband spectral modulation pixel in 4 neighborhoods of the fully transparent pixels, thereby effectively avoiding crosstalk between pixels.
5. The pixel-level spectral modulation-based super-resolution spectral imaging filter structure of claim 1, wherein the data information content of the spectral imaging data formed by the filter structure is greatly improved, and the image resolution is improved by approximately 4 times.
CN202011341571.3A 2020-11-25 2020-11-25 Super-resolution spectral imaging filter structure based on pixel-level spectral modulation Active CN112504454B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011341571.3A CN112504454B (en) 2020-11-25 2020-11-25 Super-resolution spectral imaging filter structure based on pixel-level spectral modulation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011341571.3A CN112504454B (en) 2020-11-25 2020-11-25 Super-resolution spectral imaging filter structure based on pixel-level spectral modulation

Publications (2)

Publication Number Publication Date
CN112504454A true CN112504454A (en) 2021-03-16
CN112504454B CN112504454B (en) 2022-11-22

Family

ID=74966090

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011341571.3A Active CN112504454B (en) 2020-11-25 2020-11-25 Super-resolution spectral imaging filter structure based on pixel-level spectral modulation

Country Status (1)

Country Link
CN (1) CN112504454B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116679469A (en) * 2023-05-12 2023-09-01 浙江大学 Space-time co-modulation adjustable filter chip and spectrum imaging restoration method and system thereof

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101309429A (en) * 2007-05-15 2008-11-19 索尼株式会社 Apparatus and method for processing video signal, imaging device and computer program
CN101345248A (en) * 2007-07-09 2009-01-14 博立码杰通讯(深圳)有限公司 Multi-optical spectrum light-sensitive device and preparation thereof
US20150212294A1 (en) * 2013-07-30 2015-07-30 Panasonic Intellectual Property Management Co., Ltd. Imaging apparatus, imaging system that includes imaging apparatus, electron mirror system that includes imaging apparatus, and ranging apparatus that includes imaging apparatus
US20150276478A1 (en) * 2011-11-04 2015-10-01 Imec Spectral camera with mosaic of filters for each image pixel
CN108051087A (en) * 2017-12-14 2018-05-18 西安理工大学 A kind of eight passage multispectral camera design methods for fast imaging
CN108426639A (en) * 2017-02-14 2018-08-21 芯视达系统公司 Multispectral sensor-based system and method
CN109141632A (en) * 2018-11-06 2019-01-04 德淮半导体有限公司 Pixel unit, imaging sensor and its manufacturing method and imaging device
CN109827658A (en) * 2019-03-22 2019-05-31 天津津航技术物理研究所 The gazing type spectrum chip structure and preparation method thereof of Oriented Green vegetation detection
CN110095189A (en) * 2019-04-24 2019-08-06 西安理工大学 A kind of eight spectral coverage multispectral camera design methods of binocular
CN110324546A (en) * 2018-03-28 2019-10-11 黑魔法设计私人有限公司 Image processing method and filter array
CN110649056A (en) * 2019-09-30 2020-01-03 Oppo广东移动通信有限公司 Image sensor, camera assembly and mobile terminal
CN110719447A (en) * 2019-10-12 2020-01-21 南京威派视半导体技术有限公司 Image sensor with multichannel narrowband color filter array
JP2020150267A (en) * 2020-05-21 2020-09-17 ソニー株式会社 Solid state image sensor, electronic apparatus, and manufacturing method of solid state image sensor

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101309429A (en) * 2007-05-15 2008-11-19 索尼株式会社 Apparatus and method for processing video signal, imaging device and computer program
CN101345248A (en) * 2007-07-09 2009-01-14 博立码杰通讯(深圳)有限公司 Multi-optical spectrum light-sensitive device and preparation thereof
US20150276478A1 (en) * 2011-11-04 2015-10-01 Imec Spectral camera with mosaic of filters for each image pixel
US20150212294A1 (en) * 2013-07-30 2015-07-30 Panasonic Intellectual Property Management Co., Ltd. Imaging apparatus, imaging system that includes imaging apparatus, electron mirror system that includes imaging apparatus, and ranging apparatus that includes imaging apparatus
CN108426639A (en) * 2017-02-14 2018-08-21 芯视达系统公司 Multispectral sensor-based system and method
CN108051087A (en) * 2017-12-14 2018-05-18 西安理工大学 A kind of eight passage multispectral camera design methods for fast imaging
CN110324546A (en) * 2018-03-28 2019-10-11 黑魔法设计私人有限公司 Image processing method and filter array
CN109141632A (en) * 2018-11-06 2019-01-04 德淮半导体有限公司 Pixel unit, imaging sensor and its manufacturing method and imaging device
CN109827658A (en) * 2019-03-22 2019-05-31 天津津航技术物理研究所 The gazing type spectrum chip structure and preparation method thereof of Oriented Green vegetation detection
CN110095189A (en) * 2019-04-24 2019-08-06 西安理工大学 A kind of eight spectral coverage multispectral camera design methods of binocular
CN110649056A (en) * 2019-09-30 2020-01-03 Oppo广东移动通信有限公司 Image sensor, camera assembly and mobile terminal
CN110719447A (en) * 2019-10-12 2020-01-21 南京威派视半导体技术有限公司 Image sensor with multichannel narrowband color filter array
JP2020150267A (en) * 2020-05-21 2020-09-17 ソニー株式会社 Solid state image sensor, electronic apparatus, and manufacturing method of solid state image sensor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BERT GEELEN: "A tiny, VIS-NIR snapshot multispectral camera", 《SPIE》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116679469A (en) * 2023-05-12 2023-09-01 浙江大学 Space-time co-modulation adjustable filter chip and spectrum imaging restoration method and system thereof

Also Published As

Publication number Publication date
CN112504454B (en) 2022-11-22

Similar Documents

Publication Publication Date Title
Geelen et al. A compact snapshot multispectral imager with a monolithically integrated per-pixel filter mosaic
Lapray et al. Multispectral filter arrays: Recent advances and practical implementation
US9681057B2 (en) Exposure timing manipulation in a multi-lens camera
US8149400B2 (en) Coded aperture snapshot spectral imager and method therefor
CN101102388B (en) Image sensor and image sensing method using the same
CN102037717B (en) Capturing and processing of images using monolithic camera array with hetergeneous imagers
CN103579268B (en) There is the lens arra of the segmentation imageing sensor of color filter
CN102037734B (en) Camera sensor correction
US9110240B2 (en) Phase gratings with odd symmetry for high-resolution lensed and lensless optical sensing
JP4500360B2 (en) Image processing device
CN103703413B (en) Color is used to be correlated with the system and method for the depth of field in wavefront coded expansion lens system
US20070279618A1 (en) Imaging Apparatus And Image Improving Method
EP3129813B1 (en) Low-power image change detector
JP2001169050A (en) Photosensor array having many different sensor areas
CN102450020A (en) Four-channel color filter array interpolation
KR20160065464A (en) Color filter array, image sensor having the same and infrared data acquisition method using the same
EP3700194A1 (en) Imaging device and method, and image processing device and method
CN112924028B (en) Light field polarization imaging detection system for sea surface oil spill
CN111741239B (en) Image sensor and electronic device
US20140054448A1 (en) Color filter array and image sensor
US20140307135A1 (en) Color imaging element
CN112504454B (en) Super-resolution spectral imaging filter structure based on pixel-level spectral modulation
CN109827658B (en) Staring type spectrum chip structure for green vegetation detection and preparation method thereof
US8213710B2 (en) Apparatus and method for shift invariant differential (SID) image data interpolation in non-fully populated shift invariant matrix
CN111787237B (en) Pixel, image sensor, focusing method and device and terminal equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant