CN112504454B - Super-resolution spectral imaging filter structure based on pixel-level spectral modulation - Google Patents
Super-resolution spectral imaging filter structure based on pixel-level spectral modulation Download PDFInfo
- Publication number
- CN112504454B CN112504454B CN202011341571.3A CN202011341571A CN112504454B CN 112504454 B CN112504454 B CN 112504454B CN 202011341571 A CN202011341571 A CN 202011341571A CN 112504454 B CN112504454 B CN 112504454B
- Authority
- CN
- China
- Prior art keywords
- spectral
- pixels
- modulation
- full
- transmission
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003595 spectral effect Effects 0.000 title claims abstract description 79
- 238000000701 chemical imaging Methods 0.000 title claims abstract description 34
- 238000003384 imaging method Methods 0.000 claims abstract description 9
- 238000001228 spectrum Methods 0.000 claims description 34
- 238000000411 transmission spectrum Methods 0.000 claims description 27
- 230000005540 biological transmission Effects 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 11
- 238000000034 method Methods 0.000 abstract description 11
- 239000004065 semiconductor Substances 0.000 abstract description 4
- 230000000295 complement effect Effects 0.000 abstract description 3
- 229910044991 metal oxide Inorganic materials 0.000 abstract description 3
- 150000004706 metal oxides Chemical class 0.000 abstract description 3
- 230000003287 optical effect Effects 0.000 abstract description 3
- 239000010408 film Substances 0.000 description 8
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003333 near-infrared imaging Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
- G01J2003/2826—Multispectral imaging, e.g. filter imaging
Abstract
The invention belongs to the technical field of filters, and particularly relates to a super-resolution spectral imaging filter structure based on pixel-level spectral modulation, which is composed of 4 different pixel-level interference films and full-transmission spectral bands, wherein full-transmission pixels are arranged in 4 neighborhoods of all spectral modulation pixels, and 4 different channels of narrow-band spectral modulation pixels are arranged in 4 neighborhoods of all full-transmission pixels. The novel structure comprises 5 different optical channels, can form a snapshot type spectral imaging sensor based on a CMOS (complementary metal oxide semiconductor) process and having pixel-level spectral modulation capability and high-resolution and super-resolution imaging capabilities, and effectively avoids crosstalk among pixels.
Description
Technical Field
The invention belongs to the technical field of filters, and particularly relates to a super-resolution spectral imaging filter structure based on pixel-level spectral modulation.
Background
The spectral imaging technology is a non-contact nondestructive detection technology and can be used for detecting multiple information of the appearance and the components of a substance in real time. Two-dimensional space and one-dimensional spectral information of a substance can be simultaneously obtained through a spectral imaging technology. The traditional spectral imaging equipment usually adopts core light splitting elements with larger volume, such as prisms, gratings and the like, and along with the rising of detection requirements and the improvement of diversity and complexity of detection environments, a microminiature spectral imaging chip based on thin film light splitting with small volume, low quality, high imaging efficiency and low cost is disclosed.
The spectral imaging micro-system based on film light splitting mainly carries out process processing of an interference film on a CMOS sensor based on a semiconductor process according to an interference light splitting principle, spectral imaging sensor chips with different structures are designed according to different imaging modes, and the spectral imaging micro-system based on film light splitting can be divided into a line scanning type chip, a tile type chip and a mosaic type chip according to chip structures. The mosaic type spectral imaging sensing chip can realize visible-near infrared imaging of pixel-level spectral modulation by changing the interference film structures corresponding to different pixels on the CMOS sensor.
The existing mosaic type visible light range or visible-near infrared range spectral imaging sensor chip is composed of a CMOS sensor substrate and interference films with pixel-level spectral modulation structures grown on pixels of a photosensitive surface of the CMOS sensor substrate. For an n × n channel spectral imaging chip, an n × n filter unit group arrangement format is generally adopted, filter unit groups composed of n × n channels are periodically arranged in the range of the sensor photosurface, and a spectral modulation filter structure is configured as shown in the 2 × 2 channel example of fig. 1: the spectral modulation filter structure comprises 4 channels, wherein each channel is arranged according to a Chinese character 'tian' to form a filtering unit group, and the filtering unit groups are periodically arranged to form a complete spectral modulation filter covering a photosensitive surface within the range of the photosensitive surface. For a sensor comprising 2 hx 2r picture elements, hx r groups of filter cells may be formed. In the data acquisition process, the image information and the spectrum information of the object in the target scene can be obtained by shooting once according to snapshot acquisition.
However, the prior art has the following disadvantages:
disadvantage 1: inter-pixel crosstalk
The traditional mosaic type spectrum modulation structure adopts an interference film to perform pixel-level narrow-band spectrum modulation, adjacent pixels are narrow-band modulation of light waves with different central wavelengths, photoelectric crosstalk exists between the adjacent pixels, and the accuracy of acquired spectrum data is influenced.
And (2) a defect: loss of image resolution
The existing nxn channel spectral imaging sensor chip can acquire image information and scene information of a target object by shooting once in the data acquisition process. In the data processing process, the two-dimensional mixed information needs to be restored to a two-dimensional image information and one-dimensional spectrum information form, and in the process, the original data needs to be down-sampled. Take a 4-spectral-band snapshot type spectral imaging chip with a spatial resolution of 2h × 2r pixels (the number of row pixels is 2h, and the number of column pixels is 2 r) as an example: in a single filter unit group, the spectral bands are arranged according to 2 × 2 channels, and h × r filter unit groups can be formed. In the down-sampling process, image information in each filter unit group is combined to form h multiplied by r multiplied by 4 spectral imaging data, wherein the first two dimensions are the spatial resolution of the data, and the third dimension is the number of spectral channels of the data. That is, in the process, the image dimension resolution needs to be sacrificed for extracting the spectral dimension data, and when the number of spectral dimension channels is increased, the image resolution of the data is further reduced. Data acquisition is carried out under the mosaic structure, and the image resolution performance of the sensor is difficult to improve.
Disclosure of Invention
Technical problem to be solved
The technical problem to be solved by the invention is as follows: how to put forward a pixel level spectrum modulation new structure that can be used for super-resolution spectral imaging, this structure still arranges in the form of mosaic mainly, but require it on the basis of obtaining the dimensional information of spectrum, realize the pixel of image dimension is lossless to be imaged, super-resolution is imaged.
(II) technical scheme
In order to solve the technical problem, the invention provides a super-resolution spectral imaging filter structure based on pixel-level spectral modulation, the filter structure adopts a mosaic type spectral imaging sensor chip structure, each filter unit is composed of a filter structure with 16 pixels arranged according to 4 × 4, each filter unit group contains 8 spectral modulation spectral bands of 4 × 2, 4 spectral modulation channels are contained, and the rest 8 pixels are full-transmission spectral bands, and 5 channels are counted.
Wherein, each filtering unit group in the filter structure traverses from left to right and from top to bottom in sequence, which is:
a spectrum section 4, a full transmission spectrum section, a spectrum section 1 and a full transmission spectrum section;
a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3;
a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band;
full transmittance spectrum, spectrum 3, full transmittance spectrum, spectrum 2.
The sensor edge pixels are not considered, all the 4 neighborhoods of all the spectrum modulation pixels are full-transmission pixels, and all the 4 neighborhoods of all the full-transmission pixels are 4 different channels of the narrow-band spectrum modulation pixels.
In the global range of the spectral imaging filter chip, the 4 × 4 arrayed optical filtering unit groups containing 8 spectral modulation spectral bands and 8 full transmission spectral bands are arrayed periodically according to two dimensions to form a pixel-level modulated high-resolution imaging filter structure.
(III) advantageous effects
The invention provides a novel structure of a spectrum modulation filter of an interference film based on CMOS (complementary metal oxide semiconductor) sensor integrated pixel level modulation, which is formed by arranging 4 different pixel level interference films and a total transmission spectrum section according to the following sequence: each filtering unit group consists of 4 multiplied by 4 pixels, and the pixel level modulation structure is traversed from left to right and from top to bottom in sequence, namely a spectrum section 4, a full transmittance spectrum section, a spectrum section 1 and a full transmittance spectrum section; a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3; a spectrum section 1, a full transmission spectrum section, a spectrum section 4 and a full transmission spectrum section; full transmittance spectrum, spectrum 3, full transmittance spectrum, spectrum 2. And 4, all the pixels in the 4 neighborhoods of all the spectral modulation pixels are full-transmission pixels without considering the edge pixels of the sensor, and all the pixels in the 4 neighborhoods of all the full-transmission pixels are 4 different channels of the narrow-band spectral modulation pixels.
The novel structure comprises 5 different optical channels, and can form a snapshot type spectral imaging sensor based on a CMOS (complementary metal oxide semiconductor) process and having pixel-level spectral modulation capability and high-resolution and super-resolution imaging capabilities.
Compared with the prior art, the invention has the following beneficial effects:
(1) Avoiding inter-pixel crosstalk
The invention adopts a novel pixel-level modulation spectral imaging filter structure, can realize that all pixels (edge is not counted) in 4 neighborhoods of the pixels of each snapshot type spectral modulation structure are all full-transmission pixels, and all the 4 neighborhoods of all the full-transmission pixels are 4 different channels of narrow-band spectral modulation pixels, thereby effectively avoiding crosstalk among the pixels.
(2) The invention adopts a novel pixel-level modulation spectral imaging filter structure, and can acquire more spatial information while acquiring spectral information. Calculating image information in a certain spectral modulation pixel by using image information of a full-transmission pixel around the spectral modulation pixel through numerical modeling; meanwhile, the spectral information of a full-transmission spectrum section can be resolved according to the spectral information around the full-transmission spectrum section by an interpolation method, the spatial resolution is improved while the spectral resolution is not sacrificed, and finally high-resolution and super-resolution imaging of 5-channel pixel-level spectral modulation is realized.
In other words, for a sensor with 2h × 2r pixel spatial resolution, edge pixels are removed, and spectral imaging data with dimensions of (2 h-2) × (2 r-2) × 4 can be acquired by adopting the structure, in a practical situation, h >1, r >1 is greatly improved compared with the h × r × 4 data information amount of the existing structure, and the image resolution is improved by nearly 4 times.
Drawings
Fig. 1 is a schematic diagram of a 2 × 2 mosaic type spectral imaging sensor chip filter unit group structure.
Fig. 2 is a schematic diagram of a filter unit group structure of a pixel-level spectrally modulated 5-channel high-resolution imaging filter.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
In order to solve the problems in the prior art, the invention provides a super-resolution spectral imaging filter structure based on pixel-level spectral modulation, the filter structure adopts a mosaic type spectral imaging sensor chip structure, each filter unit is composed of a filter structure with 16 pixels arranged according to 4 multiplied by 4, each filter unit group comprises 8 spectral modulation spectral bands of 4 multiplied by 2, 4 spectral modulation channels are included, and the rest 8 pixels are full transmission spectral bands, and 5 channels are counted.
As shown in fig. 2, each filtering unit group in the filter structure traverses from left to right and from top to bottom sequentially as follows:
a spectrum section 4, a full transmission spectrum section, a spectrum section 1 and a full transmission spectrum section;
a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3;
a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band;
full-transmission spectrum section, spectrum section 3, full-transmission spectrum section and spectrum section 2.
And 4 neighborhoods of all the spectral modulation pixels are all the full-transmission pixels without considering the edge pixels of the sensor, and 4 different channels of the narrow-band spectral modulation pixels are all arranged in 4 neighborhoods of all the full-transmission pixels.
Therefore, in the global range of the spectral imaging filter chip, the 4 multiplied by 4 arranged filtering unit groups containing 8 spectral modulation spectral bands and 8 full transmission spectral bands are arranged periodically according to two dimensions to form a high-resolution imaging filter structure with pixel level modulation.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, it is possible to make various improvements and modifications without departing from the technical principle of the present invention, and those improvements and modifications should be considered as the protection scope of the present invention.
Claims (1)
1. A super-resolution spectral imaging filter structure based on pixel-level spectral modulation is characterized in that the filter structure adopts a mosaic type spectral imaging sensor chip structure, each filtering unit is formed by a filtering structure in which 16 pixels are arranged according to 4 x 4, each filtering unit comprises 8 spectral modulation spectral bands of 4 x 2, including 4 spectral modulation channels, and the rest 8 pixels are full transmission spectral bands, and the total number of the spectral modulation spectral bands is 5;
the filter units in the filter structure traverse from left to right and from top to bottom in sequence, and the sequence is as follows:
a spectral band 4, a full transmission spectral band, a spectral band 1 and a full transmission spectral band;
a full transmission spectrum section, a spectrum section 2, a full transmission spectrum section and a spectrum section 3;
a spectral band 1, a full transmission spectral band, a spectral band 4 and a full transmission spectral band;
a full transmission spectrum section, a spectrum section 3, a full transmission spectrum section and a spectrum section 2;
all the pixels in 4 neighborhoods of all the spectral modulation pixels are full-transmission pixels, and all the pixels in 4 neighborhoods of all the spectral modulation pixels are 4 different channels of the narrow-band spectral modulation pixels, without considering the edge pixels of the sensor;
in the global range of the spectral imaging filter chip, the 4 multiplied by 4 arranged filtering units containing 8 spectral modulation spectral bands and 8 full transmission spectral bands are arranged according to two-dimensional periodicity to form a high-resolution imaging filter structure with pixel level modulation;
the filter structure realizes that all pixels in 4 neighborhoods of all the spectral modulation structure pixels in a snapshot mode are full-transmission pixels, and all the 4 neighborhoods of all the full-transmission pixels are 4 different channels of narrow-band spectral modulation pixels, so that crosstalk among the pixels is effectively avoided;
the data information quantity of the spectral imaging data formed by the filter structure is greatly improved, and the image resolution is improved by nearly 4 times.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011341571.3A CN112504454B (en) | 2020-11-25 | 2020-11-25 | Super-resolution spectral imaging filter structure based on pixel-level spectral modulation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011341571.3A CN112504454B (en) | 2020-11-25 | 2020-11-25 | Super-resolution spectral imaging filter structure based on pixel-level spectral modulation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112504454A CN112504454A (en) | 2021-03-16 |
| CN112504454B true CN112504454B (en) | 2022-11-22 |
Family
ID=74966090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011341571.3A Active CN112504454B (en) | 2020-11-25 | 2020-11-25 | Super-resolution spectral imaging filter structure based on pixel-level spectral modulation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112504454B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020150267A (en) * | 2020-05-21 | 2020-09-17 | ソニー株式会社 | Solid state image sensor, electronic apparatus, and manufacturing method of solid state image sensor |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008288629A (en) * | 2007-05-15 | 2008-11-27 | Sony Corp | Image signal processing apparatus, imaging device, image signal processing method, and computer program |
| CN101345248B (en) * | 2007-07-09 | 2010-07-14 | 博立码杰通讯(深圳)有限公司 | Multi-optical spectrum light-sensitive device and preparation thereof |
| EP2773929B1 (en) * | 2011-11-04 | 2023-12-13 | IMEC vzw | Spectral camera with mosaic of filters for each image pixel |
| JP6260006B2 (en) * | 2013-07-30 | 2018-01-17 | パナソニックＩｐマネジメント株式会社 | IMAGING DEVICE, IMAGING SYSTEM USING THE SAME, ELECTRONIC MIRROR SYSTEM, AND RANGING DEVICE |
| US10444415B2 (en) * | 2017-02-14 | 2019-10-15 | Cista System Corp. | Multispectral sensing system and method |
| CN108051087B (en) * | 2017-12-14 | 2020-02-18 | 西安理工大学 | Eight-channel multispectral camera design method for rapid imaging |
| JP7349806B2 (en) * | 2018-03-28 | 2023-09-25 | ブラックマジック デザイン ピーティーワイ リミテッド | Image processing method and filter array |
| CN109141632A (en) * | 2018-11-06 | 2019-01-04 | 德淮半导体有限公司 | Pixel unit, imaging sensor and its manufacturing method and imaging device |
| CN109827658B (en) * | 2019-03-22 | 2020-07-28 | 天津津航技术物理研究所 | Staring type spectrum chip structure for green vegetation detection and preparation method thereof |
| CN110095189B (en) * | 2019-04-24 | 2020-09-25 | 西安理工大学 | Binocular eight-spectral-band multispectral camera design method |
| CN110649056B (en) * | 2019-09-30 | 2022-02-18 | Oppo广东移动通信有限公司 | Image sensor, camera assembly and mobile terminal |
| CN110719447A (en) * | 2019-10-12 | 2020-01-21 | 南京威派视半导体技术有限公司 | Image sensor with multichannel narrowband color filter array |
-
2020
- 2020-11-25 CN CN202011341571.3A patent/CN112504454B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020150267A (en) * | 2020-05-21 | 2020-09-17 | ソニー株式会社 | Solid state image sensor, electronic apparatus, and manufacturing method of solid state image sensor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112504454A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101102388B (en) | Image sensor and image sensing method using the same | |
| Lapray et al. | Multispectral filter arrays: Recent advances and practical implementation | |
| CN103579268B (en) | There is the lens arra of the segmentation imageing sensor of color filter | |
| Geelen et al. | A compact snapshot multispectral imager with a monolithically integrated per-pixel filter mosaic | |
| US8149400B2 (en) | Coded aperture snapshot spectral imager and method therefor | |
| JP4500360B2 (en) | Image processing device | |
| JP2006033493A (en) | Imaging apparatus | |
| US20130278802A1 (en) | Exposure timing manipulation in a multi-lens camera | |
| CN102457683A (en) | Lens and colour filter arrangement, super-resolution camera system and method | |
| CN101106641B (en) | Method and apparatus for image processing | |
| CN102450020A (en) | Four-channel color filter array interpolation | |
| KR20160065464A (en) | Color filter array, image sensor having the same and infrared data acquisition method using the same | |
| CN103688536A (en) | Image processing device, image processing method, and program | |
| US9188480B2 (en) | Color filter array and image sensor | |
| JP2018107794A (en) | Image generating apparatus and imaging apparatus | |
| US20140307135A1 (en) | Color imaging element | |
| CN111741239B (en) | Image sensor and electronic device | |
| US8976275B2 (en) | Color imaging element | |
| CA2443658C (en) | Method and apparatus for sensing and interpolating color image data | |
| CN112504454B (en) | Super-resolution spectral imaging filter structure based on pixel-level spectral modulation | |
| WO2014084730A1 (en) | System and method for generating image using multiple lenses and multiple imagers | |
| US8711257B2 (en) | Color imaging device | |
| US6970608B1 (en) | Method for obtaining high-resolution performance from a single-chip color image sensor | |
| EP1528814A2 (en) | Imaging apparatus | |
| CN216873256U (en) | Image sensor for partially shielding phase focusing by shared micro-lens |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |