CN111854956A - Multispectral imaging system based on micro-lens array and image reconstruction method - Google Patents

Multispectral imaging system based on micro-lens array and image reconstruction method Download PDF

Info

Publication number
CN111854956A
CN111854956A CN202010738422.4A CN202010738422A CN111854956A CN 111854956 A CN111854956 A CN 111854956A CN 202010738422 A CN202010738422 A CN 202010738422A CN 111854956 A CN111854956 A CN 111854956A
Authority
CN
China
Prior art keywords
image
sub
array
linear gradient
lens array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010738422.4A
Other languages
Chinese (zh)
Inventor
吴衡
马建文
罗劭娟
陈梅云
赵艮平
程良伦
王涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202010738422.4A priority Critical patent/CN111854956A/en
Publication of CN111854956A publication Critical patent/CN111854956A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
    • G01J3/0208—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G01J2003/2826—Multispectral imaging, e.g. filter imaging

Abstract

The application discloses multispectral imaging system and image reconstruction method based on a micro-lens array, which can realize imaging of different continuous wave bands on a detected target object and divide the imaging into a plurality of sub-images by adopting the micro-lens array and a linear gradient filter, and the linear gradient filter is obliquely arranged at a certain angle relative to the micro-lens array so that an area array CCD detector obtains an image with high quality and high resolution, and can rapidly reconstruct the image with high quality by a maximum posterior estimation method. Meanwhile, the embodiment of the application can realize the image data acquisition of the detected target object only by adopting the area array CCD detector, the micro lens array and the linear gradient filter, and the linear gradient filter is convenient to install, reduces the hardware cost and has a compact structure.

Description

Multispectral imaging system based on micro-lens array and image reconstruction method
Technical Field
The application relates to the technical field of multispectral imaging, in particular to a multispectral imaging system based on a micro-lens array and a corresponding image reconstruction method.
Background
The spectral imaging technology is an important component in the technical field of optical imaging and detection. The spectral imaging technology and the optical imaging technology are integrated, and the spectral information and the spatial characteristic information of the measured object can be simultaneously obtained. The spectral imaging technology has the advantage of multispectral resolution, and can directly identify the target object from the space, so that the method has extremely high application value in the aspects of target identification, natural disaster early warning, agricultural product detection and the like.
The currently mainstream multispectral imaging technology includes dispersive type, interference type, and optical filter type. The dispersive multispectral imaging technology utilizes prism or grating light splitting and then imaging by an imaging system. The interference type multispectral imaging technology detects an interference pattern of a measured object by using an interferometer, and then obtains spectral information through Fourier transform calculation. The dispersive and interference multispectral imaging systems have complex structures and high requirements on hardware. The filter type multispectral imaging technology is characterized in that a filter is added into an imaging system for light splitting to generate light with various spectrums, and then imaging is carried out. Compared with a dispersion type and an interference type, the optical filter type multispectral imaging system is simple in structure, high in spatial resolution and spectral resolution and wide in application prospect in the actual civil and military fields. However, the existing optical filter type multispectral imaging system has the problems of poor imaging quality, low resolution, high hardware requirement and the like. Therefore, an optical filter type multispectral imaging system with compact structure, good imaging quality and low hardware cost is urgently needed to be developed.
Disclosure of Invention
The application provides a multispectral imaging system and an image reconstruction method based on a micro-lens array, which are used for solving the technical problems of poor imaging quality, high hardware cost and complex structure in the prior art.
In view of the above, the present application provides in a first aspect a multispectral imaging system based on a microlens array, comprising: the device comprises a calculation module, an area array CCD detector, a micro-lens array and a linear gradient filter;
the area array CCD detector, the micro lens array and the linear gradient filter are sequentially arranged along a light path, so that incident light rays sequentially pass through the linear gradient filter and the micro lens array to reach the area array CCD detector;
the center of a target surface of the area array CCD detector is arranged at the focus of the micro lens array, the linear gradient filter is arranged at a preset distance d relative to the micro lens array, and the linear gradient filter is rotated clockwise so as to be obliquely arranged relative to the micro lens array, and the size of the linear gradient filter is consistent with that of the micro lens array;
and the computing module is connected with the area array CCD detector and is used for reconstructing the image of the detected target object shot by the area array CCD detector based on a prestored maximum posterior estimation image reconstruction algorithm.
Preferably, the microlens array comprises a plurality of coplanar microlenses, the microlenses are uniformly distributed, and the microlenses are spherical lenses.
Preferably, the refractive index n of the microlens is 1.52mm, the diameter D of the microlens is 2.3mm, and the distance l between adjacent microlenses is 0.0625 mm.
Preferably, the focal length f of the microlens array is 6 mm.
Preferably, the linear graded filter is disposed at a preset distance d of 0.5mm with respect to the microlens array.
Preferably, the horizontal included angle of the linear graded filter with respect to the microlens array is 7.125 °.
The application also provides an image reconstruction method, and the multispectral imaging system based on the micro-lens array comprises the following steps:
s1: the emergent light of the target object to be detected is incident to the linear gradient filter, and the emergent light is divided into light rays with different wave bands through the linear gradient filter;
s2: light rays with different wave bands are obliquely incident into each micro lens of the micro lens array, the micro lens array is an m multiplied by n array, and then the light rays incident from each micro lens form images on a target surface of the area array CCD detector and form images of m multiplied by n detected target objects;
s3: and reconstructing the image of the detected target object acquired by the area array CCD detector through a computing module based on a prestored maximum posterior estimation image reconstruction algorithm.
Preferably, the step S3 specifically includes:
s301: dividing the images of m multiplied by n measured objects acquired by the area array CCD detector into corresponding sub-images, and recording each sub-image as SiM × n sub-images can be obtained. From this, a sequence of partial images S can be obtained, denoted by
S＝[S1,S2,...,Si,...,Sm*n]；
S302: selecting any sub-image at the most middle position of the sub-image sequence S as a target image SfAnd registering other sub-images with the target image to obtain a new registered sub-image sequence Q (Q ═ Q)1,Q2,...,Qi,Qf,...,Qm*n]Wherein Q isiFor the i-th registered sub-image S in the sub-image sequence Si，QfIs a target image Sf；
S303: by means of the sub-image sequence Q, a mathematical model is established of,
Figure BDA0002605944640000031
the formula is simplified to obtain Z ═ HX + N, wherein Z ═ QTZ is the reconstructed sub-image, H is the image transformation matrix, and X is the image to be reconstructedReconstructing an image, N being a noise vector;
s304: obtaining the target image by optimizing a conditional probability density function between X and Z, the conditional probability density function being
Figure BDA0002605944640000032
Wherein M is-1E is an autocorrelation matrix, T is a transpose of the matrix;
iterative computation is carried out on the formula to obtain
Figure BDA0002605944640000033
And obtaining a reconstructed image of the object to be detected.
According to the technical scheme, the embodiment of the application has the following advantages:
the embodiment of the application provides a multispectral imaging system and an image reconstruction method based on a micro-lens array, and the multispectral imaging system and the image reconstruction method can realize imaging of different continuous wave bands on a detected target object and divide the imaging into a plurality of sub-images by adopting the micro-lens array and a linear gradient filter, and the linear gradient filter is obliquely arranged at a certain angle relative to the micro-lens array, so that an area array CCD detector obtains an image with high quality and high resolution, and the image with high quality can be rapidly reconstructed by a maximum posterior estimation method. Meanwhile, the embodiment of the application can realize the image data acquisition of the detected target object only by adopting the area array CCD detector, the micro lens array and the linear gradient filter, and the linear gradient filter is convenient to install, reduces the hardware cost and has a compact structure.
Drawings
Fig. 1 is a schematic structural diagram of a multispectral imaging system based on a microlens array according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a microlens array of a multispectral imaging system based on a microlens array according to an embodiment of the present application;
fig. 3 is a flowchart of an image reconstruction method according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Example one
For easy understanding, referring to fig. 1, the present application provides a multispectral imaging system based on a microlens array, comprising: the device comprises a computing module 101, an area array CCD detector 102, a micro-lens array 103 and a linear gradient filter 104;
the area array CCD detector 102, the micro lens array 103 and the linear gradient filter 104 are sequentially arranged along a light path, so that incident light rays sequentially pass through the linear gradient filter 104 and the micro lens array 103 to reach the area array CCD detector 102;
the center of a target surface of the area array CCD detector 102 is arranged at the focus of the micro lens array 103, the linear gradient filter 104 is arranged at a preset distance d relative to the micro lens array 103, and the linear gradient filter 104 is rotated clockwise, so that the linear gradient filter 104 is obliquely arranged relative to the micro lens array 103, and the size of the linear gradient filter 104 is consistent with that of the micro lens array 103;
the calculating module 101 is connected to the area array CCD detector 102, and is configured to reconstruct an image of the target 105 to be detected, which is captured by the area array CCD detector 102, based on a pre-stored maximum posterior estimated image reconstruction algorithm.
It can be understood that the center of the target surface of the area array CCD detector 102 is located at the focus of the microlens array 103, and meanwhile, the microlens array 103 is aligned with the area array CCD detector 102 in both horizontal and vertical directions, so that the microlens array 103 can just image on the target surface of the area array CCD detector 102, and the imaging resolution and quality are improved.
It should be noted that the pixel size of the area array CCD detector 102 is 3840 × 5120, and the preset distance d of the linear graded filter 104 relative to the microlens array 103 satisfies 0 < d < 1mm, so that the microlens array 103 can fully absorb the light incident from the linear graded filter 104, reduce the light energy loss, and improve the image quality; in addition, the horizontal included angle between the linear graded filter 104 and the microlens array 103 controls the spectral resolution of the imaging system, and the spectral resolution of the imaging system can be affected by setting the size of the horizontal included angle, and the horizontal included angle in this embodiment is 7.125 °, which makes the spectral resolution relatively high and the imaging effect better.
Further, referring to fig. 2, the microlens array 103 includes a plurality of microlenses 113 coplanar, the plurality of microlenses 113 are uniformly distributed, and the microlenses 113 are spherical lenses.
The plurality of microlenses 113 are identical.
Further, the refractive index n of the microlens 113 is 1.52mm, the diameter D of the microlens 113 is 2.3mm, and the pitch l between adjacent microlenses 113 is 0.0625 mm.
Further, the focal length f of the microlens array 103 is 6 mm.
Further, the linear graded filter 104 is disposed at a preset distance d of 0.5mm with respect to the microlens array 103.
Example two
The second embodiment provides an image reconstruction method, which is based on the multispectral imaging system based on the microlens array of the first embodiment, and referring to fig. 3, the method includes the following steps:
s1: the emergent light of the measured target object is incident to the linear gradient filter, and the emergent light is divided into light rays with different wave bands through the linear gradient filter;
s2: the light rays with different wave bands are obliquely incident into each micro lens of the micro lens array, the micro lens array is an m multiplied by n array, and then the light rays incident from each micro lens form images on the target surface of the area array CCD detector and form images of m multiplied by n detected target objects;
s3: and reconstructing the image of the detected target object acquired by the area array CCD detector through a computing module based on a prestored maximum posterior estimation image reconstruction algorithm.
Further, step S3 specifically includes:
s301: dividing the images of m × n measured objects acquired by an area array CCD detector into corresponding sub-images, and recording each sub-image as SiM × n sub-images can be obtained. From this, a sequence of partial images S can be obtained, denoted by
S＝[S1,S2,...,Si,...,Sm*n]；
It should be noted that the sub-image sequence S may be formed by sequentially arranging sub-images at any one corner of the microlens array from the beginning to the sub-images at the opposite corner.
S302: selecting any sub-image at the most middle position of the sub-image sequence S as a target image SfAnd registering other sub-images with the target image to obtain a new registered sub-image sequence Q (Q)1,Q2,...,Qi,Qf,...,Qm*n]Wherein Q isiFor the i-th registered sub-image S in the sequence Si，QfIs a target image Sf；
Note that the target image SfIs any sub-image at the most middle position of the sub-image sequence S, and when two sub-images are arranged at the most middle position, any one sub-image can be selected as the target image SfFor example, when the sequence of sub-images is S ═ S1,S2,...,Si,...,S80]Then, S can be selected40Or S41The remaining 79 sub-images are combined with the target image S as the target image40Or S41Carrying out registration to obtain a new sub-image sequence Q ═ Q1,Q2,...,Qi,...,Q80]。
S303: by means of the sub-image sequence Q, a mathematical model is established of,
Figure BDA0002605944640000061
the formula is simplified to obtain Z ═ HX + N, wherein Z ═ QTZ is a reconstructed sub-image, H is an image transformation matrix, X is an image to be reconstructed, and N is a noise vector;
s304: obtaining the target image by optimizing a conditional probability density function between X and Z, the conditional probability density function being
Figure BDA0002605944640000062
Wherein M is-1E is an autocorrelation matrix, T is a transpose of the matrix;
iterative computation is carried out on the formula to obtain
Figure BDA0002605944640000063
And obtaining a reconstructed image of the object to be detected.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (8)

1. A multi-spectral imaging system based on a microlens array, comprising: the device comprises a calculation module, an area array CCD detector, a micro-lens array and a linear gradient filter;
the area array CCD detector, the micro lens array and the linear gradient filter are sequentially arranged along a light path, so that incident light rays sequentially pass through the linear gradient filter and the micro lens array to reach the area array CCD detector;
the center of a target surface of the area array CCD detector is arranged at the focus of the micro lens array, the linear gradient filter is arranged at a preset distance d relative to the micro lens array, and the linear gradient filter is rotated clockwise so as to be obliquely arranged relative to the micro lens array, and the size of the linear gradient filter is consistent with that of the micro lens array;
and the computing module is connected with the area array CCD detector and is used for reconstructing the image of the detected target object shot by the area array CCD detector based on a prestored maximum posterior estimation image reconstruction algorithm.
2. The microlens array based multispectral imaging system as recited in claim 1, wherein the microlens array comprises a plurality of microlenses that are coplanar, the plurality of microlenses being uniformly distributed, the microlenses being spherical lenses.
3. The microlens array based multispectral imaging system as recited in claim 2, wherein the refractive index n of the microlenses is 1.52mm, the diameter D of the microlenses is 2.3mm, and the spacing l between adjacent microlenses is 0.0625 mm.
4. The microlens array based multispectral imaging system as recited in claim 1, wherein the focal length f of the microlens array is 6 mm.
5. The microlens array based multispectral imaging system as recited in claim 1, wherein the linear graded filter is disposed at a predetermined distance d of 0.5mm from the microlens array.
6. The microlens array based multispectral imaging system as recited in claim 1, wherein the linear graded filter is horizontally angled at 7.125 ° with respect to the microlens array.
7. An image reconstruction method based on the multispectral imaging system based on the microlens array as claimed in any one of claims 1 to 6, comprising the following steps:
s1: the emergent light of the target object to be detected is incident to the linear gradient filter, and the emergent light is divided into light rays with different wave bands through the linear gradient filter;
s2: light rays with different wave bands are obliquely incident into each micro lens of the micro lens array, the micro lens array is an m multiplied by n array, and then the light rays incident from each micro lens form images on a target surface of the area array CCD detector and form images of m multiplied by n detected target objects;
s3: and reconstructing the image of the detected target object acquired by the area array CCD detector through a computing module based on a prestored maximum posterior estimation image reconstruction algorithm.
8. The image reconstruction method according to claim 7, wherein the step S3 specifically includes:
s301: dividing the images of m multiplied by n measured objects acquired by the area array CCD detector into corresponding sub-images, and recording each sub-image as SiM × n sub-images can be obtained. From this, a sequence of partial images S can be obtained, denoted by
S＝[S1,S2,...,Si,...,Sm*n]；
S302: selecting any sub-image at the most middle position of the sub-image sequence S as a target image SfAnd connecting the other sub-images with the target imageRegistering the images to obtain a new registered sub-image sequence Q ═ Q [ < Q >1,Q2,...,Qi,Qf,...,Qm*n]Wherein Q isiFor the i-th registered sub-image S in the sub-image sequence Si，QfIs a target image Sf；
S303: by means of the sub-image sequence Q, a mathematical model is established of,
Figure FDA0002605944630000021
the formula is simplified to obtain Z ═ HX + N, wherein Z ═ QTZ is a reconstructed sub-image, H is an image transformation matrix, X is an image to be reconstructed, and N is a noise vector;
s304: obtaining the target image by optimizing a conditional probability density function between X and Z, the conditional probability density function being
Figure FDA0002605944630000022
Wherein M is-1E is an autocorrelation matrix, T is a transpose of the matrix;
iterative computation is carried out on the formula to obtain
Figure FDA0002605944630000023
And obtaining a reconstructed image of the object to be detected.
CN202010738422.4A 2020-07-28 2020-07-28 Multispectral imaging system based on micro-lens array and image reconstruction method Pending CN111854956A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010738422.4A CN111854956A (en) 2020-07-28 2020-07-28 Multispectral imaging system based on micro-lens array and image reconstruction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010738422.4A CN111854956A (en) 2020-07-28 2020-07-28 Multispectral imaging system based on micro-lens array and image reconstruction method

Publications (1)

Publication Number Publication Date
CN111854956A true CN111854956A (en) 2020-10-30

Family

ID=72947682

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010738422.4A Pending CN111854956A (en) 2020-07-28 2020-07-28 Multispectral imaging system based on micro-lens array and image reconstruction method

Country Status (1)

Country Link
CN (1) CN111854956A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112924029A (en) * 2021-01-26 2021-06-08 中国科学院空天信息创新研究院 Multispectral camera manufacturing method and multispectral camera system

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112924029A (en) * 2021-01-26 2021-06-08 中国科学院空天信息创新研究院 Multispectral camera manufacturing method and multispectral camera system

Similar Documents

Publication Publication Date Title
CN106803892B (en) A kind of light field high-resolution imaging method based on Optical field measurement
US8432479B2 (en) Range measurement using a zoom camera
CN104168429B (en) A kind of multiple aperture subrane high resolution imaging apparatus and its imaging method
CN104457708B (en) A kind of compact multispectral camera
US8773550B2 (en) Range measurement using multiple coded apertures
US9473700B2 (en) Camera systems and methods for gigapixel computational imaging
CN106872037B (en) Fast illuminated compact optical field imaging full-polarization spectrum detection device and method
US10645281B1 (en) Method and system for snapshot multi-spectral light field imaging
Genser et al. Camera array for multi-spectral imaging
Badham et al. Photonic integrated circuit-based imaging system for SPIDER
Oktem et al. High-resolution multi-spectral imaging with diffractive lenses and learned reconstruction
CN111201783A (en) Imaging apparatus and method, and image processing apparatus and method
CN111854956A (en) Multispectral imaging system based on micro-lens array and image reconstruction method
Horstmeyer et al. Modified light field architecture for reconfigurable multimode imaging
CN112611455B (en) Multi-angle and multi-spectral frequency coding imaging method and device
US10446600B2 (en) Imaging system and imaging device having a random optical filter array
CN105444888A (en) Chromatic aberration compensation method of hyperspectral imaging system
US20200348175A1 (en) Wide-angle computational imaging spectroscopy method and apparatus
CN106791355B (en) Single pixel imaging method and system based on image frequency domain splicing
CN113645422A (en) Image sensor with a plurality of pixels
CN110307781B (en) Interferometric imaging system
CN103558160A (en) Method and system for improving resolution ratio of spectral imaging space
CN104142131B (en) Phase imaging system
Oberdörster et al. Correcting distortion and braiding of micro-images from multi-aperture imaging systems
KR20200038835A (en) Image sensor and method to sense image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20201030