CN113658069A - Hyperspectral microscopic image flat field correction method and system based on common flat field extraction - Google Patents

Hyperspectral microscopic image flat field correction method and system based on common flat field extraction Download PDF

Info

Publication number
CN113658069A
CN113658069A CN202110930502.4A CN202110930502A CN113658069A CN 113658069 A CN113658069 A CN 113658069A CN 202110930502 A CN202110930502 A CN 202110930502A CN 113658069 A CN113658069 A CN 113658069A
Authority
CN
China
Prior art keywords
hyperspectral
flat field
common
matrix
correction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110930502.4A
Other languages
Chinese (zh)
Other versions
CN113658069B (en
Inventor
谷延锋
王煜坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN202110930502.4A priority Critical patent/CN113658069B/en
Publication of CN113658069A publication Critical patent/CN113658069A/en
Application granted granted Critical
Publication of CN113658069B publication Critical patent/CN113658069B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

A hyperspectral microscopic image flat field correction method and system based on common flat field extraction relates to the field of image flat field correction. The invention aims to solve the problem that the common illumination level field item cannot be extracted at present and is used for correcting a hyperspectral microscope system, so that the problem of uneven illumination of a cancer cell tissue hyperspectral microscope image is solved, and the problem of low tissue pathological classification precision of cancer cells is caused. The method comprises the following steps: collecting hyperspectral microscopic images, and dividing the images into a training set and a test set; training the CFE model to obtain a trained CFE model; solving the trained CFE model to obtain a common flat field and a corrected hyperspectral microscopic image; scoring the common flat field by using the test set, and obtaining the common rank of the flat field items according to the scoring to obtain an optimal CFE model; and inputting the hyperspectral microscopic image to be detected into the optimal CFE model to obtain the hyperspectral microscopic image after flat field correction. The method is used for flat field correction of cancer cell tissue hyperspectral microscopic images.

Description

Hyperspectral microscopic image flat field correction method and system based on common flat field extraction
Technical Field
The invention belongs to the field of image flat field correction, and particularly relates to a hyperspectral microscopic image flat field correction method and system based on common flat field extraction.
Background
At today's high medical level, cancer is a leading cause of death in humans. Histopathological observations under the microscope are "gold standards" for determining the specific type of cancer and subsequent treatment regimens. Besides the change of spatial morphology, biochemical components of the cancerated cells are also changed, so that spectral information of the cells, such as absorption peaks and spectral envelope changes, are caused. Therefore, the classification of the tissue pathology of cancer cells is now the focus of research in this field.
At present, the hyperspectral microimaging can obviously improve the tissue pathological classification precision aiming at cancer cells, but due to the reasons of uneven light source illumination of a microscope system, interference of detector sensitivity, microscope objective dispersion, coupling error of an imager and a microscope and the like, the illumination of the obtained hyperspectral microimages is often uneven, so that the gray values of the same cells at different spatial positions are different, and the imaging quality and subsequent evaluation are influenced. At present, in order to overcome the phenomenon of uneven illumination in a microscope system, black and white balance data under illumination similar to actual measurement data needs to be collected to correct the illumination. However, the conditions that ensure the same illumination conditions are very harsh; the white balance data is generally measured for multiple times and an average value is taken, so that the influence of bubbles and dust in the slice is reduced; and for special microscope systems, the white balance data cannot be acquired through actually measured data. Therefore, under the condition of not collecting white balance data, it is very important to extract common uneven light level field information from a plurality of hyperspectral microscopic data actually measured under similar lighting conditions, but at present, a common light level field item cannot be extracted to be used for correcting a hyperspectral microscopic system, so that the existing uneven light phenomenon is solved, and the current histopathological classification precision of cancer cells is low.
Disclosure of Invention
The invention aims to solve the problem that a common illumination flat field item in a hyperspectral microscope system cannot be extracted at present, so that the problem of uneven illumination of a cancer cell tissue hyperspectral microscope image is solved, and the problem of low tissue pathological classification precision of cancer cells is caused.
The specific process of the hyperspectral microscopic image flat field correction method based on common flat field extraction comprises the following steps:
collecting a plurality of hyperspectral microscopic images under the same illumination condition, and dividing the images into a training set and a test set;
the illumination conditions include: the light source intensity of the microscope, the position of a color filter, the position of a field diaphragm of a condenser, the position of an aperture diaphragm of the condenser, the position of a light path conversion rod of a trinocular interface and the integration time of a spectral imager;
the test set is a set consisting of a plurality of pairs of hyperspectral microscopic images with partially overlapped spaces;
the training set is a set of hyperspectral microimages except the test set;
step two, establishing a CFE model, and training the CFE model by using a training set to obtain the trained CFE model, wherein the method comprises the following steps:
secondly, rearranging the spatial dimension of the training set obtained in the first step to generate a space-spectrum two-dimensional hyperspectral matrix:
secondly, carrying out logarithm operation on each element in the generated space-spectrum two-dimensional hyperspectral matrix to obtain a hyperspectral logarithm matrix;
step two, obtaining a trained CFE model under low-rank constraint by using the hyperspectral logarithmic matrix obtained in the step two;
step three, solving the trained CFE model obtained in the step two by using a common orthogonal base extraction method to obtain a common flat field and a corrected hyperspectral microscopic image;
step four, scoring the common flat field obtained in the step three by using a test set, and obtaining the common rank of a flat field item according to the score, so as to obtain a trained optimal CFE model;
and fifthly, inputting the hyperspectral microscopic image to be detected into the trained optimal CFE model to obtain the hyperspectral microscopic image after flat field correction.
The hyperspectral microimage flat field correction system based on common flat field extraction is used for realizing a hyperspectral microimage flat field correction method based on common flat field extraction.
The invention has the beneficial effects that:
the invention provides a CFE (Common Flat-field Extraction, CFE for short) model, which effectively corrects uneven illumination in a hyperspectral microscopy system by low-rank Common Flat-field Extraction on the premise of not needing actually measured white balance data, generates a cancer cell histopathology hyperspectral microscopic image with Flat, improves the precision of cancer cell histopathology classification, and brings a large amount of precisely available hyperspectral microscopic image resources for histopathology application.
Drawings
FIG. 1 is a flow chart of a CFE method implementation;
FIG. 2 is a graph of the effect of an uneven illumination phenomenon on a hyperspectral microimage;
FIG. 3 shows the flat field terms extracted by CFE and various comparison methods;
FIG. 4 is a visual assessment of CFE versus various comparison methods;
FIG. 5 is a graph of CFE correction scores versus various comparison methods;
fig. 6 shows the relationship between the common rank C and the correction score.
Detailed Description
The first embodiment is as follows: the specific process of the hyperspectral microscopic image flat field correction method based on common flat field extraction in the embodiment comprises the following steps:
collecting a plurality of hyperspectral microscopic images under the same illumination condition, and dividing the hyperspectral microscopic images into a training set and a testing set;
the illumination conditions include: the light source intensity of the microscope, the position of a color filter, the position of a field diaphragm of a condenser, the position of an aperture diaphragm of the condenser, the position of a light path conversion rod of a trinocular interface, the integration time of a spectral imager and the like;
the classification into a training set and a test set is classified according to evaluation requirements;
the evaluation requirements are as follows: since the objective of flat field correction is to obtain an image with uniform illumination, if the spatial position information of the hyperspectral microscopic image after correction under the same illumination condition is completely the same, the gray scale information of each spectrum section after correction should be the same theoretically. The maximum manifestation of the uneven illumination phenomenon is Vignetting (Vignetting), which is embodied as a dark-angle-shaped template (as shown in fig. 2) radiating from the center to the periphery, so that the difference between the center gray scale and the corner gray scale of a general measured image is most obvious. In response to this phenomenon, a pair of spatially partially overlapped images is generally adopted in the test set acquisition process, and the overlapped part is about 1/4 of the whole image, namely the lower right corner of the previous image and the upper left corner of the next image are overlapped with each other, so as to compare the parts with the most obvious illumination difference. In the corrected pair of test set data, the smaller the gray difference of the overlapped part is, the more effective the correction method is.
Step two, establishing a CFE model, and training the CFE (Common Flat-field Extraction, CFE for short) model by utilizing a training set to obtain the trained CFE model, wherein the method comprises the following steps:
secondly, rearranging the spatial dimension of the training set obtained in the first step to generate a space-spectrum two-dimensional hyperspectral matrix, wherein the specific process comprises the following steps:
step two, acquiring an illumination process of the hyperspectral microscope system:
the illumination process of the hyperspectral microscope system can be described by a linear system model:
Figure BDA0003211144410000031
wherein I (x, y, lambda) represents a hyperspectral microimage,
Figure BDA0003211144410000032
the corrected hyperspectral microscopic image without vignetting influence is represented, F (x, y, lambda) represents the linear attenuation of a hyperspectral microscopic system, namely a flat field item, Z (x, y, lambda) represents the additive offset of the system, actually corresponds to the noise (dark current and other light source interference) of the image when the microscopic system is not illuminated by a light source, namely a dark field item, and (x, y, lambda) represents the gray value of the lambda wave band at the (x, y) th pixel in the image space;
Figure BDA0003211144410000034
representing the Schur-Hadamard product, i.e. multiplication of corresponding elements of the matrix;
compared with fluorescence and phase difference microscopy systems, a conventional bright field microscopy system usually ignores a dark field item in bright field microscopic imaging because of strong illumination light source and extremely weak dark current effect, and the illumination process of a hyperspectral microscopy system can be expressed as follows:
Figure BDA0003211144410000033
and secondly, rearranging two-dimensional gray level images corresponding to each wave band of the hyperspectral image I (x, y, lambda) into one-dimensional vectors to generate a space-spectrum two-dimensional hyperspectral matrix I (m, lambda), wherein m represents a rearranged space dimension index, and lambda represents a wave band.
Secondly, carrying out logarithm operation (represented by italics) on each element in the generated space-spectrum two-dimensional high spectrum matrix to enable a common flat field to be in an additive relation with the corrected image, and thus the high spectrum logarithm matrix can be represented as:
Figure BDA0003211144410000041
wherein F (m, λ) represents a logarithmic consensus flat field,
Figure BDA0003211144410000042
and (3) representing the corrected hyperspectral logarithmic image, wherein N is the total number of the test samples in the test set, N is 1 and …, and N represents the nth sample in the training set.
Step two, acquiring a well-trained CFE model under low-rank constraint by using the hyperspectral logarithmic matrix acquired in the step two:
step two, three and one, acquiring each hyperspectral logarithm matrix InThe expression of (m, lambda) is as follows: for matrix with rank R
Figure BDA0003211144410000043
There are full rank matrices
Figure BDA0003211144410000044
Figure BDA0003211144410000045
Such that:
X＝PQT
so each hyperspectral logarithm matrix In(m, λ) can be represented as
Figure BDA0003211144410000046
Wherein the content of the first and second substances,
Figure BDA0003211144410000047
Figure BDA0003211144410000048
is a hyperspectral logarithmic matrix InA low rank representation of (m, λ),
Figure BDA0003211144410000049
represents a common orthogonal group, and represents a common orthogonal group,
Figure BDA00032111444100000410
and
Figure BDA00032111444100000411
a low rank representation of the composition of the common flat field F (m, λ),
Figure BDA00032111444100000412
representing a common orthogonal basis coefficient matrix;
Figure BDA00032111444100000413
Figure BDA00032111444100000414
is a corrected hyperspectral logarithmic image
Figure BDA00032111444100000415
Low rank representation of (a); m and Λ respectively represent the total number of pixels and the number of spectral bands of the hyperspectral logarithmic matrix; c represents the rank of the flat field, RnRank of the Hyperspectral logarithmic matrix after each correction, ()TIndicating transposition.
Step two, three and two, and each hyperspectral logarithm matrix I obtained according to step two, three and onenThe representation of (m, λ) yields a CFE model trained under low rank constraints:
Figure BDA00032111444100000416
Figure BDA00032111444100000417
Figure BDA00032111444100000418
wherein N represents the total number of hyperspectral images in the training set; i | · | purple windFRepresents the Frobenius norm; e denotes an identity matrix and subscripts denote the matrix size.
Step three, solving the trained CFE model obtained in the step two by using a Common Orthogonal Basis Extraction (COBE) method to obtain a Common flat field and a corrected hyperspectral microscopic image, and the method comprises the following steps:
step three, step one, utilizing the common orthogonal base for obtaining the well-trained CFE model
Figure BDA00032111444100000419
Step three one to one, to step two three one acquired hyperspectral logarithm matrix InAnd (m, lambda) reducing the dimension, wherein the hyperspectral logarithmic matrix after dimension reduction is as follows:
Figure BDA0003211144410000051
step three, two pairs of common orthogonal bases
Figure BDA0003211144410000052
Was initialized to obtain a common standard base:
Figure BDA0003211144410000053
Figure BDA0003211144410000054
wherein the content of the first and second substances,
Figure BDA0003211144410000055
M-P inverse, U, of the representation matrixnAnd WnIs a hyperspectral logarithmic matrix In' (m, lambda) QR decomposition, UnIs a unitary matrix;
Figure BDA0003211144410000056
represents UnThe coefficient matrix of (2).
For any ni,nj＝1,....,N，ni≠njThe method comprises the following steps:
Figure BDA0003211144410000057
Figure BDA0003211144410000058
wherein
Figure BDA0003211144410000059
And
Figure BDA00032111444100000510
represents ZnAnd
Figure BDA00032111444100000511
column l.
From this, it is found that, in the range of l 1, …, C,
Figure BDA00032111444100000512
together forming an orthogonal basis
Figure BDA00032111444100000513
Wherein the content of the first and second substances,
Figure BDA00032111444100000514
the orthogonal basis coefficient matrix corresponding to the nth training sample in the training set is calculated as follows:
Figure BDA00032111444100000515
wherein, (.)-1Representing the inverse of the matrix;
step three, one and three, iterative solution of the first substrate of the common standard base by using an alternating least square method
Figure BDA00032111444100000516
Figure BDA00032111444100000517
Step three-four, in the range of l 1, …, C-1, ensuring orthogonality with the other l bases, find the l +1 th base:
Figure BDA00032111444100000518
wherein, (.)lRepresenting the corresponding variable after the first iteration;
step three-one-five, and step three-one-three-four are executed until the convergence condition is met, and finally the method is obtained
Figure BDA0003211144410000061
Are common orthogonal groups.
The convergence condition is fl+1Epsilon, indicates that no orthogonal basis exists within the error range, and epsilon indicates the minimum allowable error;
step two, obtaining a common flat field and a corrected hyperspectral microscopic image by using the common orthogonal base obtained in the step one:
step three, step two, obtain the coefficient matrix of the common orthogonal base:
orthogonal basis coefficient matrix obtained in three or two inspection steps
Figure BDA0003211144410000062
Difference between them
Figure BDA0003211144410000063
And obtaining the average value as the coefficient matrix of the common orthogonal base
Figure BDA0003211144410000064
For each set of coefficient matrices in the training set
Figure BDA0003211144410000065
Can both ensure
Figure BDA0003211144410000066
Sufficiently small that a coefficient matrix of common orthogonal bases can be obtained
Figure BDA0003211144410000067
Step three and two, obtaining a common flat field item F (x, y, lambda) according to the common orthogonal base obtained in the step three and the coefficient matrix obtained in the step three and two;
Figure BDA0003211144410000068
and step three, subtracting the high spectral logarithm matrix and the flat field item, and combining the index and rearrangement operation to obtain a corrected nth high spectral microscopic image:
Figure BDA0003211144410000069
step four, scoring the common flat field obtained in the step three by using a test set, and obtaining the common rank of a flat field item according to the score, so as to obtain an optimal CFE model, wherein the method comprises the following steps:
amplifying the overlapped parts in the data of a pair of test sets with 10 times of spatial resolution, selecting a wave band with uniform illumination (640.6 nm is selected in the experiment), and selecting the spatial overlapped parts of each pair of data in the test sets by adopting a sub-pixel registration method;
and step two, when the correction score is calculated, the gamma value can generate large-range change due to the dynamic range of the corrected hyperspectral image, so that when the advantages of all correction methods are compared, the dynamic ranges of the hyperspectral microscopic images obtained by different flat-field correction methods are consistent, and the obtained correction score is meaningful. The invention uniformly matches the dynamic range of the hyperspectral microscopic image after flat field correction with the hyperspectral microscopic image without flat field correction, and concretely comprises the following steps:
Figure BDA0003211144410000071
where med (-) represents the median of the image, var (-) represents the standard deviation of the image, IcorrRepresenting the corrected image, ImeasRepresenting an uncorrected image.
Step four, acquiring a correction scoring formula of the space overlapping part of each pair of data in the test set, and substituting the matching result into the correction scoring formula to acquire a correction score:
calculate a correction score for the spatially overlapping portion of each pair of data in the test set:
Figure BDA0003211144410000072
wherein In',1(x, y, lambda) and In',2(x, y, λ) represents the spatially overlapping portions of the hyperspectral microscopy images in the nth' pair of test sets. Γ () range is [0, ∞), the smaller the correction score in the range, the more excellent the correction method, N '∈ [1, N']And N' is the sum of the hyperspectral microimages in the test setLogarithm.
The criteria for the correction scores were: gamma (·) ═ 0 shows that the gray scale of the overlapped part of the corrected hyperspectral microscopic image is completely consistent, and the correction result is perfect; gamma (·) < 1 is more than 0, which means that the gray level difference between the overlapped parts of the hyperspectral microscopy images after correction is smaller than the deviation before correction, and the correction is considered to be effective; gamma (·) is more than or equal to 1, the gray difference between the overlapped parts of the hyperspectral microscopic images after correction is unchanged or larger than the deviation before correction, and the correction is considered to be invalid.
Step four, determining a common rank C according to the specific correction score minimum value:
the convergence criterion of CFE is fl+1ε, according to the experiment, the value of ε is [0.01,0.5 ]]And testing the relation between epsilon and C within the range, and finding the closest epsilon to be 0.3 under the condition that C is less than or equal to 10, wherein C can be confirmed to be 6 according to the corresponding relation of the third step and the second step. The above experiment shows that the common rank C of the flat field terms is not greater than 6. Therefore, the rank of the flat field term is determined according to the minimum value of the specific correction score in the range of C ═ 1.·,6, so as to obtain the trained optimal CFE model (as fig. 6 shows the relationship between the common rank C and the correction score).
And fifthly, inputting the hyperspectral microscopic image to be detected into the trained optimal CFE model to obtain the hyperspectral microscopic image after flat field correction.
The second embodiment is as follows: the hyperspectral microimage flat field correction system based on common flat field extraction is used for realizing the hyperspectral microimage flat field correction method based on common flat field extraction in the specific embodiment.
Example (b):
the following examples were used to demonstrate the beneficial effects of the present invention:
the hyperspectral microscopy system consists of a microscope (Creative U1, Prosumer, China), a push-scan hyperspectral imager (SOC710VP, Surface Optics, USA) and a computer, and hyperspectral microscopy software of the Surface Optics is used for collecting hyperspectral microscopy data. The collection waveband of the hyperspectral imager is 400-1000 nm, the spatial resolution of the collected image is 696 multiplied by 520, the cut-off frequency of the hyperspectral microscope system is 700nm because the existing microscope does not adjust the light path aiming at the near infrared waveband, and 90 wavebands (431.4-656 nm) are reserved in the data. 20 hematoxylin-eosin (H & E) stained lung cancer pathological tissue slices are selected in the experiment, and hyperspectral microimaging (20 x objective magnification) is performed on the slices under the same illumination condition to obtain 70 hyperspectral microimages as a training set; meanwhile, according to evaluation indexes, 20 pairs of hyperspectral images with 1/4 overlapped visual fields are collected to serve as a test set. The number of training sets was controlled to be 10,20, …,70 and the number of test sets was constant to be 20 in the experiment. CIDRE, BaSiC and CellProfiler are selected as comparison methods of CFE, and FIG. 3 is a flat field term extracted by the method and various comparison methods. The evaluation index is a correction score.
TABLE 1 comparison of CFE to correction score for comparative methods
Figure BDA0003211144410000081
As can be seen from the qualitative observation shown in fig. 4 and the quantitative evaluation shown in table 1 and fig. 5, the hyperspectral microscopic image flat field correction model and method based on low-rank common flat field extraction in the present example have a small and stable correction score, and the splicing edge of the images after correction is most unobvious, which illustrates the accuracy of the flat field correction method and can effectively correct the uneven illumination phenomenon of the hyperspectral microscopic images.
To sum up, in order to correct the uneven illumination phenomenon of the hyperspectral microscopy image without the help of actually measured black and white balance data, the embodiment provides a hyperspectral microscopy image flat field correction model and a hyperspectral microscopy image flat field correction method based on low-rank common flat field extraction, which can effectively correct uneven illumination in a hyperspectral microscopy system and generate a flat hyperspectral microscopy image.

Claims (10)

1. The hyperspectral microscopic image flat field correction method based on common flat field extraction is characterized by comprising the following specific processes:
collecting a plurality of hyperspectral microscopic images under the same illumination condition, and dividing the images into a training set and a test set;
the illumination conditions include: the light source intensity of the microscope, the position of a color filter, the position of a field diaphragm of a condenser, the position of an aperture diaphragm of the condenser, the position of a light path conversion rod of a trinocular interface and the integration time of a spectral imager;
the test set is a set consisting of a plurality of pairs of hyperspectral microscopic images with partially overlapped spaces;
the training set is a set of hyperspectral microimages except the test set;
step two, establishing a CFE model and training the CFE model by utilizing a training set to obtain the trained CFE model, wherein the method comprises the following steps:
secondly, rearranging the space dimension of the training set obtained in the first step to generate a space-spectrum two-dimensional hyperspectral matrix:
secondly, carrying out logarithm operation on each element in the generated space-spectrum two-dimensional hyperspectral matrix to obtain a hyperspectral logarithm matrix;
step two, obtaining a trained CFE model under low-rank constraint by using the hyperspectral logarithmic matrix obtained in the step two;
step three, solving the trained CFE model obtained in the step two by using a common orthogonal base extraction method to obtain a common flat field and a corrected hyperspectral microscopic image;
step four, scoring the common flat field obtained in the step three by using a test set, and obtaining the common rank of a flat field item according to the score, so as to obtain a trained optimal CFE model;
and fifthly, inputting the hyperspectral microscopic image to be detected into the trained optimal CFE model to obtain the hyperspectral microscopic image after flat field correction.
2. The hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 1, characterized in that: in the second step, the spatial dimension of the training set obtained in the first step is rearranged to generate a space-spectrum two-dimensional hyperspectral matrix, and the method comprises the following steps:
step two, acquiring an illumination process of the hyperspectral microscope system;
the illumination process of the hyperspectral system is described by a linear system model as follows:
Figure FDA0003211144400000011
wherein I (x, y, lambda) represents a hyperspectral microimage,
Figure FDA0003211144400000012
representing a corrected hyperspectral microscopic image without vignetting influence, wherein F (x, y, lambda) represents the linear attenuation of a hyperspectral microscopic system, namely a flat field term, and (x, y, lambda) represents the gray value of a lambda wave band at the (x, y) th pixel in an image space;
Figure FDA0003211144400000013
represents the Schur-Hadamard product;
secondly, rearranging two-dimensional gray level images corresponding to each wave band of the hyperspectral microscopic image into one-dimensional vectors to generate a space-spectrum two-dimensional hyperspectral matrix I (m, lambda); m denotes a rearranged spatial dimension index, and λ denotes a band.
3. The hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 2, characterized in that: in the second step, logarithm operation is performed on each element in the generated space-spectrum two-dimensional hyperspectral matrix to obtain a hyperspectral logarithm matrix, which is as follows:
Figure FDA0003211144400000021
wherein F (m, λ) represents a logarithmic consensus flat field,
Figure FDA0003211144400000022
and (3) representing the corrected hyperspectral logarithm matrix, wherein N is the total number of the test samples in the test set, N is 1, …, and N represents the nth sample in the training set.
4. The hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 3, characterized in that: in the second step three, the trained CFE model under the low-rank constraint is obtained by using the hyperspectral logarithmic matrix obtained in the second step two, and the method comprises the following steps:
step two, three and one, for matrix with rank R
Figure FDA0003211144400000023
Each hyperspectral logarithm matrix In(m, λ) is represented by the following formula:
Figure FDA0003211144400000024
wherein the content of the first and second substances,
Figure FDA0003211144400000025
is a matrix of rank R
Figure FDA0003211144400000026
The full rank of (c) is not greater than,
Figure FDA0003211144400000027
is a hyperspectral logarithmic matrix InA low rank representation of (m, λ),
Figure FDA0003211144400000028
represents a common orthogonal group, and represents a common orthogonal group,
Figure FDA0003211144400000029
and
Figure FDA00032111444000000210
a low rank representation of the composition of the common flat field F (m, λ),
Figure FDA00032111444000000211
representing a common orthogonal basis coefficient matrix;
Figure FDA00032111444000000212
is a corrected hyperspectral logarithmic matrix
Figure FDA00032111444000000213
Low rank representation of (a); m and Λ respectively represent the total number of pixels and the number of spectral bands of the hyperspectral logarithmic matrix; c represents the rank of the flat field, RnRank of the Hyperspectral logarithmic matrix after each correction, ()TRepresenting a transpose;
wherein X is PQT；
Step two, three and two, and each hyperspectral logarithm matrix I obtained according to step two, three and onenThe representation of (m, λ) yields a CFE model trained under low rank constraints:
Figure FDA00032111444000000214
Figure FDA00032111444000000215
Figure FDA00032111444000000216
wherein N represents the total number of hyperspectral microimages in the training set; i | · | purple windFRepresents the Frobenius norm; e denotes an identity matrix and subscripts denote the matrix size.
5. The hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 4, characterized in that: in the third step, the trained CFE model obtained in the second step is solved by using a common orthogonal base extraction method to obtain a common flat field and a corrected hyperspectral microscopic image, and the method comprises the following steps:
step three, step one, obtaining common orthogonal base of the well-trained CFE model
Figure FDA0003211144400000031
The method comprises the following steps:
step three one to one, to step two three one acquired hyperspectral logarithm matrix InAnd (m, lambda) reducing the dimension to obtain a hyperspectral logarithm matrix after dimension reduction:
Figure FDA0003211144400000032
wherein the content of the first and second substances,
Figure FDA0003211144400000033
is an orthogonal basis coefficient matrix of the nth training sample in the training set;
step three, two pairs of common orthogonal bases
Figure FDA0003211144400000034
Was initialized to obtain a common standard base:
Figure FDA0003211144400000035
Figure FDA0003211144400000036
wherein the content of the first and second substances,
Figure FDA0003211144400000037
M-P inverse, U, of the representation matrixnAnd WnIs a hyperspectral logarithmic matrix In' (m, lambda) QR decomposition, UnIs a unitary matrix;
Figure FDA0003211144400000038
represents UnA coefficient matrix of (a);
for any ni,nj＝1,....,N，ni≠njThe common criteria are as follows:
Figure FDA0003211144400000039
Figure FDA00032111444000000310
wherein z isn,lAnd
Figure FDA00032111444000000311
represents ZnAnd
Figure FDA00032111444000000312
column l;
step three, one and three, iterative solution of the first substrate of the common standard base by using an alternating least square method
Figure FDA00032111444000000313
Figure FDA00032111444000000314
Step three-four, in the range of l 1.., C-1, the l +1 st base is found with the assurance of orthogonality to the other l bases:
Figure FDA00032111444000000315
wherein, (.)lRepresenting the corresponding variable after the first iteration;
step three-one-five, and step three-one-three-four are executed until the convergence condition is met, and finally the method is obtained
Figure FDA0003211144400000041
Is a common orthogonal group;
the convergence condition is fl+1ε, ε represents the minimum allowable error;
step three and two, utilizing the common orthogonal base obtained in the step three and one
Figure FDA0003211144400000042
And acquiring a common flat field and a corrected hyperspectral microscopic image.
6. The hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 5, characterized in that: the above-mentioned
Figure FDA0003211144400000043
Is an orthogonal basis coefficient matrix of the nth training sample in the training set, and the formula is as follows:
Figure FDA0003211144400000044
7. the hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 6, characterized in that: in the third step, the common orthogonal base obtained in the third step is utilized
Figure FDA0003211144400000045
The method for acquiring the common flat field and the corrected hyperspectral microscopic image comprises the following steps:
step three, step two, obtain the coefficient matrix of the common orthogonal base:
for each set of coefficient matrices in the training set
Figure FDA0003211144400000046
Guaranteed orthogonal basis coefficient matrix
Figure FDA0003211144400000047
Difference between them
Figure FDA0003211144400000048
Sufficiently small so that an average value is obtained
Figure FDA0003211144400000049
The coefficient matrix of the common orthogonal base is obtained;
and step III, obtaining a common flat field item F (x, y, lambda) according to the common orthogonal base obtained in the step III and the coefficient matrix obtained in the step III:
Figure FDA00032111444000000410
and step three, subtracting the high spectral logarithm matrix and the flat field item, and combining the index and rearrangement operation to obtain a corrected nth high spectral microscopic image:
Figure FDA00032111444000000411
8. the hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 7, characterized in that: in the fourth step, the common flat field obtained in the third step is evaluated and analyzed by using a test set, and the method comprises the following steps:
amplifying the overlapped parts in the data of a pair of test sets with a plurality of times of spatial resolution, selecting a wave band with uniform illumination, and selecting the spatial overlapped parts of each pair of hyperspectral microscopic images in the test sets by adopting a sub-pixel registration method;
step two, matching the dynamic range of the hyperspectral microscopic image after flat field correction with the hyperspectral microscopic image without flat field correction to obtain a matching result:
Figure FDA0003211144400000051
wherein med (-) represents the median of the image, var (-) represents the standard deviation of the image, IcorrRepresenting the corrected image, ImeasRepresenting an uncorrected image;
step four, acquiring a correction scoring formula of the space overlapping part of each pair of data in the test set, and substituting the matching result into the correction scoring formula to acquire a correction score;
the formula of the correction score of the spatial overlapping part of each pair of data is as follows:
Figure FDA0003211144400000052
wherein In',1(x, y, lambda) and In',2(x, y, λ) represents the spatial overlap of the hyperspectral microscopy images in the nth ' pair of test sets, Γ () ranges from [0, ∞ ], N ' e [1, N ']N' is the total logarithm of the hyperspectral microimages in the test set;
fourthly, determining a common rank C according to the minimum correction score to obtain an optimal CFE model, wherein the method comprises the following steps:
according to the convergence condition fl+1If the value of the correction score is larger than epsilon, testing the relation between epsilon and the rank C of the flat field term in the value range of the epsilon value, determining the value range of C, and then determining the specific value of C according to the minimum value of the specific correction score in the value range of C, thereby obtaining an optimal CFE model;
the value range of the epsilon value is set according to experience.
9. The hyperspectral microscopic image flat field correction method based on common flat field extraction according to claim 8, characterized in that: the calibration scoring standard in the fourth step and the third step is as follows:
gamma (·) ═ 0 shows that the gray scale of the overlapped part of the corrected hyperspectral microscopic image is completely consistent, and the correction result is perfect; gamma (·) < 1 is more than 0, which means that the gray level difference between the overlapped parts of the hyperspectral microscopy images after correction is smaller than the deviation before correction, and the correction is effective; gamma (·) is equal to or more than 1, which indicates that the gray difference between the overlapped parts of the hyperspectral microscopic images after correction is unchanged or larger than the deviation before correction, and indicates that the correction is invalid.
10. High spectrum microscopic image flat field correction system based on total flat field extraction, its characterized in that: the system is used for realizing the hyperspectral microscopic image flat field correction method based on common flat field extraction according to any one of claims 1 to 9.
CN202110930502.4A 2021-08-13 2021-08-13 Hyperspectral microscopic image flat field correction method and system based on shared flat field extraction Active CN113658069B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110930502.4A CN113658069B (en) 2021-08-13 2021-08-13 Hyperspectral microscopic image flat field correction method and system based on shared flat field extraction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110930502.4A CN113658069B (en) 2021-08-13 2021-08-13 Hyperspectral microscopic image flat field correction method and system based on shared flat field extraction

Publications (2)

Publication Number Publication Date
CN113658069A true CN113658069A (en) 2021-11-16
CN113658069B CN113658069B (en) 2024-04-09

Family

ID=78479870

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110930502.4A Active CN113658069B (en) 2021-08-13 2021-08-13 Hyperspectral microscopic image flat field correction method and system based on shared flat field extraction

Country Status (1)

Country Link
CN (1) CN113658069B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117405605A (en) * 2023-12-15 2024-01-16 浙江大学 Correction system and method for eliminating non-uniform light field distribution in hyperspectral image acquisition

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446582A (en) * 2018-01-25 2018-08-24 西安电子科技大学 Hyperspectral image classification method based on textural characteristics and affine propagation clustering algorithm
CN108562541A (en) * 2018-04-23 2018-09-21 南京大学 Based on matrix decomposition without lens holographic microphotography speckle noise minimizing technology and device
CN109460788A (en) * 2018-10-29 2019-03-12 西安电子科技大学 Based on low-rank-sparse information combinational network hyperspectral image classification method
US20190261913A1 (en) * 2016-10-18 2019-08-29 The Johns Hopkins University Hyperspectral imaging for passive detection of colorectal cancers
CN111860612A (en) * 2020-06-29 2020-10-30 西南电子技术研究所（中国电子科技集团公司第十研究所） Unsupervised hyperspectral image hidden low-rank projection learning feature extraction method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190261913A1 (en) * 2016-10-18 2019-08-29 The Johns Hopkins University Hyperspectral imaging for passive detection of colorectal cancers
CN108446582A (en) * 2018-01-25 2018-08-24 西安电子科技大学 Hyperspectral image classification method based on textural characteristics and affine propagation clustering algorithm
CN108562541A (en) * 2018-04-23 2018-09-21 南京大学 Based on matrix decomposition without lens holographic microphotography speckle noise minimizing technology and device
CN109460788A (en) * 2018-10-29 2019-03-12 西安电子科技大学 Based on low-rank-sparse information combinational network hyperspectral image classification method
CN111860612A (en) * 2020-06-29 2020-10-30 西南电子技术研究所（中国电子科技集团公司第十研究所） Unsupervised hyperspectral image hidden low-rank projection learning feature extraction method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117405605A (en) * 2023-12-15 2024-01-16 浙江大学 Correction system and method for eliminating non-uniform light field distribution in hyperspectral image acquisition
CN117405605B (en) * 2023-12-15 2024-03-05 浙江大学 Correction system and method for eliminating non-uniform light field distribution in hyperspectral image acquisition

Also Published As

Publication number Publication date
CN113658069B (en) 2024-04-09

Similar Documents

Publication Publication Date Title
EP1428016B1 (en) Method of quantitative video-microscopy and associated system and computer software program product
Nardiello et al. Observing multiple stellar populations with VLT/FORS2-Main sequence photometry in outer regions of NGC 6752, NGC 6397, and NGC 6121 (M 4)
CN100551081C (en) A kind of method and device of realizing white balance correction
AU2002334590A1 (en) Method quantitative video-microscopy and associated system and computer software program product
Reddy et al. Accurate histopathology from low signal-to-noise ratio spectroscopic imaging data
Larsen et al. Detailed abundances from integrated-light spectroscopy: Milky Way globular clusters
Heikkinen et al. Spectral imaging using consumer-level devices and kernel-based regression
Prisinzano et al. Star formation in the outer Galaxy: membership and fundamental parameters of the young open cluster NGC 1893
CN107274360B (en) A kind of high spectrum image denoising method based on Fisher dictionary learning, low-rank representation
Sasikumar et al. Applicability of colour transfer techniques in Twelve fringe photoelasticity (TFP)
CN113658069B (en) Hyperspectral microscopic image flat field correction method and system based on shared flat field extraction
Gargiulo et al. SIPGI: an interactive pipeline for spectroscopic data reduction
Wang et al. A novel low rank smooth flat-field correction algorithm for hyperspectral microscopy imaging
CN109991174A (en) A kind of Model transfer method between EO-1 hyperion camera and conventional spectrograph
An et al. Estimation of chlorophyll distribution in banana canopy based on RGB-NIR image correction for uneven illumination
El-Rifai et al. Enhanced spectral reflectance reconstruction using pseudo-inverse estimation method
US9122904B2 (en) Method for optimization of quantitative video-microscopy and associated system
Jiang et al. Application of image analysis for algal biomass quantification: a low-cost and non-destructive method based on HSI color space
Dobrolyubova et al. Automatic image analysis algorithm for quantitative assessment of breast cancer estrogen receptor status in immunocytochemistry
Zhang et al. Deep point spread function photometric catalog of the VVV survey data
CN114936980A (en) Hyperspectral microscopic image flat field correction method and system based on low-rank smoothing characteristic
CN110751144B (en) Canopy plant hyperspectral image classification method based on sparse representation
Quintana et al. Blur-specific no-reference image quality assesment for microscopic hyperspectral image focus quantification
Liu et al. Colorimetrical evaluation of color normalization methods for H&E-stained images
CN108470325B (en) Space-time three-dimensional noise identification and compensation method for area array staring infrared remote sensing image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant