A kind of histotomy classification method based on micro- high light spectrum image-forming technology
Technical field
The invention belongs to medical image signal processing technology fields, and in particular to one kind is based on micro- high light spectrum image-forming technology
Histotomy classification method.
Background technique
Medicine high light spectrum image-forming is that a foundation is more in clinical medicine, iconography, medical energy converter, histopathologic analysis etc.
Comprehensive crossover technology on gate technique basis belongs to the new field application of hyperspectral technique in recent years.
Micro- hyperspectral technique integrates the micro- Hyperspectral imaging of the available more high spatial resolution of micro imaging system,
To study micro-scale object (such as histotomy, cell, microorganism).Micro- high light spectrum image-forming technology fusion
Spectrum analysis and optical imagery two traditional optical diagnostic techniques can be provided simultaneously of both biological organization sample map
Information.Spectrum analysis can obtain certain point on slice sample, can be in the complete spectrum data of a certain wavelengths of interest range
Histiocytic biochemical component information is analyzed;Optical image technology then has recorded the grayscale or color image of sample,
Biopsy tissues are analyzed from form.
After the micro- high spectrum image for obtaining histotomy, it usually needs carry out data prediction and feature extraction
Process, emphasis are that the morphology and spectrum dimensional difference to cancerous issue and normal tissue are analyzed.Currently, correlation is ground both at home and abroad
The method for studying carefully use still focuses mostly in shallow-layer learning algorithm, common are principal component analysis PCA and support vector machines etc., or
It is spectral modeling cartography SAM, curve of spectrum matching method etc. based on spectral information.Turn for example, Akbari et al. obtains lung
Tumour is moved in the micro- high spectrum image of 450~950nm, is classified using SVM, to the detection sensitivity of lung cancer metastasis tissue
Reach 92.6%;Li Qingli of East China Normal University et al. acquires motor nerve cells and sense using microspectrum imaging system
Feel nerve cell, improved spectral modeling drawing SAM algorithm is used for the classification of nerve cell.Although these methods achieve
As a result, but either nicety of grading, computational efficiency or adaptivity and portability, it all needs to further increase.Its result reaches
Less than accurate medical treatment requirement pinpoint for disease, especially when facing more complicated disease subcategory type and branch, figure
As feature is not obvious enough, spectral signature difference very little, traditional algorithm and simple discrimination model it is difficult to extract to further feature come into
The effective identification and classification of row.
Summary of the invention
In order to overcome the shortcomings of above-mentioned existing method, the purpose of the present invention is to provide one kind to be based on micro- high light spectrum image-forming
The histotomy classification method of technology, it is intended to improve whole nicety of grading and speed, and improve the automation number to pathological section
According to acquisition and assorting process.
Solution of the invention is as follows:
The histotomy classification method based on micro- high light spectrum image-forming technology, comprising the following steps:
1) system modelling
1.1) the micro- high-spectral data of training set is pre-processed, to eliminate influence of noise, and it is superfluous to eliminate data
It is remaining;
1.2) training of three classes convolutional neural networks (CNN) model
1.2a) one-dimensional CNN model
CNN model is established, pretreated data are input in CNN model, in the pretreated data
The one-dimensional curve of spectrum (one-dimensional spectroscopic data) training CNN model, thus realize spectrum dimension feature extraction and classifying;Wherein,
The substantially network number of plies is determined according to sample size and spectral Dimensions, then network structure is adjusted according to training result, is adjusted
Save each layer parameter optimization network model;
1.2b) two dimension CNN model
Use for reference step 3a) determine model parameter and structure, establish a two dimension CNN model;
Principal component analysis (PCA) is carried out to pretreated data, approximation of the m principal component as original image before choosing
Expression, the input by K × K neighborhood territory pixel point (i.e. K × K × m) of current pixel as two dimension CNN model;After training,
The pretreated data are converted into series of features vector;In addition, extracting spectral line characteristic from pretreated data
(one-dimensional spectroscopic data);The feature of these two aspects is inputted LR layers jointly to classify, to realize map-spectrum dimension joint
Feature extraction and classifying;
1.2c) three-dimensional CNN model
Use for reference step 1.2a) and step 1.2b) determination model parameter and structure (while use for reference one-dimensional CNN model and
Two-dimentional CNN model), establish a three-dimensional CNN model;
Using K × K of current pixel × b neighborhood as the input of three-dimensional CNN model, wherein b is spectral coverage number, by training
Afterwards, by the input of obtained series of features vector LR layer classify, thus realize the union feature of map-spectrum dimension extract and
Classification;
2) for the histotomy classification of practical micro- high spectrum image to be measured
Referring to step 1.1), practical micro- high-spectral data to be measured is pre-processed;
Quantitative and qualitative analysis is carried out to pretreated data, according to sample size (size of the order of magnitude) and Spectral Properties
It levies (spectral coverage number and spectral resolution), assessment only carries out spectrum dimensional feature and extracts with whether classification can satisfy requirement;
If it is, by pretreated data input step 1.2a) training after one-dimensional CNN model, realize spectrum dimension
Feature extraction and classifying, the model output classification results as final classification result;
If it is not, then respectively referring to step 1.2b) and step 1.2c), pass through the two-dimentional CNN model and three-dimensional after training
CNN model realization map-spectrum dimension union feature is extracted (wherein, for two-dimentional CNN model, will specifically locate in advance with classification
Data after reason carry out PCA processing, and input of the K × K × m as two dimension CNN model combines one-dimensional spectroscopic data at LR layers, obtain
Classification results out;For three-dimensional CNN model, K × K × b neighborhood obtains classification knot directly as the input of three-dimensional CNN model
Fruit);Finally puts to the vote to the classification results of two models output and (carries out Decision fusion using ballot method and linear opinion pond),
Obtain final classification result.
Wherein, step 1.1) specifically may is that first using low-pass filtering come the influence of Removing Random No, then to each
Spectral coverage image carries out a tape handling, while the influence of high-frequency noise is eliminated with S-G first derivative;And subtracted using whitening processing
Correlation between small data.
Step 1.2a) it can specifically include following steps:
(1) it initializes:
Random initializtion network parameter θ, iter=0, err=0, nb=0, determine each channel type and activation primitive type;
Determine mode input n1, export np, the number of iterations ImaxWith learning rate α;
(2) repetitive exercise:
One-dimensional spectroscopic data is inputted first, if xiIt is i-th layer of input, calculates each network layer output;
Wherein, WiAnd biRespectively i-th layer of weight matrix and bias matrix, s are excitation function, and P (y=l), which is predicted, to be worked as
Belong to the probability of l class in preceding iteration;
Then cost function J (θ) and partial derivative δ are calculatedi;
Wherein, m is number of training, and Y is target output, and y is prediction output,Represent dot product function;
Gradient descent method undated parameter θ is constantly used in training;
Finally as the return value of cost function is smaller and smaller, network is gradually trained to optimal, reaches CNN after given threshold
Model training is completed.
The present invention has following technical effect that
The present invention studies institutional framework form in pathological section with spectral information, and the convolution of deep learning is utilized
Neural network model carries out the extraction and classification of further feature to the two, perfect while improving whole nicety of grading and speed
To the automatic data collection and assorting process of pathological section.
Detailed description of the invention
Fig. 1 is the framework schematic diagram of experimental provision of the invention.
Fig. 2 is CNN modeling procedure figure of the invention.
Fig. 3 is that the map based on two-dimentional CNN model-spectrum ties up union feature extraction and classification process figure.
Fig. 4 is histotomy classification method schematic diagram of the invention.
Fig. 5 is that the classification results of CNN disaggregated model and other methods of the invention compare.
Specific embodiment
To keep technical solution of the present invention and advantage more explicit, below in conjunction with the drawings and specific embodiments to this hair
It is bright to be described in detail.
As shown in Figure 1, Microscopic hyperspectral imaging device is by Hyperspectral imager, biomicroscope system and control computer
Composition, system includes 256 wave bands altogether, and spectral region 400nm~1000nm, averaged spectrum resolution ratio 3nm, spatial resolution can
To reach 0.5 μm, picture size 753*696.
When experiment according to different target can select different amplification microcobjective (such as: 4 ×, 10 ×, 20 ×,
40 ×, 100 ×), the intensity of light source is adjusted, pays attention to be saturated, adjustment focus adjusting mechanism guarantees that sample is in optimum position, chooses mesh
Region is marked, the micro- high spectrum image of this region pathological section is acquired.Keep light-source brightness and amplification factor constant, mobile slice
Other regions are chosen, or replace other pathological sections, acquire multiple micro- high spectrum images with identical method, and classify and deposit
Storage.By taking gastric cancer pathological section as an example, stomach organization and normal tissue sections are respectively from multiple patients with gastric cancer and normal human's sample
This, in vitro pathological tissues need to carry out H-E dyeing after the processing such as embedding, slice, dewaxing.Doctor carries out detailed after dyeing
Thin label, divides cancerous region and normal region, carries out data acquisition and classification based training based on this when experiment.
Step 1: micro- high-spectral data pretreatment: the influence in view of light source and sensor to image quality is adopted first
With low-pass filtering come the influence of Removing Random No;Then a tape handling is carried out to each spectral coverage image, while is led with S-G single order
Number eliminates the influence of high-frequency noise;Finally carry out whitening processing, primarily to reduce correlation between each feature and
Data complexity is conducive to the stability and high efficiency of later period model.
Wherein, the main process of whitening processing is as follows:
(1) it assumes initially that original high-spectral data is x, constructs the autocorrelation matrix R of initial datax=E (xxT)≠I
(2) it then looks for matrix B and transformation y=Bx is carried out to x, so that autocorrelation matrix Ry=BE (xxT)BT=Ι
(3) transformation B=Λ is carried out to B-1/2ΦT, R can be obtainedy=(Λ-1/2ΦT)ΦΛΦT(Λ-1/2ΦT)T=I
Eventually by the transformation of B so that each component of y is uncorrelated, achieve the purpose that eliminate data redundancy.
Step 2: spectrum dimensional feature extracts and classification:
Pretreated data are input in CNN model and are trained, 1-D CNN model are initially set up, according to sample
Quantity and spectral Dimensions determine the substantially network number of plies, are then adjusted according to training result to network structure, adjust each layer ginseng
Number optimization network model.Finally obtained CNN model has seven layers altogether, contains input layer I1, two convolutional layer C2 and C4, and two
Pond layer P3 and P5, full articulamentum F6 and output layer O7.First convolutional layer includes 8 convolution kernels, and convolution kernel size is 5;The
Two convolution kernels include 16 convolution kernels, and convolution kernel size is 5.Thereafter the pond maxpooling layer, followed by one are all connected with
Full articulamentum contains 100 neurons.Meanwhile can effectively it prevent using nonlinear function ReLU and dropout method
Fitting, when dropout parameter is set as 0.25, convergence speed is most fast.
It is as shown in Figure 2 for the main process of one-dimensional curve of spectrum training CNN model:
(1) it initializes: random initializtion network parameter θ, iter=0, err=0, nb=0, determine each channel type and activation
Type function.Determine mode input n1, export np, the number of iterations ImaxWith learning rate α.
(2) repetitive exercise:
One-dimensional spectroscopic data is inputted first, calculates each network layer output,
Wherein, WiAnd biRespectively i-th layer of weight matrix and bias matrix, y are the probability of the every one kind of current home.
Then cost function J (θ) and partial derivative are calculated.
Wherein, m is number of training, and Y is target output, and y is prediction output,Represent dot product function.
Gradient descent method undated parameter θ is constantly used in training,
Finally as the return value of cost function is smaller and smaller, network is gradually trained to optimal, reaches CNN after certain threshold value
Model training is completed.
Step 3: spectrum dimension union feature extracts and classification for map-:
In order to more effectively using spatial informations such as the tissue texture of pathological section and eucaryotic cell structures, need to tie up in spectrum
On the basis of learn more detailed image dimensional feature, model performance and tumor tissues classification effectiveness can be effectively improved.It can specifically adopt
It is realized respectively with 2-D CNN model and 3-D CNN model:
A 2-D CNN model is established, 3 layers of convolution, ReLU activation primitive and max-pooling layers are included.In order to make
Model has better Generalization Capability, needs the number of parameters of Controlling model, the i.e. number of plies of model and every layer of scale, therefore make here
With lesser convolutional network, every layer of filter number is also few.Principal component point is carried out to initial data first before training
It analyses (PCA), approximate expression of the m principal component as original image before choosing, (i.e. by K × K neighborhood territory pixel point of current pixel
K × K × m) input as model.In view of picture size, eucaryotic cell structure and biopsy tissues feature, neighborhood window K value 45,
Convolution kernel size is 5, and 32,64 and 128 convolution kernels are respectively set in three convolutional layers.The number of the selected principal component of change can be passed through
It measures to adjust the spectral information of reservation, experiments have shown that good nicety of grading can be obtained by retaining 3~5 principal components.For example, can
PCA treated approximate expression of preceding 3 principal components as initial data is chosen, by 45 × 45 neighborhoods (i.e. 45 of current pixel
× 45 × 3) it is used as mode input.In the training process using ReLU activation and dropout method, can effectively inhibit to intend
It closes.Initial data is converted into series of features vector after CNN training, the input of these features is classified for LR layers, together
When also feature is used as to input the one-dimensional spectroscopic data, to realize that map-spectrum dimension union feature extracts and classification, such as Fig. 3
It is shown.
A 3-D CNN model is established, using K × K of current pixel × b neighborhood as the input of 3-D CNN model,
Middle b is spectral coverage number, every layer of convolution kernel size 5 × 5 × 32, and layer 2 × 2 kernel in pond is sampled, recently enter LR layers and divided
Class, to realize that map-spectrum dimension union feature extracts and classifies.In the training process using ReLU activation and dropout
Method can effectively inhibit over-fitting.Under suitable 3-D CNN framework, neighborhood territory pixel point in all spectral coverages can use,
Sufficiently spectrum and space characteristics of the study to tumor tissues.
Step 4: combining classifiers and result visualization:
Select final classification knot of one of the classification results of the above all kinds of CNN models as tumor tissues and normal tissue
Fruit.
For histotomy high spectrum image, we more want to recognize that medical pathologies variation and training obtain between feature
Relationship, corresponding different cancer clinical performance, model learning to which kind of feature it can be explained.Therefore in 2-D
A deconvolution network is added in CNN model, deconvolution network here itself is used only to detection one without learning ability
Trained CNN.Using the characteristic pattern that each layer obtains as input, after the feature map of activation by anti-pond, instead
The operation such as activation, deconvolution reconstructs corresponding input stimulus, the stimulation of these reconstruct is shown to this until being originally inputted layer
The more useful information of feature, by analyzing these information come implementation model tuning, and between learned feature and cancer classification
Clinical interpretation is studied.
As shown in figure 4, histotomy classification method of the invention, first carries out practical micro- high-spectral data to be measured pre-
Processing;Then quantitative and qualitative analysis is carried out to pretreated data, according to sample size (size of the order of magnitude) and spectrum
Feature, assessment only carry out spectrum dimensional feature and extract with whether classification can satisfy requirement;If it is, by pretreated data
One-dimensional CNN model after input training realizes the feature extraction and classifying of spectrum dimension, and the classification results of model output are as most
Whole classification results;If it is not, then passing through the two-dimentional CNN model and three-dimensional CNN model realization map-spectrum dimension after training respectively
Union feature extracts and classification, then puts to the vote the classification results of two models output (using ballot method and linear opinion pond
Carry out Decision fusion), obtain final classification result.
It is compared using method of the invention with other methods, accuracy, sensitivity and the specificity of distinct methods
As a result as shown in Figure 5.The results showed that the histotomy classification side proposed by the present invention based on micro- high light spectrum image-forming technology
Method, can be with the structure feature of high efficiency extraction pathological tissues and normal tissue and spectral signature difference, the CNN model established based on this
The accurate identification to the two may be implemented, realize automatic data collection and assorting process to pathological section.