CN108509998A - A kind of transfer learning method differentiated on different devices for target - Google Patents
A kind of transfer learning method differentiated on different devices for target Download PDFInfo
- Publication number
- CN108509998A CN108509998A CN201810291322.4A CN201810291322A CN108509998A CN 108509998 A CN108509998 A CN 108509998A CN 201810291322 A CN201810291322 A CN 201810291322A CN 108509998 A CN108509998 A CN 108509998A
- Authority
- CN
- China
- Prior art keywords
- data
- matrix
- migration
- model
- different devices
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000013526 transfer learning Methods 0.000 title claims abstract description 13
- 239000011159 matrix material Substances 0.000 claims abstract description 82
- 238000013508 migration Methods 0.000 claims abstract description 49
- 230000005012 migration Effects 0.000 claims abstract description 49
- 238000012549 training Methods 0.000 claims abstract description 41
- 230000003595 spectral effect Effects 0.000 claims description 56
- 238000013528 artificial neural network Methods 0.000 claims description 29
- 230000009466 transformation Effects 0.000 claims description 26
- 238000012360 testing method Methods 0.000 claims description 23
- 238000001228 spectrum Methods 0.000 claims description 18
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims description 9
- 239000004576 sand Substances 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 238000010200 validation analysis Methods 0.000 claims description 5
- 238000010238 partial least squares regression Methods 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 abstract description 6
- 230000007704 transition Effects 0.000 abstract description 4
- 238000004611 spectroscopical analysis Methods 0.000 abstract 3
- 238000012545 processing Methods 0.000 abstract 1
- 240000008042 Zea mays Species 0.000 description 13
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 description 13
- 235000002017 Zea mays subsp mays Nutrition 0.000 description 13
- 235000005822 corn Nutrition 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 9
- 238000012795 verification Methods 0.000 description 7
- 238000002329 infrared spectrum Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 210000002569 neuron Anatomy 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 102100033824 A-kinase anchor protein 12 Human genes 0.000 description 1
- 101000779382 Homo sapiens A-kinase anchor protein 12 Proteins 0.000 description 1
- 238000004497 NIR spectroscopy Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000000411 transmission spectrum Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The present invention provides a kind of transfer learning methods differentiated on different devices for target, include the following steps：S1, sample establish serial number, acquire spectroscopic data on different devices successively；S2, the spectroscopic data on distinct device is divided using identical data set；S3, the spectroscopic data for passing through step S2 processing on distinct device is modeled respectively, selects best model；S4, solution room transition matrix is carried out based on the obtained data of step S3；S5, the data of the poor equipment of performance are subjected to spatial alternation using the spatial migration matrix in step S4, re -training preserves relevant parameter.The present invention is moved to the feature space of higher model in a certain equipment in an other equipment by the method for combining spatial migration, to promote the performance of shallow-layer neural network model in an other equipment.
Description
Technical Field
The invention belongs to the field of computer and optical technology application, and particularly relates to a transfer learning method for identifying targets on different devices based on a near infrared spectrum technology.
Background
Transfer learning for target identification is a new technical field of computer and spectrum application, taking two types of corn kernels as an example, because in the actual market, the corn kernels may have misoperation situations of mixing cheap seeds or different types of kernels in enterprise production. The near infrared spectrum technology and the computer technology are applied, and the aim is to realize the condition of identifying the mixture of two types of corn seeds by the aid of a mode identification method. However, in practical applications, after a model for identifying corn kernels is successfully trained on one equipment, the model for identifying corn kernels is distorted when the model is used on another new equipment, that is, the identification capability of the model on the new equipment is reduced. More data is therefore often required to be collected on new equipment, resulting in a time consuming process. In this regard, the migration learning can migrate a model with higher performance by improving two types of models on different devices, thereby realizing the consistency of device models with lower data cost.
The corn haploid identification technology based on the near infrared spectrum technology can obtain the internal substance information of the non-uniformly distributed grains by adopting a diffuse transmission method to obtain the spectrum of the grains. But due to the complex spectral composition, the diffuse transmission mechanism has not been well-established theoretical explanation. In practical application, seed seeds are identified through spectra, a correct identification rate of haploid seeds needs to be improved as much as possible by combining a computer mode identification algorithm, a high identification accuracy rate is obtained when seed type identification is carried out by using a neural network model, and practical requirements are met. However, in engineering mass production of equipment, even near infrared detectors produced by the same manufacturer have differences due to process differences in the assembly process of the equipment. Therefore, when a trained model on one device is applied to another device, the performance of the original good model may be degraded. Thus, using transfer learning may reduce the need for training data when additional devices retrain the model.
Therefore, in recent years, with the field of artificial intelligence, extensive research is being carried out in the field of image, text and voice, and particularly, improving model performance by using transfer learning becomes a research hotspot problem. In order to better utilize past data, a weight-based example migration method, namely, tragaboost (SVM), is proposed by a student of shanghai transport university in 2009, and achieves the purpose of sample migration by attenuating irrelevant task source domains (weights of samples so as to keep samples similar to current task target domains). Yao, 2010 et al proposed an example migration of a multi-source domain task based on TrAdaBoost. The parameter migration problem of face recognition on different devices is solved by a cascade Bayes mode based on parameter migration proposed by Asahi, Cao, Asahi and the like in 2013, cross domain emotion classification is solved by TrAdaBoost by Xinchang and the like in 2017, Pooja Rawade is used for a cross domain recommendation system in 2017, and DANN migration learning is proposed by Gravin and the like in 2016, so that migration learning among different domains during deep neural network learning can be effectively improved, but for a shallow neural network, negative migration still exists in the method and influences performance. Therefore, research on transfer learning still has its academic and practical value.
Disclosure of Invention
Technical problem to be solved
In view of the above technical problems, the present invention provides a migration learning method for discrimination of targets on different devices. In the research object classification, the shallow neural network has shown excellent classification performance and can be used on automatic equipment. Therefore, the migration learning is further researched on the basis of the shallow neural network, and the problem of improving the model performance of the target identification model on different automatic equipment can be solved. Therefore, the characteristic space of a higher model on one device is transferred to another device by combining a space transfer method, so that the performance of a shallow neural network model on another device is improved.
(II) technical scheme
According to an aspect of the present invention, there is provided a migration learning method for discrimination of targets on different devices, comprising the steps of:
s1, establishing serial numbers of samples, and sequentially collecting spectral data on different devices;
s2, dividing the spectrum data on different devices by adopting the same data set;
s3, respectively modeling the spectral data processed in the step S2 on different devices, and selecting the best model;
s4, solving a space migration matrix based on the data obtained in the step S3;
and S5, carrying out space transformation on the data of the equipment with poor performance by using the space migration matrix in the step S4, retraining and storing related parameters.
Preferably, in step S1, spectral data is collected in the same order on each spectrometer device in transmission or diffuse transmission, and corresponding label data is obtained.
Preferably, in step S2, a sequence of the same size as the number of samples without repetition is generated while dividing the sequence into three data sets: training set, verifying set and testing set; the data collected on each device is divided into three data sets corresponding to the sequence.
Preferably, step S3 includes:
s31, respectively carrying out spectrum characteristic normalization on the data collected on different devices;
s32, reducing the dimensionality of the normalized spectral data to obtain a score matrix XSAnd its transformation matrix ST;
s33, training a model;
s34, obtaining the optimal model by comparing the performance of the model trained by the data on each device, saving all the parameters of the model and the scoring matrix XsAnd a score matrix and a transformation matrix ST on other devices.
Preferably, in step S31, the intensity of the spectrum data is normalized by a Zero-phase component analysis method to obtain normalized spectrum data; the calculation formula for the characteristic normalization of the spectral data is as follows:
wherein, X represents a characteristic matrix,a zero-mean matrix obtained by subtracting the mean vector from the feature matrix X is represented, U represents the base of the space, S represents a diagonal matrix, eps is a small constant,representing the normalized feature matrix;
in step S32, the spectral data is dimensionality reduced by using partial least squares regression to obtain a score matrix XsAnd its transformation matrix ST; the calculation formula for reducing the dimension of the spectral data is as follows:
wherein,is normalized feature matrix, y is label data, and p is feature matrixQ is the projection direction of the label data y, the transformation matrix ST is calculated based on p obtained each time, and the scoring matrix XSIs composed ofMatrix after ST projection;
step S33 includes:
s331, inputting the spectral data subjected to dimensionality reduction into a neural network, and adjusting parameters of the neural network;
and S332, fine adjustment of neural network parameters and performance evaluation.
Preferably, in step S331, the feature classification is performed on the reduced-dimension spectral data by using an L2 norm regularization back propagation neural network method, where a cost function is in the form of:
in the formula, m represents the number of samples, xiDenotes the ith sample, yiTrue value, h, of label data representing the ith sampleθ(. The) is a model hypothesis of the neural network, lambda is a penalty parameter, and theta represents a parameter to be learned by the model.
Preferably, the model to-be-learned parameter θ is initialized in a random manner, and the update formula is as follows:
where ε is a learning rate parameter, JbAs a cost function.
Preferably, in step S4, the calculation formula of the spatial transition matrix S is:
wherein, the optimal model is the jth equipment, and other multiple equipments are calculated in turn, are respectively notWith the score matrix on the devices Ii, Ij is the device with the highest identification performance, and Ii is the device with the worse identification performance.
Preferably, step S5 includes:
s51, carrying out space migration transformation on the data of the equipment with poor performance;
and S52, retraining and storing the related parameters.
Preferably, in step S51, the calculation formula for performing the spatial migration transformation on the data of the poor-performance device is as follows:
wherein,and the new scoring matrix is obtained after the scoring matrix passes through the space transformation matrix S. In turn, the scoring matrices of the validation set and the test set are similarly transformed.
(III) advantageous effects
According to the technical scheme, the transfer learning method for identifying the targets on different devices has at least one of the following beneficial effects:
(1) the invention provides a method based on feature space mapping, which realizes the purpose of transferring the feature space on one device to another device and designing and identifying the method from the perspective of high-dimensional space image geometry, thereby effectively improving the identification performance of the model;
(2) the neural network model classifier adopted by the invention can realize parallel computation in embedded hardware systems such as FPGA and the like, thereby improving the training and predicting speed of the model and ensuring the identification operation efficiency;
(3) the method combines the space migration with the neural network model classifier, and can effectively combine the practical application, so that the method has universality.
Drawings
Fig. 1 is a schematic diagram of steps of a migration learning method for identifying targets on different devices according to an embodiment of the present invention.
FIG. 2 is a detailed flowchart of a migration learning method for identifying targets on different devices according to an embodiment of the present invention;
FIG. 3 is a diagram of two different near infrared spectrometer devices used in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.
In an exemplary embodiment of the present invention, a migration learning method for discrimination of targets on different devices is provided. Fig. 1 is a schematic diagram of steps of a migration learning method for identifying targets on different devices according to an embodiment of the present invention. As shown in fig. 1, the present invention relates to a transfer learning method for identifying targets on different devices, comprising the following steps:
s1, establishing serial numbers of samples, and sequentially collecting spectral data on different devices;
s2, dividing the spectrum data on different devices by adopting the same data set;
s3, respectively modeling the spectral data processed in the step S2 on different devices, and selecting the best model;
s4, solving a space migration matrix based on the data obtained in the step S3;
and S5, carrying out space transformation on the data of the equipment with poor performance by using the space migration matrix in the step S4, retraining and storing related parameters.
The present invention is further described in detail below with reference to two types of corn kernels as examples.
Fig. 2 is a flowchart of a migration learning method for identifying two types of corn kernels on two different devices based on a near infrared spectroscopy technology according to an embodiment of the present invention, and for convenience of description, only the parts related to the embodiment of the present invention are shown. As shown in fig. 2, the method comprises the steps of:
step S1: and establishing serial numbers for the samples, and sequentially acquiring spectral data on different devices.
Step S1 includes:
and a substep S11, sequentially marking the corn samples of the A and B varieties with serial numbers. Specifically, corn kernels (Jingyu 16, nong Da 108) of two varieties A and B are respectively placed into marked storage boxes and are arranged in sequence in an experiment box.
Sub-step S12, collecting spectra in the same order on each spectrometer device in transmission or diffuse transmission mode, and obtaining corresponding label data, one of which is 1 and the other of which is 0.
In one embodiment, one sample of the A and B varieties is taken out alternately, and the transmission spectrum data of each device is obtained on the I1 spectrometer and the I2 spectrometer respectively, and is stored for the A and B varieties respectively. In the embodiment, near infrared spectrum data sets X of two types of kernels A and B of corn and corresponding spectrum label data sets y of haploid and heterozygous corn are acquired in a diffuse transmission mode. The spectral data set X comprises original spectral intensity data, spectral size information and other information, the spectral dimension n is 125 dimensions, the spectral wavelength range is 900nm-1700nm, and floating point number values are represented. In the spectral tag data set y, the tag information is an a spectral tag 1 and a B spectral tag 0.
Step S2: the same dataset partition is used for spectral data on different devices.
Step S2 includes:
in sub-step S21, a sequence is generated that is the same size as the number of samples and does not overlap, and the sequence is divided into three data sets: training set, verifying set and testing set;
the spectrum data set is divided, including adopting the random mode of not putting back to extract the spectrum, generate 1 to m random number set R, m is the sample total number, correspond the random number set with spectral data and label data, divide into three data set with m spectral data in the spectral data set according to the proportion: training set, validation set and test set. The training set is used for training the model, the verification set is used for adjusting parameters, and the test set is used for testing the performance of the model.
Preferably, the ratio of the training set, the validation set and the test set can be set as 2: 1: 2 or 3: 1: 1.
sub-step S22, the data collected by device I1 is scaled using the sequence generated in sub-step S21.
Sub-step S23, the data collected by device I2 is scaled using the sequence generated in sub-step S21.
Step S3: the spectral data processed in step S2 on different devices are modeled, respectively, and the best model is selected.
Step S3 includes:
and a substep S31 of normalizing spectral characteristics of the data I1 and I2 of the device, respectively, to normalize intensity of the spectral data detected by the near-infrared spectrometer to obtain normalized spectral data, wherein the detection instrument adopted in the embodiment is a micro nir-1700 near-infrared spectrometer. The correlation between features can be well removed using Zero-phase component analysis (ZCA).
Specifically, in this embodiment, Zero-phasecomponents analysis (ZCA) is performed by using the training sets obtained in substeps S22 and S23 to obtain a normalization matrix thereof, and the calculation method is as follows:
in the formula, a feature matrix X represents original spectral data of a training set, the size of the feature matrix X is n multiplied by m, n represents a single spectral feature dimension, and m is the number of training samples. mean (X) represents the average of the training set spectra,and (3) subtracting the mean vector mean (X) from the training set feature matrix X to obtain spectral data which is a zero mean matrix. svd (-) is singular value decomposition (singular value decomposition), U is the space base vector obtained after decomposition, and S is the diagonal matrix obtained. eps represents a small constant, preferably 1 e-6.And obtaining a new characteristic matrix for training set data after ZCA characteristic normalization.
The above-mentioned normalized matrix Z is obtained by calculation according to Zero-phase components analysis (ZCA) method using the spectral data in the training set, and Z isThe normalized spectral data of the validation set and test set are then calculated according to Z. The normalized data of the verification set and the test set is calculated according to the average value of the verification set and the test set and the above U and S, and the spectrum data after the normalization of the training set, the verification set and the test set are respectively marked as Tr, Va and Te. And storing the data obtained after normalization.
And a substep S32, reducing the dimensionality of the normalized spectral data. The dimension of the spectral feature is reduced, and effective spectral feature information is extracted. Partial least squares regressionThe method can retain some weak component characteristics compared with the principal component analysis method, and the embodiment adopts a partial least squares regression method to reduce the dimension of the spectral data to obtain a score matrix XsAnd its transformation matrix ST.
Specifically, in this embodiment, the normalized training set data and the corresponding label data obtained in the substep S31 are used, and the dimension k of the data is set at the same time, preferably, k is the median of the set {3,10,20}, and the calculation form is, in combination with the partial least squares regression method:
in the formula, the first step is that,representing the normalized spectral data Tr, p of the training set as the training set dataQ is the projection direction of the spectral label data set y corresponding to the training set. Meanwhile, p, q in each iteration satisfy the constraint of a modulo length of 1. Thereby obtaining a score matrix X of the training setsAnd its transformation matrix ST, score matrix XsIs composed ofAnd (3) calculating the ST according to p obtained each time after the ST is projected to obtain the matrix, and respectively calculating by using the same linear transformation through Va and Te to obtain the scoring matrix of the verification set and the test set. Accordingly, the feature data is used as the low-dimensional feature data after dimension reduction.
Substep S33: and (5) training the model.
The sub-step S33 includes:
and a substep S331, inputting the spectral data after dimension reduction into a neural network, and adjusting parameters of the neural network.
The spectral feature classification is to identify and distinguish the spectrum of the haploid grain from the spectrum of the heterozygous grain, and the neural network classifier can extract hidden features in the features, so that the method is suitable for the situations of complex features. In this embodiment, an L2 norm regularization back propagation neural network (back propagation neural network) method is adopted to perform feature classification on the spectral data after dimensionality reduction. The reverse neural network has three layers, namely an input layer, a hidden layer and an output layer, the number of neurons of the input layer and the hidden layer is the dimension k, and the number of neurons of the output layer is 1 or 2.
Specifically, the training set data characteristics (the scoring matrix X obtained in step S32) are useds) And accessing the neural network with the k-k-1 structure. The cost function of the neural network is:
in the formula, m represents the number of training set samples, xiRepresenting a single i-th training sample, yiRepresents the spectral label data corresponding to the ith sample, hθ(. The) is a model hypothesis of the neural network, lambda is a penalty parameter, theta represents a learning parameter set in the inverse neural network, theta is initialized in a random mode, and an updating formula of the theta is as follows:
in the formula, epsilon is a learning rate parameter, and Jb is a cost function.
Preferably, the training number N is set to 800 times, and λ may be set {1,10 }-1,10-2,10-3,10-4,10-5,10-6And f, obtaining the optimal value of lambda according to 50 average accuracy rates of the verification set, and determining the approximate range of k according to the fixed feature dimension. In this embodiment λ is 10-4While determining k in the range of [4,10 ]]In the meantime.
And a substep S332 of fine adjustment of neural network parameters and performance evaluation.
Fixing punishment parameters, obtaining a training set and a testing set, utilizing a neural network model obtained by training the training set, utilizing the testing set to carry out final model performance evaluation, determining the k value of the neuron number of the optimal hidden layer, exhaustively calculating the feature dimension in the determined range, repeating the process for N times, utilizing the distribution condition of the accuracy of the testing set to finally determine the optimal feature dimension, simultaneously obtaining the optimal model performance evaluation, exhaustively calculating the k value in the range of 3 to 10, and preferably, repeating the step for 50 times to obtain the average accuracy and the standard deviation of the testing set as the basis for judging the model performance. In this specific embodiment, as shown in table 1 and table 2, the decibels are the relationship between the Average accuracy of the training set (Average training accuracy), the Average accuracy of the test set (Average accuracy), and the standard difference value (SD of test accuracy) of the data collected by the devices I1 and I2, which change with the value of k, and the Average accuracy and the standard difference value are integrated, where the optimal k obtained in this embodiment is 5. It can be seen from tables 1 and 2 that the model performance of the I2 device is higher than that of the I1 device.
TABLE 1
TABLE 2
Substep S34, obtaining the optimal model by comparing the performance of the model trained by the data on each device, saving all parameters of the model, and scoring matrix XsAnd a score matrix and a transformation matrix ST on other devices.
Specifically, fixing the neural network parameter θ obtained by the device I2 in step S331, the penalty parameter λ, and the optimal low-dimensional space dimension k determined in step S332, reacquiring the training set and the test set, training a model using the training set, performing model training according to the determined parameters, storing the obtained training set mean, the characteristic normalization matrix, the partial least square transformation matrix, and the neural network parameters, and simultaneously generating a model parameter file, and the characteristic normalization matrix and the partial least square transformation matrix of the I1 device.
Step S4: solving the spatial migration matrix is performed based on the data obtained in step S3.
Specifically, the score matrix obtained by the partial least squares method in step S34 for each device is readCalculating a space migration matrix S:
in the formula, the average recognition rate in the device I2 is higher than that in the device I1, I2 is the device with the highest recognition performance, and I1 is the device with the poorer recognition performance. Therefore, S represents a spatial transition matrix calculated using the scoring matrices obtained by the two partial least squares method.
Step S5: and performing space transformation on the data of the equipment with poor performance by using the space migration matrix in the step S4, retraining and storing the relevant parameters.
Step S5 includes:
and a substep S51, performing space migration transformation on the data of the device I1 with poor performance.
Specifically, the characteristics obtained by the device I1 in step S34 will be paired with the spatial transition matrix S in step S4The transformation is carried out, and the calculation method is as follows:
in the formula, the first step is that,features after spatial migration. Similarly, the scoring matrix of the test set data is transformed similarly to the sum verification set.
And a substep S52 of re-executing the step S33, re-training, and obtaining new model performance after spatial migration, wherein the result is shown in Table 3, and the neural network parameters theta and the spatial migration matrix S are saved at the same time.
TABLE 3
The embodiment of the invention obtains data of different wavelength points and dimension information thereof under two near-infrared spectrometers by adopting a computer unified programming model, realizes a method for identifying corn haploid grains by respectively normalizing spectral characteristics, reducing dimensions, combining a neural network and a space migration method, and effectively improves the model identification rate and the stability by designing a migration algorithm from the geometric angle.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is the best embodiment. Based on such understanding, the technical solutions of the present invention may be substantially or partially embodied in the form of a software product, which is stored in a storage medium and includes instructions for enabling a terminal device (which may be a personal computer, a server, or an embedded device) to execute the method according to the embodiments of the present invention. The test platform of the embodiment: a CPU: pentium (R) Dual-Core CPU E5800@3.20 GHz; memory: 4.00 GB; the Windows 732-bit operating system; software: matlab R2015 a.
It is to be noted that, in the attached drawings or in the description, the implementation modes not shown or described are all the modes known by the ordinary skilled person in the field of technology, and are not described in detail. Furthermore, the above definitions of the various elements and methods are not limited to the particular structures, shapes or arrangements of parts mentioned in the examples, which may be easily modified or substituted by one of ordinary skill in the art, for example:
the migration method is applied to other classifiers such as a Support Vector Machine classifier, a Bayesian classifier and the like.
In addition, unless steps are specifically described or must occur in sequence, the order of the steps is not limited to that listed above and may be changed or rearranged as desired by the desired design. The embodiments described above may be mixed and matched with each other or with other embodiments based on design and reliability considerations, i.e., technical features in different embodiments may be freely combined to form further embodiments.
In summary, the present invention provides a migration learning method for target discrimination on different devices. The invention is based on the near infrared spectrum technology, and can improve the model performance of another identification device by utilizing the good characteristic space of one identification device.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A transfer learning method for discrimination between different devices for a target, comprising the steps of:
s1, establishing serial numbers of samples, and sequentially collecting spectral data on different devices;
s2, dividing the spectrum data on different devices by adopting the same data set;
s3, respectively modeling the spectral data processed in the step S2 on different devices, and selecting the best model;
s4, solving a space migration matrix based on the data obtained in the step S3;
and S5, carrying out space transformation on the data of the equipment with poor performance by using the space migration matrix in the step S4, retraining and storing related parameters.
2. The migration learning method of claim 1, wherein in step S1, spectral data are collected in the same order on each spectrometer device in transmission or diffuse transmission mode, and corresponding label data are obtained.
3. The migration learning method according to claim 1, wherein in step S2, a sequence is generated that is the same size as the number of samples and does not overlap, and the sequence is divided into three data sets: training set, verifying set and testing set; the data collected on each device is divided into three data sets corresponding to the sequence.
4. The migration learning method according to claim 1, wherein step S3 includes:
s31, respectively carrying out spectrum characteristic normalization on the data collected on different devices;
s32, reducing the dimensionality of the normalized spectral data to obtain a score matrix XsAnd its transformation matrix ST;
s33, training a model;
s34, obtaining the optimal model by comparing the performance of the model trained by the data on each device, saving all the parameters of the model and the scoring matrix XSAnd a score matrix and a transformation matrix ST on other devices.
5. The migration learning method according to claim 4, wherein in step S31, the intensity of the spectral data is normalized by a Zero-phase component analysis method to obtain normalized spectral data; the calculation formula for the characteristic normalization of the spectral data is as follows:
wherein, X represents a characteristic matrix,a zero-mean matrix obtained by subtracting the mean vector from the feature matrix X is represented, U represents the base of the space, S represents a diagonal matrix, eps is a small constant,representing the normalized feature matrix;
in step S32, the spectral data is dimensionality reduced by using partial least squares regression to obtain a score matrix XsAnd its transformation matrix ST; the calculation formula for reducing the dimension of the spectral data is as follows:
wherein,is normalized feature matrix, y is label data, and p is feature matrixQ is the projection direction of the label data y, the transformation matrix ST is calculated based on p obtained each time, and the scoring matrix XsIs composed ofMatrix after ST projection;
step S33 includes:
s331, inputting the spectral data subjected to dimensionality reduction into a neural network, and adjusting parameters of the neural network;
and S332, fine adjustment of neural network parameters and performance evaluation.
6. The transfer learning method according to claim 5, wherein in step S331, the feature classification is performed on the dimensionality-reduced spectral data by using an L2 norm regularization back propagation neural network method, and a cost function is in a form of:
in the formula, m represents the number of samples, xiDenotes the ith sample, yiTrue value, h, of label data representing the ith sampleθ(. The) is a model hypothesis of the neural network, lambda is a penalty parameter, and theta represents a parameter to be learned by the model.
7. The transfer learning method according to claim 6, wherein the model parameter θ to be learned is initialized in a random manner, and the updating formula is:
where ε is a learning rate parameter, JbAs a cost function.
8. The migration learning method according to claim 4, wherein in step S4, the calculation formula of the spatial migration matrix S is:
wherein, the optimal model is the jth equipment, and other multiple equipments are calculated in turn, the score matrixes are respectively on different devices Ii and Ij, and Ij is identificationThe highest performance device, Ii, is the device with the poorer performance.
9. The migration learning method according to claim 4, wherein step S5 includes:
s51, carrying out space migration transformation on the data of the equipment with poor performance;
and S52, retraining and storing the related parameters.
10. The migration learning method according to claim 9, wherein in step S51, the calculation formula for performing the spatial migration transformation on the data of the poor-performance device is as follows:
wherein,and the new scoring matrix is obtained after the scoring matrix passes through the space transformation matrix S. In turn, the scoring matrices of the validation set and the test set are similarly transformed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810291322.4A CN108509998A (en) | 2018-03-30 | 2018-03-30 | A kind of transfer learning method differentiated on different devices for target |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810291322.4A CN108509998A (en) | 2018-03-30 | 2018-03-30 | A kind of transfer learning method differentiated on different devices for target |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108509998A true CN108509998A (en) | 2018-09-07 |
Family
ID=63380158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810291322.4A Pending CN108509998A (en) | 2018-03-30 | 2018-03-30 | A kind of transfer learning method differentiated on different devices for target |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108509998A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109444072A (en) * | 2018-10-12 | 2019-03-08 | 南京富岛信息工程有限公司 | The solution at deceitful peak in a kind of transfer of near-infrared spectroscopy |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080272782A1 (en) * | 2007-05-03 | 2008-11-06 | Fa-Hsuan Lin | Suppression of noise in mr images and mr spectroscopic images using signal space projection filtering |
| CN104374738A (en) * | 2014-10-30 | 2015-02-25 | 中国科学院半导体研究所 | Qualitative analysis method for improving identification result on basis of near-infrared mode |
| CN105486659A (en) * | 2015-11-23 | 2016-04-13 | 中国农业大学 | Construction method and application of corn seed variety authenticity identifying model |
| CN106680238A (en) * | 2017-01-06 | 2017-05-17 | 东北大学秦皇岛分校 | Method for analyzing material composition content on basis of infrared spectroscopy |
| CN107037001A (en) * | 2017-06-15 | 2017-08-11 | 中国科学院半导体研究所 | A kind of corn monoploid seed discrimination method based on near-infrared spectrum technique |
| CN107657279A (en) * | 2017-09-26 | 2018-02-02 | 中国科学院大学 | A kind of remote sensing target detection method based on a small amount of sample |
-
2018
- 2018-03-30 CN CN201810291322.4A patent/CN108509998A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080272782A1 (en) * | 2007-05-03 | 2008-11-06 | Fa-Hsuan Lin | Suppression of noise in mr images and mr spectroscopic images using signal space projection filtering |
| CN104374738A (en) * | 2014-10-30 | 2015-02-25 | 中国科学院半导体研究所 | Qualitative analysis method for improving identification result on basis of near-infrared mode |
| CN105486659A (en) * | 2015-11-23 | 2016-04-13 | 中国农业大学 | Construction method and application of corn seed variety authenticity identifying model |
| CN106680238A (en) * | 2017-01-06 | 2017-05-17 | 东北大学秦皇岛分校 | Method for analyzing material composition content on basis of infrared spectroscopy |
| CN107037001A (en) * | 2017-06-15 | 2017-08-11 | 中国科学院半导体研究所 | A kind of corn monoploid seed discrimination method based on near-infrared spectrum technique |
| CN107657279A (en) * | 2017-09-26 | 2018-02-02 | 中国科学院大学 | A kind of remote sensing target detection method based on a small amount of sample |
Non-Patent Citations (2)
| Title |
|---|
| LIU YANG，AND ETC: "Robust and Non-Negative Collective Matrix Factorization for Text-to-Image Transfer Learning", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| 张景祥等: "融合异构特征的子空间迁移学习算法", 《自动化学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109444072A (en) * | 2018-10-12 | 2019-03-08 | 南京富岛信息工程有限公司 | The solution at deceitful peak in a kind of transfer of near-infrared spectroscopy |
| CN109444072B (en) * | 2018-10-12 | 2021-02-09 | 南京富岛信息工程有限公司 | Method for solving rogue peaks in near-infrared spectrum model transfer |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | Deep learning with kernel regularization for visual recognition | |
| US11875488B2 (en) | Method and device for parallel processing of retinal images | |
| CN107037001A (en) | A kind of corn monoploid seed discrimination method based on near-infrared spectrum technique | |
| CN104680179B (en) | Method of Data with Adding Windows based on neighborhood similarity | |
| CN111914728A (en) | Hyperspectral remote sensing image semi-supervised classification method and device and storage medium | |
| Bongiorno et al. | Classification methods for Hilbert data based on surrogate density | |
| CN109190511B (en) | Hyperspectral classification method based on local and structural constraint low-rank representation | |
| CN110309822B (en) | Hyperspectral image band selection method based on quantum evolution particle swarm algorithm | |
| CN102142082A (en) | Virtual sample based kernel discrimination method for face recognition | |
| CN115713634B (en) | Color collocation evaluation method combining similarity measurement and visual perception | |
| CN104616022A (en) | Classification method of near infrared spectrum | |
| Zhang et al. | Hyperspectral image classification using an unsupervised neuro-fuzzy system | |
| CN103955711B (en) | A kind of mode identification method in imaging spectral target identification analysis | |
| Xu et al. | Hierarchically regularized entropy balancing | |
| Jang et al. | A reparametrization-invariant sharpness measure based on information geometry | |
| CN109325536B (en) | A kind of biomimetic pattern recognition method and its device | |
| Zhao et al. | Attention-Based CNN Ensemble for Soil Organic Carbon Content Estimation with Spectral Data | |
| CN112541530B (en) | Data preprocessing method and device for clustering model | |
| Forbes et al. | Approximate Bayesian computation with surrogate posteriors | |
| CN108509998A (en) | A kind of transfer learning method differentiated on different devices for target | |
| Clark et al. | Synthetic aperture radar physics-based image randomization for identification training: SPIRIT | |
| CN110222310A (en) | A kind of shared AI scientific instrument Data Analysis Services system and method | |
| Barde | Bayesian estimation of large-scale simulation models with Gaussian process regression surrogates | |
| Işık et al. | Consensus rule for wheat cultivar classification on VL, VNIR and SWIR imaging | |
| Hong et al. | Estimating the square root of probability density function on Riemannian manifold |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180907 |