CN111539474B - Classifier model transfer learning method - Google Patents

Classifier model transfer learning method Download PDF

Info

Publication number
CN111539474B
CN111539474B CN202010329243.5A CN202010329243A CN111539474B CN 111539474 B CN111539474 B CN 111539474B CN 202010329243 A CN202010329243 A CN 202010329243A CN 111539474 B CN111539474 B CN 111539474B
Authority
CN
China
Prior art keywords
sensor array
new
classifier model
layer
weight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010329243.5A
Other languages
Chinese (zh)
Other versions
CN111539474A (en
Inventor
赵海堂
刘航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian University of Technology
Original Assignee
Dalian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian University of Technology filed Critical Dalian University of Technology
Priority to CN202010329243.5A priority Critical patent/CN111539474B/en
Publication of CN111539474A publication Critical patent/CN111539474A/en
Application granted granted Critical
Publication of CN111539474B publication Critical patent/CN111539474B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Investigating Or Analyzing Materials By The Use Of Fluid Adsorption Or Reactions (AREA)

Abstract

The invention provides a classifier model transfer learning method, which comprises the following steps: inserting a new sensor array at the end of the life cycle of an original sensor array, and starting to acquire response signals of the original sensor array and the new sensor array; predicting the category information of the new sensor array according to the classification model information of the original sensor array, and calling the category information as the pseudo label of the new sensor array; performing feature extraction on the response signal of the new sensor array, forming a training set together with the pseudo label, and training to obtain a classifier model of the new sensor array; and adjusting the obtained classifier model parameters layer by layer to optimize. The technical scheme of the invention simplifies the process of updating the classifier model after the sensor is replaced, does not interrupt the classification process, does not discard the previous classifier model, and can also realize other standard models as core classifiers.

Description

Classifier model transfer learning method
Technical Field
The invention relates to the technical field of intelligent instruments and meters, in particular to a classifier model migration learning method.
Background
In a widely used situation, damage or poisoning of a sensor may render the sensor or sensors in the array unusable, which is permanent and the element can no longer be operated. The sensors are inevitably replaced during long-term use, and the whole sensor array is usually recalibrated after the sensors are replaced, because the difference of the sensitive layers causes the signals of the replacement elements to be greatly different from the input signals of the original system, and the classification model of the original system cannot be directly used. For these problems, it is necessary to re-model the sensor system. In practical application, in the service cycle of the detection instrument, the identification precision of the classifier (namely, the pattern identification system) in the detection instrument is still in an effective range, so that the classification information of the original system can be fully utilized. The method utilizes the classification model information of the original sensor array to calibrate the new sensor array, simplifies the modeling process of the traditional classifier, and completes the updating of the classification model of the new sensor array by forming a training sample by the pseudo label obtained by predicting the original classifier model and the output signal of the new sensor array.
The system structure of the intelligent instruments is slightly improved, and the method can be applied by adding a row of redundant sensor socket arrays, so that the updating of the classifier model in the system is finally completed.
Disclosure of Invention
In light of the above-mentioned technical problem, a classifier model migration learning method is provided. The invention mainly utilizes a classifier model transfer learning method, which is characterized by comprising the following steps:
step S1: inserting a new sensor array at the end of the life cycle of an original sensor array, and starting to acquire response signals of the original sensor array and the new sensor array;
step S2: predicting the category information of the new sensor array according to the classification model information of the original sensor array, and calling the category information as the pseudo label of the new sensor array;
step S3: performing feature extraction on the response signal of the new sensor array, forming a training set together with the pseudo label, and training to obtain a classifier model of the new sensor array;
step S4: and optimizing the classifier model parameters obtained in the step S3 by adjusting layer by layer.
Further, setting the classifier model of the original sensor array as a three-layer BP neural network, extracting network parameters of the original classifier model, namely a Weight matrix Weight and a threshold matrix Bias, in a coexistence period of two groups of sensor arrays, acquiring a feature vector of a data set by the original sensor array as an input vector X, and calculating to obtain a hidden layer output as follows:
Figure BDA0002464353860000021
the output of the output layer is:
O＝f2(HT·Weight2+Bias2)；
Weight1and Bias1Weight matrix and threshold matrix between input layer and hidden layer, Weight2And Bias2The weight matrix and the threshold matrix between the hidden layer and the output layer. f. of1Excitation function for hidden layer neurons, f2As a function of excitation of neurons in the output layer.
Network structure for setting new classifier model and original classifier modelThe models are completely the same, and the input information before the hidden layer of the original classifier model is not excited is extracted
Figure BDA0002464353860000022
Bias1Expressed as Weight0,X0In the form of 1, the feature vector of the dataset acquired for the new sensor array is taken as a new input vector XnewIn the coexistence period, the gas information sensed by the two groups of sensor arrays is the same, namely, except that the input samples are different, the output of the hidden layer and the output layer are close to the same classification result, so that an overdetermined equation set can be established:
Xnew·W1＝sum1；
obtaining a first layer weight vector W containing bias terms1Then, the new output H of the hidden layer is calculatednew＝f1(Xnew·W1) Extracting input information sum before the output layer of the original classifier model is not excited2＝HT·Weight2+Bias2Establishing an overdetermined equation set:
Hnew·W2＝sum2；
obtaining a second layer weight vector W containing bias terms2(ii) a And building the obtained weight parameters to obtain an optimized new classifier model.
Compared with the prior art, the invention has the following advantages:
the invention is mainly used for intelligent detection instrument equipment. When the sensors of these devices are replaced, since the signals of the sensor elements of the same type may also be greatly different from the signals of the original sensor elements, the recognition effect is reduced after the sensor elements are directly replaced, and the classifier model after the sensor elements are replaced needs to be updated. The technical scheme of the invention has the beneficial effects that: the process of updating the classifier model after the sensor is replaced is simplified, the classification process is not interrupted, the previous classifier model is not discarded, and other standard models can be used as core classifiers.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of the system of the present invention.
FIG. 2 is a flow chart of the classifier model transfer learning algorithm of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1-2, the present invention provides a classifier model migration learning method, which comprises the following steps:
step S1: inserting a new sensor array at the end of the life cycle of the original sensor array, and starting to acquire response signals of the original sensor array and the new sensor array. The present application is implemented by means of the system structure in fig. 1, and a row of sensor array sockets is added on the basis of the original system.
S2: and predicting the class information of the new sensor array through the classification model information of the original sensor array, and calling the class information as the pseudo label of the new sensor array. The classification model information of the original sensor array is the pattern recognition system of the array, which can also be called a classifier, and since the gas information of the original sensor array and the gas information of the new sensor array in the coexistence period are the same, the classifier of the original sensor array can be used for predicting the label of the new sensor array, which is not a standard label and is called a pseudo label.
S3: and performing feature extraction on the response signal of the new sensor array, forming a training set together with the pseudo label, and training to obtain a classifier model of the new sensor array. In the present application, the training set refers to a set of data sets with category information, the response signal of the new sensor array is subjected to feature extraction to be an input sample, the pseudo label is the category information, and the two form the training set, which can be used for training the classifier.
S4: and optimizing the classifier model parameters obtained in the step S3 by adjusting layer by layer. Setting the classifier model of the original sensor array as a three-layer BP neural network, extracting network parameters of the original classifier model, namely a Weight matrix Weight and a threshold matrix Bias, in a coexistence period of two groups of sensor arrays, acquiring a feature vector of a data set by the original sensor array as an input vector X, and calculating to obtain a hidden layer output as follows:
Figure BDA0002464353860000041
the output of the output layer is:
O＝f2(HT·Weight2+Bias2)；
Weight1and Bias1Weight matrix and threshold matrix between input layer and hidden layer, Weight2And Bias2Weight matrix and threshold matrix between the hidden layer and the output layer. f. of1Excitation function for hidden layer neurons, f2As a function of excitation of neurons in the output layer.
Setting the network structure of the new classifier model to be identical to that of the original classifier model, and extracting the input information before the hidden layer of the original classifier model is not excited
Figure BDA0002464353860000051
Bias1Expressed as Weight0,X0In the form of 1, the feature vector of the dataset acquired for the new sensor array is taken as a new input vector XnewIn the coexistence period, the gas information sensed by the two groups of sensor arrays is the same, namely, except that the input samples are different, the output of the hidden layer and the output layer are close to the same classification result, so that an overdetermined equation set can be established:
Xnew·W1＝sum1；
obtaining a first layer weight vector W containing bias terms1Then, the new output H of the hidden layer is calculatednew＝f1(Xnew·W1) Extracting input information sum before the output layer of the original classifier model is not excited2＝HT·Weight2+Bias2Establishing an overdetermined equation set:
Hnew·W2＝sum2；
obtaining a second layer weight vector W containing bias terms2(ii) a And building the obtained weight parameters to obtain an optimized new classifier model. Similarly, other optimization methods, such as genetic algorithms, may also be used as a preferred method for optimizing the model parameters herein.
Example (b):
as a preferred embodiment of the present application, an electronic nose is taken as an example. The system architecture inside the electronic nose is based on an array of independent gas sensors to drive the electronics and appropriate algorithms to achieve gas identification. In practice, however, the sensors operate for a long time, and this ideal situation is still largely unfeasible today. Due to the limited lifetime of the sensors due to unknown dynamic processes in the sensor system (e.g. sensor poisoning or aging), replacement of sensor elements is inevitably required over a long period of time. The working principle of the gas sensor for sensing the gas to be measured mainly depends on the characteristics of the gas sensitive film, and even though the sensors of the same type have different sensitive films, different response values can be generated when the same sample is measured. Therefore, after replacement of the sensor elements, a new sensor array also needs to be re-modeled.
The traditional method for obtaining a sensor classification model is to place a sensor array in a laboratory environment, provide a plurality of different chemicals to the sensor array, and establish a training sample library to train a pattern recognition system. The goal of this training process is to configure the recognition system to produce a unique classification for each chemical in order to achieve automatic recognition. The sensor array has large data quantity and high complexity, so that the traditional method is difficult to realize automation.
After the method is applied, the process of obtaining the model is quick and easy to operate, and the classifier model of the new sensor array is obtained by adopting a model transfer learning mode, so that the experiment cost is reduced. Under the condition of not carrying out laboratory measurement, a new sensor array is inserted at the end of the service cycle of the original sensor array, the classification model of the original sensor array is transferred to the new sensor array, and on the basis, the model parameters are adjusted to optimize the obtained classifier model so as to enable the classifier model to be more suitable for the detection environment. The operation of the common user is simple.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (1)

1. A classifier model transfer learning method is characterized by comprising the following steps:
s1: inserting a new sensor array at the end of the life cycle of an original sensor array, and starting to acquire response signals of the original sensor array and the new sensor array;
s2: predicting the category information of the new sensor array according to the classification model information of the original sensor array, and calling the category information as the pseudo label of the new sensor array;
s3: performing feature extraction on the response signal of the new sensor array, forming a training set together with the pseudo label, and training to obtain a classifier model of the new sensor array;
s4: adjusting the classifier model parameters obtained in the step S3 layer by layer to optimize;
setting the classifier model of the original sensor array as a three-layer BP neural network, extracting network parameters of the original classifier model, namely a Weight matrix Weight and a threshold matrix Bias, in a coexistence period of two groups of sensor arrays, acquiring a feature vector of a data set by the original sensor array as an input vector X, and calculating to obtain a hidden layer output as follows:
Figure FDA0002464353850000011
the output of the output layer is:
O＝f2(HT·Weight2+Bias2)；
among them, Weight1And Bias1Weight matrix and threshold matrix between input layer and hidden layer, Weight2And Bias2A weight matrix and a threshold matrix between the hidden layer and the output layer; f. of1Excitation function for hidden layer neurons, f2As an excitation function for neurons in the output layer;
setting the network structure of the new classifier model to be identical to that of the original classifier model, and extracting the input information before the hidden layer of the original classifier model is not excited
Figure FDA0002464353850000012
Bias1Expressed as Weight0，X01 as a form; the feature vector of the dataset acquired for the new sensor array is taken as a new input vector XnewTwo sets of transmissions due to coexistence periodThe gas information sensed by the sensor array is the same, namely except that the input samples are different, the output of the hidden layer and the output layer are close to the same class result, and accordingly an over-determined equation set can be established:
Xnew·W1＝sum1；
obtaining a first layer weight vector W containing bias terms1Then, the new output H of the hidden layer is calculatednew＝f1(Xnew·W1) Extracting input information sum before the output layer of the original classifier model is not excited2＝HT·Weight2+Bias2Establishing an overdetermined equation set:
Hnew·W2＝sum2；
obtaining a second layer weight vector W containing bias terms2(ii) a And building the obtained weight parameters to obtain an optimized new classifier model.
CN202010329243.5A 2020-04-23 2020-04-23 Classifier model transfer learning method Active CN111539474B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010329243.5A CN111539474B (en) 2020-04-23 2020-04-23 Classifier model transfer learning method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010329243.5A CN111539474B (en) 2020-04-23 2020-04-23 Classifier model transfer learning method

Publications (2)

Publication Number Publication Date
CN111539474A CN111539474A (en) 2020-08-14
CN111539474B true CN111539474B (en) 2022-05-10

Family

ID=71975456

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010329243.5A Active CN111539474B (en) 2020-04-23 2020-04-23 Classifier model transfer learning method

Country Status (1)

Country Link
CN (1) CN111539474B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6935837B1 (en) * 2020-08-20 2021-09-15 トヨタ自動車株式会社 Machine learning device and machine learning system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104268423B (en) * 2014-10-11 2018-03-27 武汉大学 Large scale dynamic evolution Sand-dust type aerosol inversion method
CN105891422B (en) * 2016-04-08 2017-08-25 重庆大学 The electronic nose Gas Distinguishing Method that the limit learns drift compensation is migrated based on source domain
CN107657313B (en) * 2017-09-26 2021-05-18 上海数眼科技发展有限公司 System and method for transfer learning of natural language processing task based on field adaptation
CN108346058A (en) * 2018-01-16 2018-07-31 湖南省中医药研究院 A kind of Manufacture of medicinal slices of TCM Dynamic and Multi dimensional Quality Monitoring Control System and method
CN109829479B (en) * 2019-01-02 2022-06-21 大连理工大学 Automatic classifier model information updating system for sensor and sensor replacing method
CN110263844B (en) * 2019-06-18 2021-04-06 北京中科原动力科技有限公司 Method for online learning and real-time estimation of road surface state
CN110489661B (en) * 2019-07-24 2022-04-26 武汉大学 Social relationship prediction method based on generation of confrontation network and transfer learning

Also Published As

Publication number Publication date
CN111539474A (en) 2020-08-14

Similar Documents

Publication Publication Date Title
EP3696736A1 (en) Device and method for deep learning-based image comparison, and computer program stored in computer-readable recording medium
CN114818579B (en) Analog circuit fault diagnosis method based on one-dimensional convolution long-short-term memory network
CN109583506B (en) Unsupervised image identification method based on parameter transfer learning
CN111699499A (en) Inspection system, image recognition system, recognizer generation system, and learning data generation device
CN111239137B (en) Grain quality detection method based on transfer learning and adaptive deep convolution neural network
CN111539474B (en) Classifier model transfer learning method
CN113516650A (en) Circuit board hole plugging defect detection method and device based on deep learning
CN117074925B (en) 3D chip test analysis method and system
CN113469119A (en) Cervical cell image classification method based on visual converter and graph convolution network
WO2022265292A1 (en) Method and device for detecting abnormal data
CN115797694A (en) Display panel microdefect classification method based on multi-scale twin neural network
CN113052295A (en) Neural network training method, object detection method, device and equipment
CN109829479B (en) Automatic classifier model information updating system for sensor and sensor replacing method
CN114357372A (en) Aircraft fault diagnosis model generation method based on multi-sensor data driving
CN116402777B (en) Power equipment detection method and system based on machine vision
CN116777892B (en) Method and system for detecting dispensing quality based on visual detection
Leng et al. Multi-layer parallel transformer model for detecting product quality issues and locating anomalies based on multiple time‑series process data in Industry 4.0
CN113066049B (en) MEMS sensor defect type identification method and system
CN115688924A (en) Multi-sample combined multivariate self-adaptive regression spline model
CN115496291A (en) Clustering type data augmented meteorological temperature prediction method based on high-precision residual defect value
CN114494211A (en) Wafer-level gas sensor chip detection method based on infrared thermal analysis
CN113033697A (en) Automatic model evaluation method and device based on batch normalization layer
CN113723431A (en) Image recognition method, image recognition device and computer-readable storage medium
CN112268901A (en) Solution concentration identification method based on k-means algorithm
US20210279561A1 (en) Computational processing system, sensor system, computational processing method, and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant