CN110378044A - Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism - Google Patents
Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism Download PDFInfo
- Publication number
- CN110378044A CN110378044A CN201910667918.4A CN201910667918A CN110378044A CN 110378044 A CN110378044 A CN 110378044A CN 201910667918 A CN201910667918 A CN 201910667918A CN 110378044 A CN110378044 A CN 110378044A
- Authority
- CN
- China
- Prior art keywords
- auxiliary variable
- time series
- variable
- time
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention relates to the Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism, soft-measuring technique fields.The following steps are included: 1, determine auxiliary variable and carry out data processing, choose and survey the relevant easy time series for surveying variable as the auxiliary variable of soft-sensing model and acquire auxiliary variable with hardly possible survey parameter of parameter with difficult；Then abnormality value removing is carried out to collected time series；2, the selection of attention mechanism and region-of-interest goes out region-of-interest relative to the difficult time delay for surveying parameter and effective time partition of the scale according to each auxiliary variable；3, the time series of each auxiliary variable is constituted matrix by the input for constructing soft-sensing model, and the region-of-interest of combination attention mechanism determines the input of soft-sensing model；4, timing convolutional neural networks soft-sensing model is established；5, training timing convolutional neural networks soft-sensing model；6, parameter is surveyed to hardly possible using step 5 trained timing convolutional neural networks model and carries out real-time estimation.
Description
Technical field
The present invention relates to the Multiple Time Scales convolutional neural networks flexible measurement methods based on attention mechanism, belong to hard measurement
Technical field.
Background technique
In the modern industrial production process, in order to realize energy conservation and maximizing the benefits, in time to important in production process
Parameter is monitored and control is of great significance.Generally for the Important Parameters in industrial processes, measurement method is main
There are on-line measurement and two kinds of off-line measurement.On-line measurement, which refers to, directly measures parameter using instrument, but equipment price is expensive,
It is difficult in maintenance, and influence of the accuracy of measurement result vulnerable to field working conditions.Off-line measurement refers to the method using off-line check
Parameter is measured, but off-line check generally requires the long period, causes the measurement result obtained offline to production process
There are larger time delays for guidance.Therefore, how real-time estimation difficulty, which surveys parameter, becomes the critical issue that process control first has to solution.
Since the 1990s, soft-measuring technique is fast-developing, be increasingly becoming the effective way that solves the above problems it
One.Soft-measuring technique is a kind of auxiliary variable that utilization is easy to get to establish prediction model and realize and survey the online of parameter to difficult
Real-time estimation, this can provide required important real time information for process monitoring, optimization and control, and then realize energy conservation and benefit
Maximized target.
Summary of the invention
The object of the present invention is to provide a kind of Multiple Time Scales convolutional neural networks hard measurement side based on attention mechanism
Method estimates the difficult real-time online for surveying parameter to realize.
To achieve the goals above, the technical solution adopted by the present invention is that:
Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism, this method content include following step
It is rapid:
Step 1 determines auxiliary variable and carries out data processing
It is preliminary to choose easy survey variable relevant to hardly possible survey parameter as hard measurement mould by the analysis to industrial technology process
The auxiliary variable of type simultaneously acquires auxiliary variable and the difficult time series for surveying parameter；
Then the exceptional value in collected data is rejected using 3 σ criterion, and data is carried out before training
Normalized；
The selection of step 2, attention mechanism and region-of-interest
Take hard attention mechanism, according to auxiliary variable each in process flow relative to the difficult time delay for surveying parameter and it is effective when
Between partition of the scale go out region-of-interest；
Step 3, the input for constructing soft-sensing model
The time series of each auxiliary variable is constituted into matrix, and the region-of-interest of attention mechanism is combined to determine hard measurement mould
The input of type；
Step 4 establishes timing convolutional neural networks soft-sensing model
It determines the initial parameter of timing convolutional neural networks model, and preceding Xiang Xunlian is carried out to network；Wherein initial parameter
The convolution number of plies and the pond number of plies including timing convolutional neural networks, learning rate, the power of each hidden layer, full articulamentum and output layer
Value w and biasing b, the number and size of convolution kernel and Chi Huahe；
Step 5, training timing convolutional neural networks soft-sensing model
It is reversely finely tuned using error and carries out Training, pass through the amendment to error, Improving Working Timing convolutional neural networks
In weight w and biasing b；
Step 6 surveys parameter progress real-time estimation to hardly possible using the trained timing convolutional neural networks model of step 5
Technical solution of the present invention further improvement lies in that: in step 1, using 3 σ criterion processing data outliers when,
Reject the difficult exceptional value for surveying parameter；During rejecting difficult survey parameter exceptional value, to reject the corresponding auxiliary of the exceptional value and become
The time series of amount；
Method particularly includes:
If hardly possible surveys parameter, the sample sequence in different moments is y (k)=(y (0) ..., y (N)), to each of y (k)
Point y (i) is judged, if meeting formula (1), illustrates that the point is that abnormal point need to be removed, while it is corresponding to reject the point
Each auxiliary variable data；
In formula, y is the mean value of variable y (k)；σ is the standard deviation of variable y (k)；
The data of each variable are subjected to Min-Max standardization according to formula (2), are converted into index without dimension assessment
Value makes each variable data be in same number of levels, carries out comprehensive test analysis；
In formula,For the time series after variable i normalization, ximinFor the minimum value of variable i, ximaxThe maximum of variable i
Value.
Technical solution of the present invention further improvement lies in that: in step 2, simultaneously by the analysis to specific industrial technology process
In conjunction with expertise, judge each auxiliary variable relative to the difficult delay parameter for surveying parameter；According to each auxiliary variable in process flow
The length of time of middle effect determines the time scale of each auxiliary variable, thus constructs in each auxiliary variable time series
Region-of-interest；
Method particularly includes:
If a certain sample frequency is fiCertain auxiliary variable sample data in the difficult All Time surveyed in parameter sampling interval T
Sequence is xi(k)=(xi(0),…,xi(Ni- 1)), NiFor the length of the auxiliary variable time series；It is obtained by expertise
The auxiliary variable is Td relative to the difficult time delay probable ranges for surveying parameterimin~Tdimax, when effect a length of Tsimin~Tsimax, then
Time range in the region-of-interest of the auxiliary variableAre as follows:
The then time series in the auxiliary variable time region-of-interestAre as follows:
Technical solution of the present invention further improvement lies in that: in step 3, by the original time series of each auxiliary variable with
And the time series in the concerned period carries out Feature Compression, then by each auxiliary variable time sequence after Feature Compression
Column constitute input of the two-dimentional input matrix as soft-sensing model；
Specific processing method are as follows:
3-1), Feature Compression process:
(1) auxiliary variable All Time sequence signature compression process:
It is x in the difficult All Time sequence for surveying auxiliary variable in parameter sampling interval Ti(k)=(xi(0),…,xi(Ni-
1)), numerical value number is m in the time series after Feature Compression, and the time series after Feature Compression is x 'i(k)=(x 'i
(0),……x′i(m-1))。
Feature Compression degree liAre as follows:
Feature Compression process formula:
(2) time series Feature Compression process in auxiliary variable region-of-interest:
Time series in certain auxiliary variable region-of-interest isFeature pressure
Numerical value number is n in time series after contracting, and the time series after Feature Compression is
Feature Compression degreeAre as follows:
Feature Compression process formula are as follows:
3-2), the input matrix of soft-sensing model is constructed:
(1) the two-dimentional input matrix of the All Time Sequence composition of auxiliary variable are as follows:
In formula,Respectively by All Time sequence x '0With x 'r-1The transposition of constituted vector, m are characterized compression
The numerical value number contained in each auxiliary variable time series afterwards, r are the number of auxiliary variable；
(2) the two-dimentional input matrix that time series is constituted in auxiliary variable region-of-interest are as follows:
In formula,Respectively by time series in region-of-interestWithThe transposition of constituted vector, n are characterized
After compression in each auxiliary variable region-of-interest time series numerical value number, r be auxiliary variable number.
Technical solution of the present invention further improvement lies in that: in step 4, soft-sensing model be multichannel convolutive nerve net
Network, the convolution number of plies and the pond number of plies in each channel, each hidden layer, the weight w of full articulamentum and biasing b, convolution kernel and Chi Huahe
Number and size can be respectively set according to the input data feature in each channel；It is extracted using one-dimensional convolution pond mode in each channel
The feature of each column inputs full articulamentum after finally carrying out Fusion Features to the feature that each channel is extracted；
Wherein, Fusion Features method particularly includes:
Based on the Multiple Time Scales convolutional neural networks model of attention mechanism to the Fusion Features process of each channel characteristics
It is completed in full articulamentum, Fusion Features formula are as follows:
In formula, yk-1For fused full articulamentum,Respectively channel 0, channel i and channel n
Full articulamentum, a are the feature vector corresponding position of full articulamentum, and MAX () is the maximum value for seeking feature.
Technical solution of the present invention further improvement lies in that: in step 5, have the reversed fine tuning of supervision with reference to BP neural network
In reversed error correction algorithms realize hierarchical optimization weight w and biasing b, the reverse train in timing convolutional neural networks is to have
Supervised training.
By adopting the above-described technical solution, the technical effect that the present invention obtains has:
1, the Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism that the present invention establishes can be very
Good solves the problems, such as that surveying parameter to hardly possible carries out real-time estimation.The soft-sensing model of foundation has good generalization ability, not only
Guidance can be provided for operator, also provide prerequisite for subsequent industrial intelligent control.
2, process characteristic of the present invention according to process industry, substantially determines each auxiliary variable relative to difficulty in conjunction with expertise
Survey the time delay and duration of parameter, operand needed for reducing sequential coupling also avoids strong nonlinearity between dependent variable, strong
Time delay and duration are difficult to the awkward situation accurately measured caused by the intervention of coupling, time lag and control system.
3, the introducing of the attention mechanism in the present invention, can be very well in view of wrapping in the long-term sequence of each auxiliary variable
Important notable feature contained in short subsequence containing more information content.Meanwhile so that soft-sensing model can be with reference to each
The global characteristics and local feature of auxiliary variable time series.
4, Feature Compression is carried out to time series in the present invention, efficiently solved between each auxiliary variable because of sample frequency not
With caused by length of time series it is inconsistent, it is difficult to the problem of constructing convolutional neural networks mode input, at the same also avoid because
Same auxiliary variable neighbouring sample point data it is identical and caused by data redundancy.
5, Feature fusion of the invention can merge the feature in each channel very well, and fused feature reference is each logical
Otherness between road feature, while reducing the redundancy between each channel characteristics.
Detailed description of the invention
Fig. 1 is the flow chart of the method for the present invention；
Fig. 2 is the hard measurement scheme that the method for the present invention is applied to after clinker fCao；
Fig. 3 is the multivariable Multiple Time Scales timing convolutional neural networks model based on attention mechanism that the present invention designs
Structure chart；
Fig. 4 is the timing convolution process in a certain channel；
Fig. 5 is binary channels Fusion Features process；
Fig. 6 is the method for the present invention applied to the prediction knot after the training of clinker fCao soft-sensing model in cement production process
Fruit figure.
Specific embodiment
With reference to the accompanying drawing and specific embodiment is described in further details the present invention:
The invention discloses a kind of Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism, the party
Method content, Fig. 1 are the flow chart of measurement method of the present invention.
Method includes the following steps:
Step 1 determines auxiliary variable and carries out data processing
It is preliminary to choose easy survey variable relevant to hardly possible survey parameter as hard measurement mould by the analysis to industrial technology process
The auxiliary variable of type simultaneously acquires auxiliary variable and the difficult time series for surveying parameter；
Then data acquisition is carried out, and the exceptional value of data is rejected using 3 σ criterion, and the logarithm before training
According to being normalized；When using 3 σ criterion processing data outliers, the difficult exceptional value for surveying parameter is rejected；Rejecting difficult survey
During parameter exceptional value, the time series of the corresponding auxiliary variable of the exceptional value is rejected；
Specific features are as follows:
If hardly possible surveys parameter, the sample sequence in different moments is y (k)=(y (0) ..., y (N)), to each of y (k)
Point y (i) is judged, if meeting formula (1), illustrates that the point is that abnormal point need to be removed, while it is corresponding to reject the point
Each auxiliary variable data；
In formula,For the mean value of variable y (k)；σ is the standard deviation of variable y (k)；
The data of each variable are subjected to Min-Max standardization according to formula (2), are converted into index without dimension assessment
Value makes each variable data be in same number of levels, carries out comprehensive test analysis；
In formula,For the time series after variable i normalization, ximinFor the minimum value of variable i, ximaxThe maximum of variable i
Value.
The selection of step 2, attention mechanism and region-of-interest
Take hard attention mechanism, according to auxiliary variable each in process flow relative to the difficult time delay for surveying parameter and it is effective when
Between partition of the scale go out region-of-interest；
Pass through the analysis to specific industrial technology process and combine expertise, judges that each auxiliary variable surveys ginseng relative to hardly possible
The delay parameter of amount；The time of each auxiliary variable is determined according to the length of time that each auxiliary variable acts in process flow
Thus scale constructs the region-of-interest in each auxiliary variable time series；
Method particularly includes:
If a certain sample frequency is fiCertain auxiliary variable sample data in the difficult All Time surveyed in parameter sampling interval T
Sequence is xi(k)=(xi(0),…,xi(Ni- 1)), NiFor the length of the auxiliary variable time series；This is auxiliary by empirical
The general time delay range for helping variable to survey parameter relative to hardly possible is Tdimin~Tdimax, effect duration range is Tsimin~Tsimax, then
Time range in the region-of-interest of the auxiliary variableAre as follows:
The then time series in the auxiliary variable time region-of-interestAre as follows:
Step 3, the input for constructing soft-sensing model
The time series of each auxiliary variable is constituted into matrix, and the region-of-interest of attention mechanism is combined to determine hard measurement mould
The input of type；
The original time series of each auxiliary variable and the time series in the concerned period are subjected to Feature Compression,
Then each auxiliary variable time series after Feature Compression is constituted into two-dimentional input matrix as the input of soft-sensing model；
Specific processing method are as follows:
3-1), Feature Compression process:
(1) auxiliary variable All Time sequence signature compression process:
It is x in the difficult All Time sequence for surveying auxiliary variable in parameter sampling interval Ti(k)=(xi(0),…,xi(Ni-
1)), numerical value number is m in the time series after Feature Compression, and the time series after Feature Compression is x 'i(k)=(x 'i
(0),……x′i(m-1))。
Feature Compression degree liAre as follows:
Feature Compression process formula:
(2) time series Feature Compression process in auxiliary variable region-of-interest:
Time series in certain auxiliary variable region-of-interest isFeature pressure
Numerical value number is n in time series after contracting, and the time series after Feature Compression is
Feature Compression degreeAre as follows:
Feature Compression process formula are as follows:
3-2), the input matrix of soft-sensing model is constructed:
(1) the two-dimentional input matrix of the All Time Sequence composition of auxiliary variable are as follows:
In formula,Respectively by All Time sequence x '0With x 'r-1The transposition of constituted vector, m are characterized compression
The numerical value number contained in each auxiliary variable time series afterwards, r are the number of auxiliary variable；
(3) the two-dimentional input matrix that time series is constituted in auxiliary variable region-of-interest are as follows:
In formula,Respectively by time series in region-of-interestWithThe transposition of constituted vector, n are characterized
After compression in each auxiliary variable region-of-interest time series numerical value number, r be auxiliary variable number.
Step 4 establishes timing convolutional neural networks soft-sensing model
It determines the initial parameter of timing convolutional neural networks model, and preceding Xiang Xunlian is carried out to network；Wherein initial parameter
The convolution number of plies and the pond number of plies including timing convolutional neural networks, learning rate, the power of each hidden layer, full articulamentum and output layer
Value w and biasing b, the number and size of convolution kernel and Chi Huahe；
Soft-sensing model is multichannel convolutive neural network, the convolution number of plies and the pond number of plies in each channel, each hidden layer, Quan Lian
The weight w and biasing b, the number and size of convolution kernel and Chi Huahe for connecing layer can be distinguished according to the input data feature in each channel
Setting；Because the data of each variable have the characteristics that timing, coupling and time lag, each channel uses one-dimensional convolution Chi Huafang
Formula extracts the feature of each column, inputs full articulamentum after carrying out Fusion Features to the feature that each channel is extracted；
Wherein, Fusion Features method particularly includes:
Based on the Multiple Time Scales convolutional neural networks model of attention mechanism to the Fusion Features process of each channel characteristics
It is completed in full articulamentum, Fusion Features formula are as follows:
In formula, yk-1For fused full articulamentum,Respectively channel 0, channel i and channel n
Full articulamentum, a are the feature vector corresponding position of full articulamentum, and MAX () is the maximum value for seeking feature.
Step 5, training timing convolutional neural networks soft-sensing model
It is reversely finely tuned using error and carries out Training, pass through the amendment to error, Improving Working Timing convolutional neural networks
In weight w and biasing b.There is the reversed fine tuning of supervision to realize with reference to the reversed error correction algorithms in BP neural network successively excellent
Change weight w and biasing b, the reverse train in timing convolutional neural networks is Training.
Step 6 surveys parameter progress real-time estimation to hardly possible using the trained timing convolutional neural networks model of step 5.
The present invention provides a kind of Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism, is used for
To the difficult real-time estimation for surveying parameter.This method combination expertise introduces the time series that attention mechanism marks off input variable
In region-of-interest.Compression is carried out by the time series in the All Time sequence to input variable and region-of-interest to solve
Each auxiliary variable because sample frequency difference due to caused by the unmatched problem of length of time series.Finally to attention mechanism module
Output and the output of global module carry out Fusion Features, retain the otherness information removal redundancy in multiple channels.Wherein, more
Being introduced into for time scale can not only be such that soft-sensing model learns to the thickness grain size characteristic in each auxiliary variable time series, also solve
The time-vary delay system problem determined between each variable.In conclusion Qiang Fei in flexible measurement method very good solution industrial process
Linearly, the influence of coupling, large dead time to prediction result, and this method is directly based upon real data, have strong applicability, at
The advantages that this is low, algorithm is simple.
It is to be estimated in real time in measurement method practical application Mr. Yu cement plant actual production to clinker fCaO below
The process of meter.Fig. 2 is the hard measurement scheme that the method for the present invention is applied to after clinker fCao.
Substantially thinking is progress variable selection first, obtains the correlated variables with clinker fCaO according to cement industry analysis,
Determine the time series of soft sensor modeling.In conjunction with expertise, the time delay and duration of each auxiliary variable are determined, determine concern area
Domain carries out Feature Compression and structure to the time series in All Time sequence and region-of-interest comprising each characteristics of variables information
Input matrix is built as in modeling data input binary channels soft-sensing model, the characteristics of according to time series, each channel uses one
The mode of Wei Juanjichiization core extracts feature, and the feature then extracted to each channel carries out Fusion Features and full articulamentum is comprehensive
It closes the fused characteristic information in each channel and passes to output layer.The Multiple Time Scales convolutional Neural based on attention mechanism of building
Network Soft Sensor Model is as shown in Figure 3.Principle, which is reversely finally finely tuned, using the error in BP neural network carries out the ginseng for having supervision
Number fine tuning, completes the building of soft-sensing model.
It specifically measures content and steps are as follows:
Step 1: determining auxiliary variable and carry out data processing
13 variables relevant to clinker fCaO are chosen to the comprehensive analysis of cement industry, it will be in each variable certain period of time
Input of the data sequence as soft-sensing model.
By cement technology it is found that sintering reaction occurs for cement slurry, most afterwards through cold through rotary kiln clinkering zone high-temperature calcination
But the solid particle material obtained is known as clinker, and a small amount of uncombined calcium oxide is contained in solidifying body and is known as free calcium
(fCaO).The excessively high stability that can make cement of free calcium levels declines, too low that cement firing energy consumption is caused to increase, therefore fCaO is needed
It controls within the scope of reasonable.During cement burning, each parameter of clinkering zone has to Guan Chong clinker fCaO content
The effect wanted, therefore firing system parameter is the principal element for realizing clinker fCaO hard measurement.The calorie source of clinkering zone is to decompose
Furnace feeds coal amount, kiln hood feeds coal amount and recycles the Secondary Air into kiln from grate-cooler, and burning zone temperature at this time affects raw material point
The calcium oxide content and generated calcium oxide generated in solution preocess is by other compounds (SiO2、Al2O3、Fe2O3) absorb feelings
Condition.High-temperature blower and kiln hood negative pressure make to generate huge draught head in kiln, guarantee that cement firing system gas circuit is unimpeded, maintain in kiln
Pressure is stablized.Rotary kiln needs kiln motor to provide power in rotating operation, ensure that material chemically reacts equal in rotary kiln
Even property, kiln owner's electromechanics stream is bigger, illustrates that the viscosity of material in kiln is bigger, kiln temperature is higher.Lower pressure comb to a certain extent
The thickness of material on grate-cooler is reacted.
As the above analysis, it chooses and 13 closely related variables of clinker fCaO content: Coaling of Decomposing Furnace,
Pressure, kiln under high-temperature blower revolving speed, decomposition furnace outlet temperature, feeding capacity, kiln end temperature, kiln hood negative pressure, secondary air temperature, two Room are combed
Electric current, kiln hood feed coal amount and three ratio (HM, IM, SM).
Then data acquisition and pretreatment are carried out.
The exceptional value of collected data is rejected using 3 σ criterion, and data are normalized before training
Processing.
If during sinter leaching, sample sequence of the clinker fCaO in different moments is y (k)=(y (0) ..., y
(N)), each of y (k) point y (i) is judged, if meeting formula (1), illustrates that the point is that abnormal point need to be removed,
The time series of the corresponding each auxiliary variable of the point should be rejected simultaneously.
In formula,For the mean value of variable y (k)；σ is the standard deviation of variable y (k)；
Since the dimension of 13 auxiliary variable data is different, then evaluation criterion is also different, in order to unify evaluation criterion,
It then needs the data of 13 auxiliary variables carrying out Min-Max standardization according to formula (2), is converted into index without dimension survey
Value is commented, the comparativity between data is met.When each variable data is in same number of levels, comprehensive test analysis can be carried out.
In formula,For the time series after variable i normalization, ximinFor the minimum value of variable i, ximaxThe maximum of variable i
Value.
Step 2: the selection of attention mechanism and region-of-interest
Hard attention mechanism is combined according to expertise, according to each auxiliary variable relative to the difficult time delay and effect for surveying parameter
Duration substantially marks off region-of-interest.
If a certain sample frequency of sinter leaching process is fiWhole of the auxiliary variable in clinker fCaO sampling interval T when
Between sequence be xi(k)=(xi(0),…,xi(Ni- 1)), NiFor the length of the auxiliary variable time series.By expert in cement plant
The empirical auxiliary variable relative to clinker fCaO general time delay range be Tdimin~Tdimax, act on duration range
For Tsimin~Tsimax, then the time range in the region-of-interest of the auxiliary variableAre as follows:
The then time series in the auxiliary variable time region-of-interestAre as follows:
Decomposition furnace outlet temperature if sample frequency is 12 times/min is complete in clinker fCaO sampling interval 60min
Portion's time series is x (k)=(x (0) ..., x (719)), and expertise obtains decomposition furnace outlet temperature relative to clinker
The general time delay range of fCaO is 57~59min, and useful effect duration is about 10~20min, then the time of the auxiliary variable closes
Infuse regionAre as follows:Time series in region-of-interest
Are as follows:
In this way, remaining 12 auxiliary variable is handled.
Step 3: constructing the input of soft-sensing model
Because in process of production, the sample frequency between each auxiliary variable may be different, need each auxiliary
The original time series of variable and the time series in the concerned period carry out Feature Compression, make after Feature Compression
Each auxiliary variable time series can constitute two-dimentional input matrix.
The sample frequency for comprehensively considering each auxiliary variable in clinker production process determines each auxiliary variable time series
Contain identical numerical value number after Feature Compression, the numerical value number in original time series should be time series after Feature Compression
The integral multiple of interior numerical value number.
The step mainly includes two treatment processes, and Feature Compression process therein is denoted as step 3- for ease of description
1, the process of the input matrix of building soft-sensing model therein is denoted as step 3-2.The following are specific implementation contents.
Step 3-1, Feature Compression process includes following two treatment process:
(1) auxiliary variable All Time sequence signature compression process:
Certain auxiliary variable x in clinker fCaO sampling interval TiAll Time sequence be xi(k)=(xi(0),…,
xi(Ni- 1)), numerical value number is m in the time series after Feature Compression, and the time series after Feature Compression is x 'i(k)=(x 'i
(0),……x′i(m-1))。
Feature Compression degree liAre as follows:
Feature Compression process formula:
(2) time series Feature Compression process in auxiliary variable region-of-interest:
Certain auxiliary variable xiTime series in region-of-interest isFeature
Numerical value number is n in compressed time series, and the time series after Feature Compression is
Feature Compression degreeAre as follows:
Feature Compression process formula are as follows:
Time series carries out feature in All Time sequence and region-of-interest such as to the decomposition furnace outlet temperature in step 3
It compresses, numerical value number is 60 in the All Time sequence after Feature Compression, the numerical value number in region-of-interest in time series
It is 60, the part in All Time sequence is amplified in this way, the time series in region-of-interest is equivalent to, and region-of-interest
It is interior to contain more information.
The All Time sequence of decomposition furnace outlet temperature after Feature Compression are as follows:
X ' (k)=(x ' (0) ... x ' (59))
The time series in decomposition furnace outlet temperature region-of-interest after Feature Compression are as follows:
In this way, remaining 12 auxiliary variable is handled.
Step 3-2, the process for constructing the input matrix of soft-sensing model is as follows:
Because the input requirements of timing convolutional neural networks model are two-dimentional tensors, i.e., input is two-dimensional matrix.We need
It will be respectively by the time series in the All Time sequence and region-of-interest of the clinker fCaO auxiliary variable after Feature Compression
It is built into input of the two-dimensional matrix as soft-sensing model.
(1) the two-dimentional input matrix of the All Time Sequence composition of auxiliary variable are as follows:
In formula,Respectively by All Time sequence x '0With x '12The transposition of constituted vector, in 60 × 13,
60 are characterized the numerical value number contained in each auxiliary variable time series after compression, and 13 be the number for being auxiliary variable；
(4) the two-dimentional input matrix that time series is constituted in auxiliary variable region-of-interest are as follows:
In formula,Respectively by time series in region-of-interestWithThe transposition of constituted vector, 60 × 13
In, 60 are characterized the numerical value number of time series in each auxiliary variable region-of-interest after compression, and 13 be the number for being auxiliary variable.
Step 4: establishing timing convolutional neural networks soft-sensing model
The Multiple Time Scales convolutional neural networks model structure based on attention mechanism in the present invention as shown in figure 3,
The soft-sensing model is binary channels convolutional neural networks, the convolution number of plies and the pond number of plies in each channel, each hidden layer, full articulamentum
Weight w and biasing b, the number and size of convolution kernel and Chi Huahe can be respectively set according to the input data feature in each channel.
Each channel is all made of the feature that one-dimensional convolution pond mode extracts each column, the full connection of feature input extracted to each channel
Layer, while binary channels Fusion Features are carried out at full articulamentum.
Strong coupling between variable each in the production process of clinker results in the time delay between variable and is difficult really
It is fixed, synchronization, different variables data may not have relevance, and each variable is different to the difficult influence degree for surveying parameter.
Using the convolution pond mode of one-dimensional, both avoided between each auxiliary variable and between each auxiliary variable and clinker fCaO
Time lag ambiguity problem, and operand needed for largely reducing sequential coupling, avoiding may in sequential coupling
Caused by characteristic information lose.The convolution pond mode in a certain channel is as shown in figure 4, the k-1 layer in the channel is through multiple convolution
The feature vector obtained later with pondization, the input as full articulamentum.The input of the full articulamentumWith outputBetween
Relationship such as following formula.
In formula,The weight of full articulamentum and biasing in respectively channel i.
Since information is extracted with otherness and redundancy in each channel, need to extract each channel otherness information and
Redundancy information carries out Fusion Features, retains otherness information and removes redundancy information.More time rulers based on attention mechanism
Degree convolutional neural networks model completes the Fusion Features process of each channel characteristics in full articulamentum.
The binary channels Fusion Features mode that the present invention uses is as shown in figure 5, Fusion Features formula are as follows:
In formula, yk-1For fused full articulamentum,Respectively the full articulamentum in channel 0, channel 1, a are complete
The feature vector corresponding position of articulamentum, MAX () are the maximum value for seeking feature.
In order to avoid over-fitting, regularization method is used before the network model output layer --- lose data
(Dropout) technology achievees the purpose that promote network model generalization ability.As shown in kth layer in Fig. 5, timing convolutional Neural net
Network output layer is summed using linear weighted function and directly calculates the difficult value for surveying parameter.Then the layer inputs xkWith the calculating between output valve y'
Formula are as follows:
Y'=wkxk+bk (12)
W in formulakAnd bkFor weight and the biasing for not being output layer.
Step 5: training timing convolutional neural networks soft-sensing model
The input matrix constructed in step 4 is input in soft-sensing model.It is reversed using the error in BP neural network
It finely tunes principle and carries out Training, the weight w and biasing b by the amendment to error, in Improving Working Timing convolutional neural networks.
Step 6: parameter being surveyed to hardly possible using step 5 trained timing convolutional neural networks model and carries out real-time estimation.
The method of the present invention is applied to the prediction result in cement production process after the training of clinker fCaO soft-sensing model such as
Shown in Fig. 6.
Embodiment described above only describe the preferred embodiments of the invention, not to model of the invention
It encloses and is defined, without departing from the spirit of the design of the present invention, those of ordinary skill in the art are to technical side of the invention
The various changes and improvements that case is made should all be fallen into the protection scope that claims of the present invention determines.
Claims (6)
1. the Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism, it is characterised in that: this method content
The following steps are included:
Step 1 determines auxiliary variable and carries out data processing
It is preliminary to choose easy survey variable relevant to hardly possible survey parameter as soft-sensing model by the analysis to industrial technology process
Auxiliary variable simultaneously acquires auxiliary variable and the difficult time series for surveying parameter；
Then the exceptional value in collected time series is rejected using 3 σ criterion, and data is carried out before training
Normalized；
The selection of step 2, attention mechanism and region-of-interest
Hard attention mechanism is taken, according to auxiliary variable each in process flow relative to the difficult time delay and effective time ruler for surveying parameter
Degree marks off region-of-interest；
Step 3, the input for constructing soft-sensing model
The time series of each auxiliary variable is constituted into matrix, and the region-of-interest of attention mechanism is combined to determine soft-sensing model
Input；
Step 4 establishes timing convolutional neural networks soft-sensing model
It determines the initial parameter of timing convolutional neural networks model, and preceding Xiang Xunlian is carried out to network；Wherein initial parameter includes
The convolution number of plies and the pond number of plies of timing convolutional neural networks, learning rate, the weight w of each hidden layer, full articulamentum and output layer
With biasing b, the number and size of convolution kernel and Chi Huahe；
Step 5, training timing convolutional neural networks soft-sensing model
It is reversely finely tuned using error and carries out Training, by the amendment to error, in Improving Working Timing convolutional neural networks
Weight w and biasing b；
Step 6 surveys parameter progress real-time estimation to hardly possible using the trained timing convolutional neural networks model of step 5.
2. the Multiple Time Scales convolutional neural networks flexible measurement method according to claim 1 based on attention mechanism,
It is characterized in that: in step 1, when using 3 σ criterion processing data outliers, rejecting the difficult exceptional value for surveying parameter；Rejecting difficult survey
During parameter exceptional value, the time series of the corresponding auxiliary variable of the exceptional value is rejected；
Method particularly includes:
If hardly possible surveys parameter, the sample sequence in different moments is y (k)=(y (0) ..., y (N)), to each of y (k) point y
(i) judged, if meeting formula (1), illustrate that the point is that abnormal point need to be removed, while it is corresponding each auxiliary to reject the point
Help variable data；
In formula,For the mean value of variable y (k)；σ is the standard deviation of variable y (k)；
The data of each variable are subjected to Min-Max standardization according to formula (2), index without dimension assessment value is converted into, makes
Each variable data is in same number of levels, carries out comprehensive test analysis；
In formula,For the time series after variable i normalization, ximinFor the minimum value of variable i, ximaxThe maximum value of variable i.
3. the Multiple Time Scales convolutional neural networks flexible measurement method according to claim 1 based on attention mechanism,
It is characterized in that: in step 2, passing through the analysis to specific industrial technology process and combine expertise, judge each auxiliary variable phase
For the difficult delay parameter for surveying parameter；Each auxiliary is determined according to the length of time that each auxiliary variable acts in process flow
Thus the time scale of variable constructs the region-of-interest in each auxiliary variable time series；
Method particularly includes:
If a certain sample frequency is fiCertain auxiliary variable sample data in the difficult All Time sequence surveyed in parameter sampling interval T
For xi(k)=(xi(0),…,xi(Ni- 1)), NiFor the length of the auxiliary variable time series；Become by the empirical auxiliary
Measuring relative to the difficult time delay range for surveying parameter is Tdimin~Tdimax, effect duration range is Tsimin~Tsimax, then the auxiliary becomes
Time range in the region-of-interest of amountAre as follows:
The then time series in the auxiliary variable time region-of-interestAre as follows:
4. the Multiple Time Scales convolutional neural networks flexible measurement method according to claim 1 based on attention mechanism,
It is characterized in that: in step 3, the original time series of each auxiliary variable and the time series in the concerned period being carried out
Then each auxiliary variable time series after Feature Compression is constituted two-dimentional input matrix as hard measurement mould by Feature Compression
The input of type；
Specific processing method are as follows:
3-1), Feature Compression process:
(1) auxiliary variable All Time sequence signature compression process:
It is x in the difficult All Time sequence for surveying auxiliary variable in parameter sampling interval Ti(k)=(xi(0),…,xi(Ni- 1)), special
Levying numerical value number in compressed time series is m, and the time series after Feature Compression is x 'i(k)=(x 'i(0),……x′i
(m-1))。
Feature Compression degree liAre as follows:
Feature Compression process formula:
(2) time series Feature Compression process in auxiliary variable region-of-interest:
Time series in certain auxiliary variable region-of-interest isAfter Feature Compression
Time series in numerical value number be n, the time series after Feature Compression is
Feature Compression degreeAre as follows:
Feature Compression process formula are as follows:
3-2), the input matrix of soft-sensing model is constructed:
(1) the two-dimentional input matrix of the All Time Sequence composition of auxiliary variable are as follows:
In formula,Respectively by All Time sequence x '0With x 'r-1The transposition of constituted vector, m are characterized after compression respectively
The numerical value number contained in auxiliary variable time series, r are the number of auxiliary variable；
(2) the two-dimentional input matrix that time series is constituted in auxiliary variable region-of-interest are as follows:
In formula,Respectively by time series in region-of-interestWithThe transposition of constituted vector, n are characterized compression
Afterwards in each auxiliary variable region-of-interest time series numerical value number, r be auxiliary variable number.
5. the Multiple Time Scales convolutional neural networks flexible measurement method according to claim 1 based on attention mechanism,
Be characterized in that: in step 4, soft-sensing model be multichannel convolutive neural network, the convolution number of plies and the pond number of plies in each channel, respectively
Hidden layer, the weight w of full articulamentum and biasing b, the number and size of convolution kernel and Chi Huahe can be according to the input numbers in each channel
It is respectively set according to feature；The feature of each column is extracted in each channel using one-dimensional convolution pond mode, is finally extracted to each channel
Feature carry out Fusion Features after input full articulamentum；
Wherein, Fusion Features method particularly includes:
Based on the Multiple Time Scales convolutional neural networks model of attention mechanism to the Fusion Features process of each channel characteristics complete
Articulamentum is completed, Fusion Features formula are as follows:
In formula, yk-1For fused full articulamentum,Respectively the complete of channel 0, channel i and channel n connects
Layer is connect, a is the feature vector corresponding position of full articulamentum, and MAX () is the maximum value for seeking feature.
6. the Multiple Time Scales convolutional neural networks flexible measurement method according to claim 1 based on attention mechanism,
It is characterized in that: in step 5, thering is the reversed fine tuning of supervision to realize with reference to the reversed error correction algorithms in BP neural network successively excellent
Change weight w and biasing b, the reverse train in timing convolutional neural networks is Training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910667918.4A CN110378044B (en) | 2019-07-23 | 2019-07-23 | Multi-time scale convolution neural network soft measurement method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910667918.4A CN110378044B (en) | 2019-07-23 | 2019-07-23 | Multi-time scale convolution neural network soft measurement method based on attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378044A true CN110378044A (en) | 2019-10-25 |
| CN110378044B CN110378044B (en) | 2021-06-11 |
Family
ID=68255212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910667918.4A Active CN110378044B (en) | 2019-07-23 | 2019-07-23 | Multi-time scale convolution neural network soft measurement method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378044B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807428A (en) * | 2019-11-05 | 2020-02-18 | 南方科技大学 | Coal sample identification method and device, server and storage medium |
| CN111222798A (en) * | 2020-01-13 | 2020-06-02 | 湖南师范大学 | Soft measurement method for key indexes of complex industrial process |
| CN111242068A (en) * | 2020-01-17 | 2020-06-05 | 科大讯飞(苏州)科技有限公司 | Behavior recognition method and device based on video, electronic equipment and storage medium |
| CN111506835A (en) * | 2020-04-17 | 2020-08-07 | 北京理工大学 | Data feature extraction method fusing user time features and individual features |
| CN112215351A (en) * | 2020-09-21 | 2021-01-12 | 浙江大学 | Enhanced multi-scale convolution neural network soft measurement method |
| CN112270996A (en) * | 2020-11-13 | 2021-01-26 | 南京信息工程大学 | Classification method for multivariable medical sensing data stream |
| CN112488392A (en) * | 2020-12-01 | 2021-03-12 | 重庆邮电大学 | Intelligent water affair daily water consumption prediction method based on machine learning |
| CN112686898A (en) * | 2021-03-15 | 2021-04-20 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN113850931A (en) * | 2021-11-29 | 2021-12-28 | 武汉大学 | Flight feature extraction method for flight abnormity |
| CN114994294A (en) * | 2022-05-20 | 2022-09-02 | 燕山大学 | Soft measurement method for free calcium of cement clinker based on attention and window gating mechanism |
| US11681914B2 (en) | 2020-05-08 | 2023-06-20 | International Business Machines Corporation | Determining multivariate time series data dependencies |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160337702A1 (en) * | 2015-05-12 | 2016-11-17 | Rovi Guides, Inc. | Methods and systems for recommending supplemental media assets based on recently mentioned media references |
| CN107015541A (en) * | 2017-04-26 | 2017-08-04 | 燕山大学 | The flexible measurement method being combined based on mutual information and least square method supporting vector machine |
| CN108197743A (en) * | 2017-12-31 | 2018-06-22 | 北京化工大学 | A kind of prediction model flexible measurement method based on deep learning |
| CN108845072A (en) * | 2018-07-06 | 2018-11-20 | 南京邮电大学 | A kind of dynamic soft-measuring method of the 4-CBA content based on convolutional neural networks |
| CN109147878A (en) * | 2018-10-08 | 2019-01-04 | 燕山大学 | A kind of clinker free calcium flexible measurement method |
| CN109919188A (en) * | 2019-01-29 | 2019-06-21 | 华南理工大学 | Timing classification method based on sparse local attention mechanism and convolution echo state network |
-
2019
- 2019-07-23 CN CN201910667918.4A patent/CN110378044B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160337702A1 (en) * | 2015-05-12 | 2016-11-17 | Rovi Guides, Inc. | Methods and systems for recommending supplemental media assets based on recently mentioned media references |
| CN107015541A (en) * | 2017-04-26 | 2017-08-04 | 燕山大学 | The flexible measurement method being combined based on mutual information and least square method supporting vector machine |
| CN108197743A (en) * | 2017-12-31 | 2018-06-22 | 北京化工大学 | A kind of prediction model flexible measurement method based on deep learning |
| CN108845072A (en) * | 2018-07-06 | 2018-11-20 | 南京邮电大学 | A kind of dynamic soft-measuring method of the 4-CBA content based on convolutional neural networks |
| CN109147878A (en) * | 2018-10-08 | 2019-01-04 | 燕山大学 | A kind of clinker free calcium flexible measurement method |
| CN109919188A (en) * | 2019-01-29 | 2019-06-21 | 华南理工大学 | Timing classification method based on sparse local attention mechanism and convolution echo state network |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807428A (en) * | 2019-11-05 | 2020-02-18 | 南方科技大学 | Coal sample identification method and device, server and storage medium |
| CN110807428B (en) * | 2019-11-05 | 2023-05-30 | 南方科技大学 | Coal sample identification method, device, server and storage medium |
| CN111222798A (en) * | 2020-01-13 | 2020-06-02 | 湖南师范大学 | Soft measurement method for key indexes of complex industrial process |
| CN111222798B (en) * | 2020-01-13 | 2023-04-07 | 湖南师范大学 | Complex industrial process key index soft measurement method |
| CN111242068A (en) * | 2020-01-17 | 2020-06-05 | 科大讯飞(苏州)科技有限公司 | Behavior recognition method and device based on video, electronic equipment and storage medium |
| CN111506835B (en) * | 2020-04-17 | 2022-12-23 | 北京理工大学 | Data feature extraction method fusing user time features and individual features |
| CN111506835A (en) * | 2020-04-17 | 2020-08-07 | 北京理工大学 | Data feature extraction method fusing user time features and individual features |
| US11681914B2 (en) | 2020-05-08 | 2023-06-20 | International Business Machines Corporation | Determining multivariate time series data dependencies |
| CN112215351A (en) * | 2020-09-21 | 2021-01-12 | 浙江大学 | Enhanced multi-scale convolution neural network soft measurement method |
| CN112270996B (en) * | 2020-11-13 | 2023-04-25 | 南京信息工程大学 | Classification method for multi-variable medical sensing data flow |
| CN112270996A (en) * | 2020-11-13 | 2021-01-26 | 南京信息工程大学 | Classification method for multivariable medical sensing data stream |
| CN112488392A (en) * | 2020-12-01 | 2021-03-12 | 重庆邮电大学 | Intelligent water affair daily water consumption prediction method based on machine learning |
| CN112686898B (en) * | 2021-03-15 | 2021-08-13 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN112686898A (en) * | 2021-03-15 | 2021-04-20 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN113850931A (en) * | 2021-11-29 | 2021-12-28 | 武汉大学 | Flight feature extraction method for flight abnormity |
| CN114994294A (en) * | 2022-05-20 | 2022-09-02 | 燕山大学 | Soft measurement method for free calcium of cement clinker based on attention and window gating mechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378044B (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378044A (en) | Multiple Time Scales convolutional neural networks flexible measurement method based on attention mechanism | |
| CN109147878B (en) | Soft measurement method for free calcium of cement clinker | |
| CN103336107B (en) | Soft measurement method for f-CaO content of cement clinker | |
| CN112001527A (en) | Industrial production process target data prediction method of multi-feature fusion deep neural network | |
| CN110782067B (en) | Sintering end point fluctuation range prediction method based on fuzzy information granulation | |
| CN104914723A (en) | Industrial process soft measurement modeling method based on cooperative training partial least squares model | |
| CN109342703B (en) | Method and system for measuring content of free calcium in cement clinker | |
| CN103116306A (en) | Automatic stepping type ordered time interval dividing method | |
| Li et al. | Dynamic time features expanding and extracting method for prediction model of sintering process quality index | |
| CN103439889B (en) | Based on the burning through point forecasting method of discharge flue gas analysis | |
| CN102004444A (en) | Multi-model predictive control method for component content in process of extracting rare earth | |
| Zheng et al. | Modeling and optimization of the cement calcination process for reducing NO x emission using an improved just-in-time Gaussian mixture regression | |
| CN112906976A (en) | Cement clinker free calcium sample data enhancement and prediction method based on R-WGAN | |
| CN106093134A (en) | The compensation method of metal oxide sensor array response drift | |
| Pathan et al. | Efficient forecasting of precipitation using LSTM | |
| CN103344740B (en) | Based on the glutamic acid production concentration online soft sensor method of multi input Wiener model | |
| Zhou et al. | A new Co/Co $ _2 $ prediction model based on labeled and unlabeled process data for sintering process | |
| CN110222825A (en) | A kind of finished cement specific surface area prediction technique and system | |
| CN114968997A (en) | Sintering end point forecasting method based on space-time feature extraction | |
| Hao et al. | R-WGAN-based multitimescale enhancement method for predicting f-CaO cement clinker | |
| Zheng et al. | Just-in-time learning for cement free lime prediction with empirical mode decomposition and database monitoring index | |
| CN103390103A (en) | Melt index online detection method based on subspace independent component regression model | |
| Yan et al. | Estimation of copper concentrate grade for copper flotation | |
| Yu et al. | Research on soft sensing of cement clinker f-CaO based on LS_SVM and burning zone temperature | |
| CN112365935A (en) | Cement free calcium soft measurement method based on multi-scale depth network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |