CN110085327A - Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism - Google Patents

Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism Download PDF

Info

Publication number
CN110085327A
CN110085327A CN201910256008.7A CN201910256008A CN110085327A CN 110085327 A CN110085327 A CN 110085327A CN 201910256008 A CN201910256008 A CN 201910256008A CN 110085327 A CN110085327 A CN 110085327A
Authority
CN
China
Prior art keywords
data
neural network
indicate
multichannel
epidemic situation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910256008.7A
Other languages
Chinese (zh)
Inventor
郝建业
侯韩旭
马钰
付博峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dongguan University of Technology
Original Assignee
Dongguan University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dongguan University of Technology filed Critical Dongguan University of Technology
Priority to CN201910256008.7A priority Critical patent/CN110085327A/en
Publication of CN110085327A publication Critical patent/CN110085327A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/80—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for detecting, monitoring or modelling epidemics or pandemics, e.g. flu

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Epidemiology (AREA)
  • Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Primary Health Care (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The present invention provides the multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism, belongs to epidemic disease monitoring technical field.The present invention first pre-processes data intensive data, standardizes, feature selecting, and the data of selection are divided into two class of weather dependent data and Influenza epidemic situation related data, generate training set；Then the multichannel LSTM neural network model including attention mechanism is established；Training set data is inputted the model to be trained, and carries out MAPE assessment, obtains trained multichannel LSTM neural network model；Test data is handled, test set is obtained；Test set data are inputted in trained LSTM neural network model and are tested；Inverse standardization finally is carried out to test output result, obtains Influenza epidemic situation predicted value.The present invention solves the problems, such as that existing Influenza epidemic situation Predicting Technique predictablity rate is lower.The present invention can be used for the influenza prediction of different zones.

Description

Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism
Technical field
The present invention relates to Influenza epidemic situation prediction techniques, belong to epidemic disease monitoring technical field.
Background technique
Influenza is a kind of acute respiratory infection as caused by influenza virus.After patient catches an illness, it is most likely that add Weight primary disease, causes secondary bacterial pneumonia and chronic heart and lung diseases etc..The outburst of Influenza epidemic situation has seasonality, because And social fear may be caused, there is larger impact (D.N.T.How, C.K.Loo, and to human health and social stability K.S.M. Sahari.Behavior recognition for humanoid robots using long short-term memory.International Journal of Advanced Robotic Systems,13(6): 1729881416663369,2016.).For example, the H1N1 type influenza of outburst in 2009, just causes in the First Year of Epidemic outbreak of disease 151,700 to 575,400 human mortality of the whole world (S.Yang, M. Santillana, and S.C.Kou.Accurate estimation of influenza epidemics using google search data via argo.Proceedings of the National Academy of Sciences,112(47):14473–14478, 2015.).Therefore, the accurate of Influenza epidemic situation, real time monitoring and early warning have critically important practical meaning to Public Hygienic Prevention department Justice.Influenza epidemic disease monitoring and warning system can provide epidemiology information for Ministry of Public Health's door, and public health department is helped to shift to an earlier date Epidemic preventing working is carried out, then coordinates each medical institutions in area and takes corresponding counter-measure (J.S.Brownstein and K.D.Mandl.Reengineering real time outbreak detection systems for influenza epidemic monitoring.In AMIA Annual Symposium Proceedings,volume 2006,page 866.American Medical Informatics Association,2006.)。
Influenza-like case (Influenza-like-illness, abbr.ILI) is the World Health Organization (World Health Organization, abbr.WHO) formulate acute respiratory infection discrimination standard.The symptom of influenza-like case is In metainfective 10 days, 38 DEG C of fever or more, and with cough (W.H.Organization et al.Who interim global epidemiological surveillance standards for influenza.Geneva:World Health Organization,pages 1–61,2012.).Our prediction target is ILI%, and calculation method is to make a definite diagnosis The ratio of influenza-like case number and medical total number of persons.Field is monitored in Influenza epidemic situation, whether ILI% is usually as judging influenza The index of outburst.When ILI% is more than certain threshold value, show that influenza season has arrived, relevant department is reminded to carry out health in time Epidemic preventing working.
In recent years, more and more scholars were conceived to accurate real time monitoring, early monitoring and the epidemic situation early warning of Influenza epidemic situation Research.By using information in web search or social network sites, such as Twitter, Google Correlate etc., stream Sense epidemic prediction have become research direction that academia more pays close attention to ([H.Achrekar, A.Gandhe, R.Lazarus, S.-H.Yu,and B.Liu. Predicting flu trends using twitter data.In Computer Communications Workshops(INFOCOM WKSHPS),2011IEEE Conference on,pages 702– 707.IEEE,2011.]、[D.A.Broniatowski,M.J. Paul,and M.Dredze.National and local influenza surveillance through twitter:an analysis of the 2012-2013influenza epidemic.PloS one,8(12):e83672,2013.]、[G.E.Hinton and R.R. Salakhutdinov.Reducing the dimensionality of data with neural networks.science, 313(5786):504–507,2006.]).Previous research method is typically based on some common Linear model, for example, least absolute shrinkage and selection operator (LASSO lasso trick algorithm) or ([the D.A. Broniatowski, M.J.Paul, and such as penalized regression penalized regression M.Dredze.National and local influenza surveillance through twitter: an analysis of the 2012-2013influenza epidemic.PloS one,8(12):e83672,2013.]、 [M.Santillana,E. O.Nsoesie,S.R.Mekaru,D.Scales,and J.S.Brownstein.Using clinicians’search query data to monitor influenza epidemics.Clinical infectious diseases:an official publication of the Infectious Diseases Society of America,59(10):1446,2014.]、[M.Santillana,D.W.Zhang,B.M.Althouse, and J.W.Ayers.What can digital disease detection learn from(an external Revision to) googleflu trends? American journal of preventive medicine, 47 (3): 341–347,2014.]).There are also some researchers to solve Influenza epidemic situation forecasting problem using the method for deep learning ([H.Hu,H.Wang,F.Wang,D.Langley,A.Avram,and M.Liu.Prediction of influenza-like illness based on the improved artificial tree algorithm and artificial neural network.Scientific reports,8(1):4895,2018.]、[Q.Xu,Y.R.Gel,L.L.R.Ramirez, K.Nezafati,Q.Zhang,and K.-L.Tsui.Forecasting influenza in hong kong with google search queries and statistical model fusion.PloS one,12(5):e0176690, 2017.]).However, these methods can not relatively accurately predict the change of Influenza epidemic situation (influenza-like case ratio ILI%) Change.Firstly, the data acquired in internet are inaccurate, feature needed for lacking Accurate Prediction Influenza epidemic situation.Using on line The result of data prediction can not accurately reflect the variation tendency of Influenza epidemic situation.Secondly, Influenza epidemic situation data complicated composition, lead to Often there is stronger noise, data diversity is strong, and traditional linear method cannot make full use of information in various dimensions input data. Again, in the deep learning method proposed before, the temporal aspect of Influenza epidemic situation data is not accounted for；Therefore it is badly in need of one kind The high Influenza epidemic situation Predicting Technique of accuracy rate.
Summary of the invention
The present invention is to solve the problems, such as that existing Influenza epidemic situation Predicting Technique predictablity rate is lower, is provided based on attention The multichannel LSTM neural network Influenza epidemic situation prediction technique of mechanism.
Multichannel LSTM neural network Influenza epidemic situation prediction technique of the present invention based on attention mechanism, by following Technical solution is realized:
Step 1: being pre-processed, being standardized to data intensive data；Then the mode based on model sequence is used to carry out The data of selection are divided into two class of weather dependent data and Influenza epidemic situation related data, generate training set by feature selecting；
Step 2: establishing the multichannel LSTM neural network model including attention mechanism；Multichannel LSTM nerve net The input of network model includes Influenza epidemic situation related data and weather dependent data；
It is trained Step 3: training set data is inputted the multichannel LSTM neural network model, and carries out MAPE Assessment, obtains trained multichannel LSTM neural network model；
Step 4: carrying out processing identical with step 1 to test data, test set is obtained；
It is tested Step 5: test set data are inputted in trained LSTM neural network model；
Step 6: carrying out inverse standardization to test output result, Influenza epidemic situation predicted value is obtained.
Present invention feature the most prominent and significant beneficial effect are:
Multichannel LSTM neural network Influenza epidemic situation prediction technique according to the present invention based on attention mechanism considers To the temporal aspect of Influenza epidemic situation data, based on shot and long term Memory Neural Networks；By designing multi-channel structure, The timing information in data is preferably extracted, not only ensure that in bottom-layer network, be independent of each other between different types of data, Also ensure the fusion of the multi-dimensional data in high-level network；And the present invention is further by addition Attention mechanism Improve the accuracy rate of prediction；To targetedly solve influenza forecasting problem.Emulation experiment shows that the method for the present invention is tested As a result accuracy rate is apparently higher than other conventional methods, can provide ILI% (influenza-like case ratio) accurately and effectively real-time Prediction.
Detailed description of the invention
Fig. 1 is the method for the present invention flow chart；
Fig. 2 is LSTM memory unit structure schematic diagram in the present invention；
Fig. 3 is Attention schematic diagram of mechanism in the present invention；
Fig. 4 is Attention-based Multi-channel LSTM model schematic in the embodiment of the present invention；
Fig. 5 is the true value and predicted value contrast curve chart of the Att-MCLSTM in the embodiment of the present invention；
Fig. 6 is the true value and predicted value contrast curve chart of the MCLSTM in the embodiment of the present invention；
Fig. 7 is the true value and predicted value contrast curve chart of the LSTM in the embodiment of the present invention；
Fig. 8 is the true value and predicted value contrast curve chart of the RNN in the embodiment of the present invention；
1. input gate, 2. out gates, 3. forget door, 4. self-loopa neurons.
Specific embodiment
Specific embodiment 1: be illustrated in conjunction with Fig. 1 to present embodiment, present embodiment provide based on attention The multichannel LSTM neural network Influenza epidemic situation prediction technique of mechanism, specifically includes the following steps:
Step 1: being pre-processed (Preprocessing), standardization (Normalization) to data intensive data； Then it uses the mode (model-based ranking) based on model sequence to carry out feature selecting, the data of selection is divided into Two class of weather dependent data and Influenza epidemic situation related data generates training set；
Step 2: establishing the multichannel LSTM neural network model including attention mechanism (Attention)；Here should Model is named as Att-MCLSTM, i.e., Attention-based multi-channel LSTM is (based on the more of attention mechanism Channel LSTM neural network).Multichannel LSTM neural network model and traditional Recognition with Recurrent Neural Network based on attention mechanism RNN (Recurrent Neural Network) is different, and LSTM neural network can solve the problem of gradient disappears (S.Hochreiter and J.Schmidhuber.Long short-term memory.Neural computation,9 (8):1735–1780,1997.)；The memory module of LSTM memory unit remains with the sequence information of Input context, in timing In terms of data processing, more traditional RNN has better effect.Multichannel LSTM neural network model (Att-MCLSTM) Input includes Influenza epidemic situation related data and weather dependent data；
It is instructed Step 3: step 1 treated training set data is inputted the multichannel LSTM neural network model Practice, and carry out MAPE assessment, obtains trained multichannel LSTM neural network model；
Step 4: carrying out processing identical with step 1 to test data, test set is obtained；
It is tested Step 5: test set data are inputted in trained LSTM neural network model；
Step 6: carry out inverse standardization in order to reconstruct initial data to test output result, it is pre- to obtain Influenza epidemic situation Measured value.
Present embodiment solves Influenza epidemic situation forecasting problem using the method for deep neural network.In view of Influenza epidemic situation number According to temporal aspect, present embodiment using shot and long term memory LSTM (Long-short term memory) neural network 0 make For basic prediction technique.Since the input data of different dimensions has different features, if cannot be filled using single network structure Divide the temporal aspect extracted in multiple data dimensions.It, can be preferably by designing multichannel (Multi-channel) structure Extract the timing information in data.It not only ensure that in bottom-layer network, be independent of each other between different types of data, also guarantee In the high-level network fusion of multi-dimensional data.Multi-level LSTM neural network model has very strong capability of fitting, this hair The bright accuracy rate that prediction is further increased by adding Attention mechanism；In Attention layers, numerical value in output sequence Probability of occurrence depend on list entries in numerical value.Attention structure allows model preferably to handle different zones Relationship between input data.
Specific embodiment 2: the present embodiment is different from the first embodiment in that, it is standardized described in step 1 Specifically include following procedure:
Min-Max standardization (also referred to as deviation standardization) will be carried out by pretreated data:
Wherein, x is the numerical value in pretreated data, xminIt is the minimum value in pretreated data, xmaxIt is pre- Maximum value in data that treated, y are value of the x after Min-Max standardization；After data normalization, data Value is all by scaling between 0 and 1.
Other steps and parameter are same as the specific embodiment one.
Specific embodiment 3: present embodiment is unlike specific embodiment two, as shown in Fig. 2, in step 2 LSTM memory unit in the multichannel LSTM neural network model includes input gate 1, out gate 2, forgets door 3 and follow certainly Ring neuron 4；The structure door structure control of LSTM memory unit transmission of the data in LSTM memory unit, including difference Data transmission between unit and the data transmission inside unit.1 control unit state renewal process of input gate, out gate 2 are controlled Whether the output sequence for making the unit can change the memory state of other units, retain or forget with forgeing 3 property of can choose of door State before.
LSTM memory unit can be indicated with following equation group:
Wherein, σ () is logistic sigmoid function (by between variable mappings to 0~1), and tanh () is hyperbolic Tangent function, σ () and tanh () are used as activation primitive；itFor the input door state of t moment, ftFor the forgetting of t moment Door state, otFor the output door state of t moment, ctIt indicates t moment location mode (activation vector), htWhen indicating t The hidden state (hidden vector) at quarter；WxiIndicate the weight matrix between input gate, input data；WhiExpression input gate, Hide (hidden-input gate) weight matrix of interlayer；WciIndicate the weight matrix between input gate, unit；WxfIt indicates to lose Forget the weight matrix between door, input data；WhfIndicate the weight matrix forgotten door, hide interlayer；WcfIt indicates to forget door, unit Between weight matrix；WxcIndicate the weight matrix between unit, input data；WhcThe weight matrix for indicating unit, hiding interlayer； WxoIndicate the weight matrix between out gate, input data；WhoThe weight matrix for indicating out gate, hiding interlayer；WcoIndicate defeated It gos out, the weight matrix between unit；xtIndicate the input value of t moment, biIndicate input gate bias term；bfIt indicates to forget door biasing ?；bcIndicate location mode bias term；boIndicate out gate bias term.
Other steps and parameter are the same as one or two specific embodiments.
Specific embodiment 4: present embodiment is unlike specific embodiment one, two or three, described in step 2 Attention mechanism can indicate are as follows:
mi=tanh (Wcmc+Wymyi) (3)
ai=∝ exp (< wm,mi>) (4)
Z=∑i ai yi (6)
Wherein, attention layer inputs n parameter y1,…,ynWith context sequence c, output vector z；I=1 ..., n；WcmIt is Context sequence weights matrix, WymIndicate input vector weight matrix；When vector z is given context sequence c, yiWeighting count Average value；Sequence miIndicate c and yiPolymerization, be calculated by tanh layers；∝ indicates in direct ratio；wmExpression is normalized Weight matrix when exponential function softmax (i.e. formula (4))；aiIndicate m in the case where given context sequence ci's Softmax result.
List entries is usually encoded to the vector of a fixed length by traditional encoding and decoding structure.However, this structure exists Some defects.When list entries is longer, this structure is difficult study to suitable vector characteristic manner.Attention mechanism Basic thought be to break traditional encoding and decoding structure, by using the intermediate result training pattern of LSTM encoder, realize choosing Learn to selecting property the information in list entries.Therefore there are relevances between output sequence and list entries, i.e., every in output sequence The probability that a numerical value occurs depends on the numerical value in list entries.
Fig. 3 is Attention schematic diagram of mechanism.Attention layers of calculating y1,…,ynWeight distribution, StIt is LSTM layers In the state at t moment, input includes Attention layers of output.LSTM layers of output sequence ..., xt-1,xt... } and in number The probability of occurrence of value depends on list entries { y1,…,yn}。
Other steps and parameter are identical as specific embodiment one, two or three.
Specific embodiment 5: present embodiment is unlike specific embodiment four, MAPE described in step 3 (mean absolute percentage) assessment specifically:
Wherein, MAPE is mean absolute percentage error,Indicate i-th of true value, piIndicate i-th of predicted value； MAPE numerical value is lower, indicates that the accuracy of model is higher.
Other steps and parameter are identical as specific embodiment four.
Specific embodiment 6: present embodiment is unlike specific embodiment five, it is defeated to test described in step 6 Result carries out inverse standardization out specifically:
Wherein,For Influenza epidemic situation predicted value, q is test output result；qmaxIndicate the maximum value in test output result, qminIndicate the minimum value in test output result.
Other steps and parameter are identical as specific embodiment five.
Embodiment
Beneficial effects of the present invention are verified using following embodiment:
The Influenza epidemic situation data that the present embodiment uses Guangzhou disease control epidemic prevention center to acquire are as data set.Wherein wrap The Influenza epidemic situation data for 9 area 2009-2017 that Guangzhou has under its command are included, data set includes 6 modules, and each module has more A dimension.Data record includes 52 weeks (week) data as unit of week every year.By data prediction, standardization and feature Selection；Feature selecting is carried out by the way of based on model sequence (model-based ranking)；Remove data set every time In a dimension, remaining all dimensions are inputted in identical prediction model, the output result of comparison model.If pre- The accuracy for surveying result is lower, illustrates that removed data dimension is more related to prediction target.By all prediction results by accurate Degree sequence, to select and prediction higher 19 dimensions of the target degree of correlation.Selected dimension and its explanation are listed in table 1, In do not list basic information module (including temporal information, region and population).
1. module of table and the description of selected dimension
The dimension data of above-mentioned selection is divided into two classes, respectively weather dependent data and Influenza epidemic situation related data.It Gas related data packets include temperature on average, the highest temperature, the lowest temperature, rainfall, air pressure, relative humidity, remaining dimension is classified as flowing Feel epidemic situation related data.All areas weather dependent data having the same and different Influenza epidemic situation related datas weekly；Cause This, the correspondence of multichannel LSTM neural network model includes two channels: influenza related channel program and weather related channel program.
As Fig. 4 illustrates Attention-based Multi-channel LSTM (being abbreviated as Att-MCLSTM) model Overall structure.Firstly, handling influenza epidemic disease using the network (LSTM 1 ..., LSTM 9) that one group of LSTM Neural memory unit forms The data of feelings related data, each region (District 1st, District 1nd ... District 9th) are separately input to In one LSTM Neural memory unit；Weather dependent data is handled with a LSTM neural network (LSTM 10) simultaneously.In order to The information extracted in different zones Influenza epidemic situation related data is combined, the output of all LSTM neural networks in first part Fusion in the fused layer (Merge 1) of layer on it.Although this group of LSTM neural network is extracted the Influenza epidemic situation in each region Information, but still need to assign weighted value to the intermediate output sequence of extracted information.Because of the Influenza epidemic situation information of different zones There is different influences to Guangzhou entirety influenza epidemiological situation.Therefore, the intermediate output sequence of LSTM neural network is successively Pass through attention layer (Attention) and full articulamentum (Dense 1).Hereafter, in higher (fused layer Merge 2) network Middle this two parts data of fusion.Finally, extracting after two layers of full articulamentum (Dense 2, Dense 3) and having merged multidimensional Spend the information of input data.
(1) selection of input data length
Use the data in continuous how many weeks that the prediction result in next week can be made to reach optimal.In order to test experiment Data verified, data are divided into training set and test set two parts by us.All experimental results are 10 repetitions The average value of experiment.
Input data length is respectively set to 6 weeks, 8 weeks, 10 weeks, 12 weeks and 14 weeks.Attention-based Each layer parameter setting of Multi-channel LSTM neural network is as shown in table 2.Activation primitive is linear activation primitive, loses letter Number is mape, and optimizer uses adam；
Each layer parameter setting of 2. neural network of table
The title of neural net layer Units
LSTM 1,…,LSTM 9 32
LSTM 10 32
Dense 1 16
Dense 2 10
Dense 3 1
Using the training of preceding 370 weeks data, remaining data are tested.Every data record include weather dependent data and The Influenza epidemic situation related data in 9 regions.Weather dependent data has 6 dimensions, and Influenza epidemic situation related data has 13 dimensions. Weather dependent data (Climate data) is input in weather related channel program (Climate-related channel), each The Influenza epidemic situation data in region are input in corresponding influenza related channel program (Influenza-related channel).Model Prediction result it is as shown in table 3:
The MAPE value of each prediction result of table 3.
Week time number MAPE
6 0.107
8 0.092
10 0.086
12 0.106
14 0.109
From table 3 it is observed that prediction effect is best when input data length is 10.This illustrates continuous 10 weeks data It can fully reflect the temporal aspect of Influenza epidemic situation data.If input data length is too short, cannot fully reflect The temporal characteristics of data；If input data length is too long, the noise of data will increase.In the following experiment, Wo Menxuan Continuous 10 weeks data are selected as input data.
(2) recruitment evaluation
Attention is verified by comparing the prediction result of Att-MCLSTM and MCLSTM (multichannel LSTM neural network) The validity of mechanism.The setting of neural network parameter and data entry device are as described in Table 1.
Secondly, by comparing the validity of the prediction result of MCLSTM and LSTM neural network verifying multi-channel structure. The setting of the neural network parameter of MCLSTM and data entry device are identical as step (1).LSTM neural network uses one LSTM layers receive all inputs.By the corresponding addition of the Influenza epidemic situation related data in all areas same week, with weather dependent data Collectively as input, thus every data record includes 19 dimensions.LSTM layers of output result is after a full articulamentum Output.The neuron number of LSTM layers and full articulamentum is respectively 32 and 1.
Finally, it was demonstrated that the effect of LSTM neural network is better than common RNN network.The setting of neural network parameter and data are defeated It is as described above to enter method.
The MAPE value of above-mentioned four kinds of methods is illustrated in table 4.As can be seen from the table, Att-MCLSTM can be obtained optimal As a result.In table 4 front two row statistics indicate that, MAPE value is increased to 0.086 from 0.105 by addition Attention mechanism. This illustrates that Attention structure allows model preferably to learn to handle the relationship between different zones input data.In table 4 Second, third row statistics indicate that, MAPE value is increased to 0.105 from 0.118 by design multi-channel structure.This explanation is more Channel design can preferably extract the temporal aspect of a variety of input datas.LSTM neural network and common RNN neural network are pre- The MAPE value for surveying result is respectively 0.118 and 0.132.This illustrates that the more common RNN neural network of LSTM neural network can be more The temporal aspect in input data is handled well.Influenza epidemic situation data are also demonstrated simultaneously with timing.
The MAPE value of each prediction result of table 4.
Method MAPE
The method of the present invention Att-MCLSTM 0.086
MCLSTM 0.105
LSTM 0.118
RNN 0.132
The true value and predicted value of four kinds of methods are respectively shown in Fig. 5, Fig. 6, Fig. 7, Fig. 8.It can be seen from the figure that this The prediction ILI% and true ILI% (Actual ILI%) of inventive method (Att-MCLSTM) are closest.Other three kinds of methods Predicted value and true value between have more apparent difference.It is demonstrated experimentally that the method for the present invention can accurately, when fully extracting The hidden feature of ordinal number evidence provides accurate Influenza epidemic situation prediction.
The present invention can also have other various embodiments, without deviating from the spirit and substance of the present invention, this field Technical staff makes various corresponding changes and modifications in accordance with the present invention, but these corresponding changes and modifications should all belong to In the protection scope of the appended claims of the present invention.

Claims (6)

1. the multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism, which is characterized in that specifically include Following steps:
Step 1: being pre-processed, being standardized to data intensive data；The ranking for being then based on model carries out feature selecting, will The data of selection are divided into two class of weather dependent data and Influenza epidemic situation related data, generate training set；
Step 2: establishing the multichannel LSTM neural network model including attention mechanism；Multichannel LSTM neural network mould The input of type includes Influenza epidemic situation related data and weather dependent data；
It is trained Step 3: training set data is inputted the multichannel LSTM neural network model, and carries out MAPE assessment, Obtain trained multichannel LSTM neural network model；
Step 4: carrying out processing identical with step 1 to test data, test set is obtained；
It is tested Step 5: test set data are inputted in trained LSTM neural network model；
Step 6: carrying out inverse standardization to test output result, Influenza epidemic situation predicted value is obtained.
2. the multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism according to claim 1, It is characterized in that, standardization described in step 1 specifically includes following procedure:
Min-Max standardization will be carried out by pretreated data:
Wherein, x is the numerical value in pretreated data, xminIt is the minimum value in pretreated data, xmaxIt is pretreatment The maximum value in data afterwards, y are value of the x after Min-Max standardization.
3. the multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism according to claim 2, Be characterized in that, the LSTM memory unit in the neural network model of multichannel LSTM described in step 2 include input gate, out gate, Forget door and self-loopa neuron；LSTM memory unit can be indicated with following equation group:
Wherein, σ () is logistic sigmoid function (by between variable mappings to 0~1), and tanh () is tanh Function；itFor the input door state of t moment, ftFor the forgetting door state of t moment, otFor the output door state of t moment, ctIndicate t Moment location mode, htIndicate the hidden state of t moment；WxiIndicate the weight matrix between input gate, input data；WhiIndicate defeated Introduction, the weight matrix for hiding interlayer；WciIndicate the weight matrix between input gate, unit；WxfIt indicates to forget door, input data Between weight matrix；WhfIndicate the weight matrix forgotten door, hide interlayer；WcfIt indicates to forget the weight matrix between door, unit； WxcIndicate the weight matrix between unit, input data；WhcThe weight matrix for indicating unit, hiding interlayer；WxoExpression out gate, Weight matrix between input data；WhoThe weight matrix for indicating out gate, hiding interlayer；WcoIndicate the power between out gate, unit Weight matrix；xtIndicate the input value of t moment, biIndicate input gate bias term；bfIt indicates to forget door bias term；bcIndicate cell-like State bias term；boIndicate out gate bias term.
4. according to claim 1, the 2 or 3 multichannel LSTM neural network Influenza epidemic situation prediction side based on attention mechanism Method, which is characterized in that attention mechanism described in step 2 can indicate are as follows:
mi=tanh (Wcmc+Wymyi) (3)
ai=∝ exp (< wm, mi>) (4)
Z=∑iaiyi (6)
Wherein, attention layer inputs n parameter y1..., ynWith context sequence c, output vector z；I=1 ..., n；WcmIt is context sequence Column weight matrix, WymIndicate input vector weight matrix；When vector z is given context sequence c, yiWeighted arithmetic mean value； Sequence miIndicate c and yiPolymerization；∝ indicates in direct ratio；wmIndicate weight matrix when being normalized；aiIt indicates given M in the case where context sequence ciNormalized result.
5. the multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism according to claim 4, It is characterized in that, the assessment of MAPE described in step 3 specifically:
Wherein, MAPE is mean absolute percentage error,Indicate i-th of true value, piIndicate i-th of predicted value.
6. the multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism according to claim 5, It is characterized in that, a pair test output result described in step 6 carries out inverse standardization specifically:
Wherein,For Influenza epidemic situation predicted value, q is test output result；qmaxIndicate the maximum value in test output result, qmin Indicate the minimum value in test output result.
CN201910256008.7A 2019-04-01 2019-04-01 Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism Pending CN110085327A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910256008.7A CN110085327A (en) 2019-04-01 2019-04-01 Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910256008.7A CN110085327A (en) 2019-04-01 2019-04-01 Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism

Publications (1)

Publication Number Publication Date
CN110085327A true CN110085327A (en) 2019-08-02

Family

ID=67413942

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910256008.7A Pending CN110085327A (en) 2019-04-01 2019-04-01 Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism

Country Status (1)

Country Link
CN (1) CN110085327A (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111553394A (en) * 2020-04-20 2020-08-18 中国长江三峡集团有限公司 Reservoir water level prediction method based on cyclic neural network and attention mechanism
CN111724897A (en) * 2020-06-12 2020-09-29 电子科技大学 Motion function data processing method and system
CN111798991A (en) * 2020-07-09 2020-10-20 重庆邮电大学 LSTM-based method for predicting population situation of new coronary pneumonia epidemic situation
CN111882157A (en) * 2020-06-24 2020-11-03 东莞理工学院 Demand prediction method and system based on deep space-time neural network and computer readable storage medium
CN111933300A (en) * 2020-09-28 2020-11-13 平安科技（深圳）有限公司 Epidemic situation prevention and control effect prediction method, device, server and storage medium
CN111968755A (en) * 2020-08-21 2020-11-20 上海海洋大学 Epidemic situation prediction model based on LSTM deep learning network model
CN112164471A (en) * 2020-09-17 2021-01-01 吉林大学 New crown epidemic situation comprehensive evaluation method based on classification regression model
CN112201361A (en) * 2020-09-01 2021-01-08 浙江大学山东工业技术研究院 COVID-19 epidemic situation prediction method based on LSTM model
CN112582074A (en) * 2020-11-02 2021-03-30 吉林大学 Bi-LSTM and TF-IDF based new crown epidemic situation prediction and analysis method
CN112946187A (en) * 2021-01-22 2021-06-11 西安科技大学 Refuge chamber real-time state monitoring method based on neural network
CN113161004A (en) * 2020-07-15 2021-07-23 泰康保险集团股份有限公司 Epidemic situation prediction system and method
CN113314231A (en) * 2021-05-28 2021-08-27 北京航空航天大学 Infectious disease propagation prediction system and device integrating spatio-temporal information
CN113380420A (en) * 2020-10-13 2021-09-10 深圳云天励飞技术股份有限公司 Epidemic situation prediction method and device, electronic equipment and storage medium
CN113434989A (en) * 2021-06-28 2021-09-24 山东大学 Pipe network leakage amount prediction method and system based on attention mechanism and LSTM
CN113744888A (en) * 2021-09-02 2021-12-03 深圳万海思数字医疗有限公司 Regional epidemic trend prediction early warning method and system
CN114983352A (en) * 2021-03-01 2022-09-02 浙江远图互联科技股份有限公司 Method and device for identifying new coronary pneumonia based on attention mechanism
CN115393678A (en) * 2022-08-01 2022-11-25 北京理工大学 Multi-modal data fusion decision-making method based on image type intermediate state
CN115631869A (en) * 2022-11-28 2023-01-20 北京理工大学 Construction method of infectious disease prediction model

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130097103A1 (en) * 2011-10-14 2013-04-18 International Business Machines Corporation Techniques for Generating Balanced and Class-Independent Training Data From Unlabeled Data Set
CN108648829A (en) * 2018-04-11 2018-10-12 平安科技（深圳）有限公司 Disease forecasting method and device, computer installation and readable storage medium storing program for executing
CN109431507A (en) * 2018-10-26 2019-03-08 平安科技（深圳）有限公司 Cough disease identification method and device based on deep learning
CN109493933A (en) * 2018-08-08 2019-03-19 浙江大学 A kind of prediction meanss of the adverse cardiac events based on attention mechanism

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130097103A1 (en) * 2011-10-14 2013-04-18 International Business Machines Corporation Techniques for Generating Balanced and Class-Independent Training Data From Unlabeled Data Set
CN108648829A (en) * 2018-04-11 2018-10-12 平安科技（深圳）有限公司 Disease forecasting method and device, computer installation and readable storage medium storing program for executing
CN109493933A (en) * 2018-08-08 2019-03-19 浙江大学 A kind of prediction meanss of the adverse cardiac events based on attention mechanism
CN109431507A (en) * 2018-10-26 2019-03-08 平安科技（深圳）有限公司 Cough disease identification method and device based on deep learning

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111553394A (en) * 2020-04-20 2020-08-18 中国长江三峡集团有限公司 Reservoir water level prediction method based on cyclic neural network and attention mechanism
CN111724897A (en) * 2020-06-12 2020-09-29 电子科技大学 Motion function data processing method and system
CN111724897B (en) * 2020-06-12 2022-07-01 电子科技大学 Motion function data processing method and system
CN111882157A (en) * 2020-06-24 2020-11-03 东莞理工学院 Demand prediction method and system based on deep space-time neural network and computer readable storage medium
CN111798991A (en) * 2020-07-09 2020-10-20 重庆邮电大学 LSTM-based method for predicting population situation of new coronary pneumonia epidemic situation
CN111798991B (en) * 2020-07-09 2022-09-02 重庆邮电大学 LSTM-based method for predicting population situation of new coronary pneumonia epidemic situation
CN113161004A (en) * 2020-07-15 2021-07-23 泰康保险集团股份有限公司 Epidemic situation prediction system and method
CN113161004B (en) * 2020-07-15 2023-11-10 泰康保险集团股份有限公司 Epidemic situation prediction system and method
CN111968755A (en) * 2020-08-21 2020-11-20 上海海洋大学 Epidemic situation prediction model based on LSTM deep learning network model
CN112201361A (en) * 2020-09-01 2021-01-08 浙江大学山东工业技术研究院 COVID-19 epidemic situation prediction method based on LSTM model
CN112164471B (en) * 2020-09-17 2022-05-24 吉林大学 New crown epidemic situation comprehensive evaluation method based on classification regression model
CN112164471A (en) * 2020-09-17 2021-01-01 吉林大学 New crown epidemic situation comprehensive evaluation method based on classification regression model
WO2021139336A1 (en) * 2020-09-28 2021-07-15 平安科技（深圳）有限公司 Epidemic prevention and control effect prediction method and apparatus, and server and storage medium
CN111933300A (en) * 2020-09-28 2020-11-13 平安科技（深圳）有限公司 Epidemic situation prevention and control effect prediction method, device, server and storage medium
CN111933300B (en) * 2020-09-28 2021-02-12 平安科技（深圳）有限公司 Epidemic situation prevention and control effect prediction method, device, server and storage medium
CN113380420A (en) * 2020-10-13 2021-09-10 深圳云天励飞技术股份有限公司 Epidemic situation prediction method and device, electronic equipment and storage medium
CN113380420B (en) * 2020-10-13 2023-10-17 深圳云天励飞技术股份有限公司 Epidemic situation prediction method and device, electronic equipment and storage medium
CN112582074B (en) * 2020-11-02 2022-10-18 吉林大学 Bi-LSTM and TF-IDF based new crown epidemic situation prediction and analysis method
CN112582074A (en) * 2020-11-02 2021-03-30 吉林大学 Bi-LSTM and TF-IDF based new crown epidemic situation prediction and analysis method
CN112946187A (en) * 2021-01-22 2021-06-11 西安科技大学 Refuge chamber real-time state monitoring method based on neural network
CN114983352A (en) * 2021-03-01 2022-09-02 浙江远图互联科技股份有限公司 Method and device for identifying new coronary pneumonia based on attention mechanism
CN113314231A (en) * 2021-05-28 2021-08-27 北京航空航天大学 Infectious disease propagation prediction system and device integrating spatio-temporal information
CN113314231B (en) * 2021-05-28 2022-04-22 北京航空航天大学 Infectious disease propagation prediction system and device integrating spatio-temporal information
CN113434989A (en) * 2021-06-28 2021-09-24 山东大学 Pipe network leakage amount prediction method and system based on attention mechanism and LSTM
CN113744888A (en) * 2021-09-02 2021-12-03 深圳万海思数字医疗有限公司 Regional epidemic trend prediction early warning method and system
CN113744888B (en) * 2021-09-02 2023-09-22 深圳万海思数字医疗有限公司 Regional epidemic trend prediction and early warning method and system
CN115393678A (en) * 2022-08-01 2022-11-25 北京理工大学 Multi-modal data fusion decision-making method based on image type intermediate state
CN115393678B (en) * 2022-08-01 2024-04-02 北京理工大学 Multi-mode data fusion decision method based on image intermediate state
CN115631869A (en) * 2022-11-28 2023-01-20 北京理工大学 Construction method of infectious disease prediction model

Similar Documents

Publication Publication Date Title
CN110085327A (en) Multichannel LSTM neural network Influenza epidemic situation prediction technique based on attention mechanism
Izakian et al. Anomaly detection and characterization in spatial time series data: A cluster-centric approach
Yan et al. An improved method for the fitting and prediction of the number of covid-19 confirmed cases based on lstm
Said et al. Predicting COVID-19 cases using bidirectional LSTM on multivariate time series
Yan et al. An improved belief entropy in evidence theory
Men et al. Estimate the incubation period of coronavirus 2019 (COVID-19)
Lawson et al. Bayesian disease mapping for public health
Bellamy et al. Analysis of clustered and interval censored data from a community‐based study in asthma
CN111477337B (en) Infectious disease early warning method, system and medium based on individual self-adaptive transmission network
Wang et al. Predicting the epidemics trend of COVID-19 using epidemiological-based generative adversarial networks
CN111081379B (en) Disease probability decision method and system thereof
Young et al. Zero‐inflated modeling part I: Traditional zero‐inflated count regression models, their applications, and computational tools
de Leon et al. General mixed‐data model: Extension of general location and grouped continuous models
Oğuz et al. Determination of Covid-19 possible cases by using deep learning techniques
Sun et al. Time pattern reconstruction for classification of irregularly sampled time series
Kao et al. Early prediction of coronavirus disease epidemic severity in the contiguous United States based on deep learning
Wiencierz et al. Restricted likelihood ratio testing in linear mixed models with general error covariance structure
CN113161004A (en) Epidemic situation prediction system and method
CN115565669B (en) Cancer survival analysis method based on GAN and multitask learning
Banyal et al. Technology landscape for epidemiological prediction and diagnosis of covid-19
CN113616209B (en) Method for screening schizophrenic patients based on space-time attention mechanism
CN115376638A (en) Physiological characteristic data analysis method based on multi-source health perception data fusion
Jang et al. Long-term influenza outbreak forecast using time-precedence correlation of web data
Abu-Dalbouh et al. Predictive data mining rule-based classifiers model for novel coronavirus (COVID-19) infected patients’ recovery in the Kingdom of Saudi Arabia
Kumar et al. Predictive analysis of novel coronavirus using machine learning model-a graph mining approach

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190802