CN116362430B

CN116362430B - Flight delay prediction method and system based on online increment MHHA-SRU

Info

Publication number: CN116362430B
Application number: CN202310644337.5A
Authority: CN
Inventors: 屈景怡; 解文凯; 徐浩源
Original assignee: Civil Aviation University of China
Current assignee: Civil Aviation University of China
Priority date: 2023-06-02
Filing date: 2023-06-02
Publication date: 2023-08-01
Anticipated expiration: 2043-06-02
Also published as: CN116362430A

Abstract

The invention belongs to the technical field of traffic delay prediction methods, and discloses a flight delay prediction method and a flight delay prediction system based on online increment MHHA-SRU. Setting initial parameters of an online incremental MHHA-SRU network model, preprocessing, fusing and encoding historical flights and meteorological data, and inputting the preprocessed historical flights and meteorological data into the online incremental MHHA-SRU network model for training to obtain a flight delay prediction model based on an online incremental MHHA-SRU network; training a model by using the online accumulated data and generating new model weights to realize online updating of the model; inputting the flight information to be detected into a flight delay prediction model, and extracting features by using an online incremental MHHA-SRU network; the classification of the flight delay level is realized by processing the flight information through a prediction structure; and realizing the display of the flight information and the real-time prediction display of the flight delay state in a visual system. The invention effectively improves the accuracy of delay prediction.

Description

Flight delay prediction method and system based on online increment MHHA-SRU

Technical Field

The invention belongs to the technical field of traffic delay prediction, and particularly relates to a flight delay prediction method and system based on online increment MHHA-SRU.

Background

Since the implementation of electronic tickets in civil aviation industry, informationized front is carried out in the transportation industry, and in the aviation transportation system of China, a large amount of data is available at any moment, and the sources and forms of the data are various. However, the currently emerging informatization and big data techniques are not fully applied, which results in that data resources are not efficiently used, resulting in waste of resources. The civil aviation industry provides great convenience for the travel of passengers and also drives the development of a plurality of industries, but the problem of flight delay is not solved effectively, which is worry. Flight delay is a key index for measuring the operation efficiency of an aviation traffic system, and accurate forecasting of the flight delay can help related departments to make effective countermeasures early, so that adverse effects caused by the flight delay are reduced. Flight delay problems have long plagued regulatory personnel, airlines, airports, and passengers. For the air management department, flight delay can increase the working time of the management personnel and make it difficult to reasonably allocate the flights. For airlines, there are mainly ground delay losses, air delay losses, passenger accommodation and catering costs, and costs for temporarily deploying the aircraft.

Early researches on flight delay prediction have focused on statistical methods and machine learning methods, such as genetic algorithms, bayesian methods, C4.5 decision trees, random forests, support vector machines, etc.; with the continuous improvement of computer computing power, the study on the flight delay is gradually transited to a deep learning model method, and the models such as a deep confidence network, a MobileNet V2, a space-time separable graph convolution space-time propagation network (STPN), a long-short time memory network (LSTM) and the like are used for carrying out the study on the flight delay prediction. Along with the great progress and development of the deep learning algorithm, the attention mechanism is free from the advent of the deep learning algorithm, and a great breakthrough is made in the fields of image processing, natural language processing and the like. The relevant scholars also start to add an attention mechanism into the flight delay prediction model, so that the model can focus on a part with relatively large influence on delay prediction results in the data set during feature extraction, and the overall prediction performance of the model is improved. However, the simple cyclic unit (SRU) does not have a structure of attention mechanism, and the feature importance cannot be effectively extracted. In addition, the use of deep learning for flight delay prediction can achieve higher accuracy, but most of the deep neural networks are trained by an offline learning mode, and the training mode cannot continuously integrate new information into an already constructed model, so that a potential obsolete model can be caused, and the offline learning has poor expandability and is not suitable for being applied to big data, particularly streaming data.

Through the above analysis, the problems and defects existing in the prior art are as follows: in the prior art, the accuracy rate of delay of flights is low, and data cannot be effectively utilized; the offline learning has poor expandability, and is not suitable for being applied to big data, especially streaming data; application and deployment practicality is poor.

Disclosure of Invention

In order to overcome the problems in the related art, the disclosed embodiments of the present invention provide a flight delay prediction method and system based on online incremental MHHA-SRU, and in particular, relate to a flight delay prediction method based on online incremental MHHA-SRU (Multi-Head Hierarchical Attention mechanism based on Simple Recurrent Units) network.

The technical scheme is as follows: the online increment based MHHA-SRU flight delay prediction method comprises the following steps:

s1, setting initial parameters of an online incremental MHHA-SRU network and initial parameters of a training mode;

s2, acquiring historical flight data and meteorological data, performing data cleaning, data labeling, data fusion, data encoding and data serialization, and dividing the data into a plurality of data subsets containing a number of samples; sequentially inputting the constructed data subsets into an online incremental MHHA-SRU network after initial parameters are set for training to obtain a flight delay prediction model based on the online incremental MHHA-SRU network;

S3, inputting the flight information data stream to be predicted into a flight delay prediction model based on an online incremental MHHA-SRU network, and extracting features of the flight information to be predicted through the online incremental MHHA-SRU network after initial parameters are set;

s4, obtaining the probability of the flights to be predicted belonging to each category through the prediction structure according to the extracted features, and obtaining delay categories of the flights to be predicted based on the probability;

s5, using an incremental learning algorithm based on regularization to send the online accumulated flight information data stream into an online incremental MHHA-SRU network for training, and updating parameters of the online incremental MHHA-SRU network;

s6, inputting the flight information data stream to be predicted into an on-line updated MHHA-SRU network for feature extraction, and obtaining a model updated delay class prediction result through a prediction structure;

and S7, carrying out visual analysis and display on the flight information data stream to be predicted and delay class statistics of the flights to be predicted on a front-end interface.

In step S1, initial parameters of the online incremental MHHA-SRU network include: the number of network layers of the SRU primary encoder and the SRU secondary encoder, the number of hidden layer nodes of the SRU primary encoder and the SRU secondary encoder, the number of total connection layer layers of the Softmax, the number of nodes of each layer of total connection layer of the Softmax, the number of attention heads and the sequence length;

Initial parameters of the training regimen include: learning rate during training, optimizer during training, EWC regular term penalty coefficientSI regularization term penalty coefficientSI damping coefficient.

In step S2, converting the historical flight data and weather data into an online mode for use, dividing the data into data subsets, and simulating a process of continuously inputting online data streams into a system;

in the data cleaning process, deleting the missing data and the data containing abnormal items in a direct deleting mode; filling data with missing values in flights in a mode of proper filling of approximate terms;

in the data labeling, processing is respectively carried out on take-off or landing flights, delay grades are divided according to calculated take-off or landing delay time, and delay grade labels are labeled for the data;

in the data fusion, the planned departure time in the flight data is used for each airport departure flight to be matched and fused with the record time in the meteorological data of the airport; matching and fusing the planned landing time in the using flight data and the recording time in the meteorological data of the airport aiming at each airport landing flight;

processing the continuous type and the discrete type characteristics in the data coding, and using Min-Max coding for the continuous type characteristics; the Catboost code is used for discrete features;

Ordering data sets according to predicted departure time in data serializationThe use length isDividing the window of the input data, converting the input data into a time sequence to generate a time sequence data set;

in the process of data segmentation into subsets containing a number of samples, each data subset is called a task, and each subset is respectively sent to an online incremental MHHA-SRU network for training.

In step S2, the built data subset is sequentially input into an online incremental MHHA-SRU network after initial parameters are set for training, to obtain a flight delay prediction model based on the online incremental MHHA-SRU network, which comprises the following steps:

sequentially inputting the constructed data subsets to an SRU primary encoder for feature extraction; the multi-head attention mechanism module calculates the attention distribution of the input features to obtain a weight coefficient matrix with the value between 0 and 1, and continuously learns the weight coefficient matrix in the training process to represent the influence degree of different features on the result, wherein the larger the weight is, the larger the influence of the features on the result is; then, the SRU secondary encoder is used for carrying out feature extraction on the output result of the primary encoder, and the different importance of each sample in the data subset is learned through a multi-head attention mechanism, wherein the importance is also reflected in a weight coefficient matrix, and the larger the weight coefficient is, the larger the influence of the sample on the prediction result is indicated; inputting the output of the secondary encoder to a Softmax full-connection layer to obtain a predicted value of each level; and performing loss calculation by utilizing a cross entropy loss function added with a regular term and improved by online increment learning, and iterating the input data subset to complete training, thereby obtaining the flight delay prediction model based on the online increment MHHA-SRU network.

In one embodiment, in the performing loss calculation by using the cross entropy loss function added with the regularization term improved by online incremental learning and iterating the input data subset to complete training, training of the model is achieved by using an incremental learning algorithm based on regularization, and the incremental learning algorithm comprises: the line increment MHHA-SRU network learns experience from the historical data set, and then utilizes the known experience to perform online learning on the newly arrived data set; the incremental learning is performed to perform line incremental MHHA-SRU network training by using a cross entropy loss function of an added regular term, wherein the cross entropy loss function of the added regular term comprises a cross entropy loss function of an EWC regular term and a cross entropy loss function of an added synaptic intelligent SI regular term;

the calculation formula of the cross entropy loss function of the elastic weight merging EWC regular term is as follows:

in the method, in the process of the invention,to add the cross entropy loss function of the EWC regularization term,for the loss function of the current task B, multi-classification tasks are performed, using cross entropy loss functions；For all of the parameters of the neural network,the network parameters of the current task B in the ith training are obtained;the method comprises the steps of training an old task A for the ith time, and obtaining optimal parameters through network training; Is a regularization coefficient;is Fisher information matrix for measuring network parameter importance,as a sign of the partial derivative,representing that the input x corresponds to the label y in the ith training;a parameter representing the network under the condition that the ith training input is x label y; for loss functionCalculating gradient value by deviator and parameterAccumulating all gradients and dividing by the number of samples N;

the calculation formula of the cross entropy loss function of the synapse intelligent SI regular term is as follows:

in the method, in the process of the invention,to add the cross entropy loss function of SI regularization term,a loss function for the current task B;to trade-off the coefficients of the current task loss and proxy loss,as for the current kth parameter,optimizing the parameter position after finishing for the last task,for the importance of this parameter to past tasks,for measuring the current parameter as the importance of the parameterAbsolute contribution of (2);is the inner product for measuring parametersThe update amplitude in the current task;is constant and is used for preventing unstable calculation caused by excessively small denominator.

In step S3, performing feature extraction on the flight information to be predicted through the online incremental MHHA-SRU network after setting the initial parameters includes: the simple circulation unit MHHA-SRU comprises a layered encoder module, a multi-head attention mechanism module and a Softmax full-connection layer classifier module, wherein the layered encoder module is constructed by taking an SRU as a unit, and comprises a primary encoder and a secondary encoder, and input data are data which are serialized and divided into a plurality of subsets; two sections of characteristic extraction are carried out through a layered encoder module built by Bi-SRU units, and a multi-head attention mechanism is added after the two encoders respectively; and finally, carrying out classification prediction of 5 levels of flight delay by a Softmax full connection layer, and selecting the level with the highest probability as the final output result.

In one embodiment, the primary and secondary encoders built from Bi-SRU units add a Bi-directional structure such that the connections in the hidden layer flow in reverse order; the formulas for the forward and backward states of the Bi-SRU cell are described as:

in the method, in the process of the invention,for the hidden state of the forward SRU unit,for the hidden state of the backward SRU unit,splice the two hidden states of the forward SRU unit and the backward SRU unit,as the final output of the Bi-SRU encoder;inputting a data set for a given set of flight weather, wherein the t-th element is。

In one embodiment, the multi-head attention mechanism module trains different flight weather samples and different weights of single flight weather samples in a sequence through a neural network, and finds the difference of influence degrees of various characteristic variables in the single flight weather data on the flight delay level; wherein each layer of attention mechanism adopts a multi-head self-attention mechanism and is derived from the self-attention mechanism; the self-attention mechanism description formula is:

in the method, in the process of the invention,representing the output of the attention mechanism, whereinAre matrices, Q represents a query matrix; k represents a keyword matrix; v represents a matrix of values;for dot product calculation, calculating dot products between the keyword matrix K and the query vector Q; As the dimension of the key vector,is the vector length;refer to normalization operations using SoftMax functions.

Under the condition that the input of the attention mechanism is not changed, the multi-head attention mechanism modifies the output into the result of splicing the outputs of different heads, and the description formula of the multi-head attention mechanism is as follows:

in the method, in the process of the invention,representing the output of a multi-headed attentiveness mechanism, whereinThe meaning of the expressed matrix is the same as the expression of the attention mechanism;for outputting matrix, usingConverting the connected multi-headed attention output into a desired dimension;describing a formula for the original attention mechanism, the i subscript indicates the i-th attention header currently being the multi-head attention mechanism,are all linear mapping parameters for calculating a matrix of weight coefficients representing the distribution of attention, wherein、Representing the unique properties or characteristics for learning on the ith head in relation to that head, respectively;representing a separate attention mechanism module.

In step S4, the prediction structure is a full-connection layer classifier that finally adopts a Softmax activation function, classifies the flight delay situation with weather into five classes, and assigns a probability value to each output classification result through the Softmax activation function, thereby indicating the possibility of belonging to each class; then taking the grade label with the highest probability as a final prediction result;

In step S5, adopting an incremental learning algorithm based on regularization, constructing a new training data set by using untrained flights and new weather data accumulated online or adding part of trained historical flights and new weather data on the basis of the new data, sending the new training data set into an online incremental MHHA-SRU network, enabling a loss function added with a regularization term to be minimum after back propagation, and storing a weight file with the minimum loss function to replace an existing weight file so as to update parameters of the online incremental MHHA-SRU network;

in step S6, inputting the flight information data stream to be predicted into an online increment MHHA-SRU network after updating parameters, obtaining the probability of the flights to be predicted belonging to each category through a prediction structure after feature extraction, and obtaining the delay category of the flights to be predicted based on the probability to realize the prediction of the flight delay level after updating the parameters.

Another object of the present invention is to provide an online delta MHHA-SRU based flight delay prediction system for implementing an online delta MHHA-SRU based flight delay prediction method, the system comprising:

the data receiving module acquires real-time flight and weather data from the office data system through the FTP service;

The data set processing module is used for carrying out data preprocessing, data set fusion and data encoding on real-time and historical flight meteorological data, and constructing training for an online initial model and subsequent online parameter learning;

the flight delay initial MHHA-SRU model training module is used for setting initial parameters of an online increment MHHA-SRU network model, inputting flight meteorological data into the model with the set parameters for training, and obtaining a flight delay prediction model based on the online increment MHHA-SRU network;

the online learning module of the online incremental MHHA-SRU network uses an online data training model for accumulating a period of time for online updating of an initial model;

the flight delay category obtaining module judges the category of the flight through the prediction structure, obtains the probability of each delay category of each flight, and takes the category corresponding to the maximum probability as a delay level prediction result;

and the delay information visual analysis module uses a browser and server architecture mode to build a front-end system and a back-end system, and the front end uses Echarts to realize visual analysis and display of big data.

By combining all the technical schemes, the invention has the advantages and positive effects that: firstly, setting initial parameters of an online increment MHHA-SRU network, carrying out data cleaning, data labeling, data fusion, data encoding and data serialization on historical flight data and meteorological data, dividing the historical flight data and the meteorological data into a plurality of subsets containing a certain sample number, and inputting the subsets into the set online increment MHHA-SRU network model for training to obtain a flight delay prediction model based on the online increment MHHA-SRU network; inputting the flight information data stream to be predicted into a flight delay prediction model based on an online incremental MHHA-SRU network, and extracting features of the flight information to be predicted through the online incremental MHHA-SRU network after initial parameters are set; the extracted characteristics obtain the probability of the flights to be predicted belonging to each category through a prediction structure, and delay categories of the flights to be predicted are obtained based on the probability; using an increment learning algorithm based on regularization to send the flight information data stream after online accumulation for a period of time into an online increment MHHA-SRU network for training, and updating parameters of the online increment MHHA-SRU network; inputting the flight information data stream to be predicted into an online updated MHHA-SRU network for prediction to obtain a delay class; and finally, carrying out visual analysis and display on the flight information data stream to be predicted and delay class statistics of the flights to be predicted on a front-end interface. The invention has higher prediction performance for the delay prediction of flights, and the prediction accuracy rates of 93.4%, 95.2%, 85.7% and 96.5% are respectively achieved at Tianjin certain international airport (ZBTJ), beijing certain international airport (ZBAD), beijing certain international airport (ZBAA) and Shijia certain international airport (ZBSJ); and the model can realize online parameter updating aiming at the distribution change of online real-time data, so that good model prediction performance can be maintained during online operation.

According to the flight delay prediction method, historical flight data is subjected to data cleaning, data marking, data fusion, data encoding and data serialization and is divided into data subsets with a certain sample number.

Aiming at the problems that the training speed of the existing flight delay prediction models is low and the prediction precision is low, the invention provides a flight delay prediction method of a simple circulation unit (MHHA-SRU) based on a multi-head hierarchical attention mechanism, which is used for mining time sequence information between front and rear flight data, adding a hierarchical attention mechanism structure to respectively extract two sections of features of single-picking data and serialized data, and paying more attention to variables with larger influence on prediction results.

Aiming at the problem that the model accuracy is reduced because the original offline model cannot update own parameters in time in the continuous generation of flights and meteorological data in actual application, the invention provides the online learning of the original offline model, and the online incremental learning capability is realized by adding an incremental learning algorithm based on regularization, so that the average prediction accuracy of the model on continuous tasks is improved, and the prediction accuracy of the model exceeds the offline learning accuracy after a plurality of batches of tasks. The flight delay real-time prediction system is simple and convenient to deploy.

The present invention provides a flight delay prediction service that can update models online, as well as providing web application usage. The method analyzes the problem that the prediction accuracy of the offline flight delay prediction model is reduced along with the time in the industry; mining time sequence information among flight data by using Bi-SRU network; the prediction accuracy of the deep learning model is improved by using a layered attention mechanism; the method solves the problems that the deep learning model is slow in training speed and low in prediction accuracy. The invention greatly improves the accuracy of the flight delay prediction model by using a simple circulation unit (MHHA-SRU) network added with a multi-head hierarchical attention mechanism. The invention designs an online data processing system which can store the flight weather information for a period of time, send the flight weather information into an online model for training after accumulating a certain amount of flight weather information, update network parameters after training is completed, and realize online updating of the model.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow chart of a method for predicting flight delay provided by an embodiment of the invention;

FIG. 2 is a schematic diagram of a flight delay prediction method according to an embodiment of the present invention;

FIG. 3 is a flow chart of data set construction provided by an embodiment of the present invention;

FIG. 4 is a schematic diagram of a serialization method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a serialized data set according to an embodiment of the present invention;

FIG. 6 is an overall block diagram of an online incremental MHHA-SRU provided by an embodiment of the present invention;

FIG. 7 is a block diagram of an MHHA-SRU network provided by an embodiment of the present invention;

FIG. 8 is a block diagram of a simple cycle unit (SRU) provided by an embodiment of the present invention;

FIG. 9 is a diagram illustrating the operation of a simple cyclic unit (SRU) provided by an embodiment of the present invention;

FIG. 10 is a basic block diagram of a layered encoder provided by an embodiment of the present invention;

FIG. 11 is a diagram illustrating the internal architecture of a multi-head self-attention mechanism provided by an embodiment of the present invention;

FIG. 12 is an average accuracy of an Elastic Weight Combining (EWC) algorithm provided by an embodiment of the present invention under different parameters;

FIG. 13 is a graph showing the average accuracy of a Synaptic Intelligent (SI) algorithm provided by an embodiment of the present invention under different parameters;

FIG. 14 is an overall block diagram of a Web-based flight delay real-time prediction system provided by an embodiment of the present invention;

FIG. 15 is a diagram of an overall technical architecture of a system provided by an embodiment of the present invention;

FIG. 16 is a functional design of the system according to an embodiment of the present invention;

FIG. 17 is a schematic diagram of an online MHHA-SRU flight delay prediction system according to an embodiment of the present invention;

in the figure: 1. a data receiving module; 2. a data set processing module; 3. a flight delay initial MHHA-SRU model training module; 4. online learning module of online increment MHHA-SRU network; 5. a flight delay category acquisition module; 6. and a delay information visual analysis module.

Detailed Description

In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit or scope of the invention, which is therefore not limited to the specific embodiments disclosed below.

The invention relates to the technical field of flight delay prediction in important places, in particular to an online flight delay prediction method, an online flight delay prediction system and application, and the online flight delay prediction method, the online flight delay prediction system and the application are used for predicting and analyzing the delay condition of flights. The method solves the defect that offline learning cannot integrate new information into the constructed model, and as more data arrives, the model continuously learns new knowledge to update parameters, thereby effectively improving the accuracy of delay prediction

In embodiment 1, as shown in fig. 1, the method for predicting flight delay provided by the embodiment of the invention includes the following steps:

Fig. 2 is a schematic diagram of a flight delay prediction method according to an embodiment of the present invention. The method mainly comprises two parts of processes, namely a prediction process; and secondly, an online training process. When the delay state of a certain flight is required to be predicted, the flight and meteorological data of the flight to be predicted are only required to be cleaned, fused, encoded and serialized and then directly input into a trained model for prediction and visualization, and the data are stored; because the flights are arranged according to the time sequence, the model does not need to be trained when new flights arrive each time, when a certain data amount is accumulated or the model needs to be trained online, the accumulated data is preprocessed and then input into an online incremental network model for training to obtain new model weights and replace the previous model weights, so that online updating of the model is realized.

In the embodiment of the present invention, in step S1, initial parameters of the online incremental MHHA-SRU network include: the number of network layers of the SRU primary encoder and the SRU secondary encoder, the number of hidden layer nodes of the SRU primary encoder and the SRU secondary encoder, the number of total connection layer layers of the Softmax, the number of nodes of each layer of total connection layer of the Softmax, the number of attention heads and the sequence length; the SRU primary and secondary encoder network layers are respectively 1 and 2, the SRU primary and secondary encoder hidden layer nodes are respectively 128 and 256, the Softmax full-connection layer number, the Softmax full-connection layer node number of each layer, the attention header number is 2, and the sequence length is 9; initial parameters of the training regimen include: the learning rate during training is 0.00001, and the optimizer during training uses Adam optimizer and EWC regular term penalty coefficientPenalty coefficient for 100, SI regular term5 and SI damping coefficient of 0.1.

In step S2, converting the historical flight data and meteorological data into an online mode for use, dividing the data into data subsets, and simulating a process of continuously inputting online data streams into a system; the data set construction flow is shown in fig. 3, and the specific method for constructing the data set by carrying out data preprocessing and data fusion, data encoding and data serialization on the flight and meteorological data is as follows: the data cleaning adopts a direct deleting and proper filling mode, and the data with missing values in flights adopts a proper filling mode of approximate items aiming at severely missing data and data containing abnormal items; the data marking is respectively processed for the take-off flight and the landing flight, and the delay level is divided according to the calculated take-off delay time T, wherein the dividing conditions are that 1, T is not delayed for 15 minutes, 2, 15 minutes are less than or equal to 60 minutes and are slightly delayed, 3, 60 minutes are less than or equal to 120 minutes and are moderate delay, 4, 120 minutes are less than or equal to 240 minutes and are highly delayed, and 5, T is more than 240 minutes and are severely delayed; the data fusion is matched and fused with the 'planned departure time' in the flight data used by each airport departure flight and the record time in the meteorological data of the airport, and the 'planned landing time' in the flight data used by each airport departure flight and the record time in the meteorological data of the airport; the data coding is respectively processed for continuous and discrete characteristics, and Min-Max coding is used for the continuous characteristics; the Catboost code is used for discrete features; the data serialization process is shown in FIG. 4, and the data sets are ordered according to the predicted take-off time and then used for length Is divided to convert the input data into a time series to generate a time series data set, the converted time series data set being shown in fig. 5.

As shown in fig. 6, training the constructed online incremental MHHA-SRU network model with the data set input with the set parameters to obtain a flight delay prediction model based on the online incremental MHHA-SRU network, which includes the following steps: inputting the constructed data set into the MHHA-SRU network shown in figure 7 for training, wherein the primary and secondary encoder structure relations are shown in figure 10; firstly, encoding input data by using a primary encoder, wherein the unit structure and the operation structure of an SRU are shown in fig. 8 and 9, the encoded data is processed by a multi-head attention mechanism as shown in fig. 11 to obtain the characteristic with larger influence on a prediction result, and the characteristic is sent to a secondary encoder for processing, and the multi-head attention mechanism is used for processing the output of the secondary encoder and then sent to a Softmax classifier for carrying out classification prediction of flight delay 5 grades; and calculating loss by using a cross entropy loss function added with a regular term, and performing back propagation calculation gradient to complete model training.

In the embodiment of the invention, in step S2, the built data subset is sequentially input into an online incremental MHHA-SRU network after initial parameters are set for training to obtain a flight delay prediction model based on the online incremental MHHA-SRU network, and the method comprises the following steps:

In an embodiment of the present invention, training of the model is achieved using a regularized incremental learning algorithm comprising: the line increment MHHA-SRU network learns experience from the historical data set, and then utilizes the known experience to perform online learning on the newly arrived data set; incremental learning performs line incremental MHHA-SRU network training by using cross entropy loss functions of added regular terms, wherein the cross entropy loss functions of the added regular terms comprise cross entropy loss functions of EWC regular terms and cross entropy loss functions of added synaptic intelligent SI regular terms.

And correcting the obtained category information through a cross entropy loss function added with a regular term, wherein the loss function is modified by adding the regular term into the cross entropy loss function. The cross entropy loss function added with the elastic weight merging (Elastic weight consolidation, EWC) regular term is shown as a formula (2) and a formula (3), and then the parameters are moved to a public area of the new task and the old task, so that a place where the loss function values of the models on the two tasks are very small is found, the important parameters of the old task are reserved after the models learn the new task, and further the prediction accuracy of the models under continuous tasks is improved.

（2）

（3）

In the method, in the process of the invention,to add the cross entropy loss function of the EWC regularization term,for the loss function of the current task B, multi-classification tasks are performed, using cross entropy loss functions；For all of the parameters of the neural network,the network parameters of the current task B in the ith training are obtained;the method comprises the steps of training an old task A for the ith time, and obtaining optimal parameters through network training;is a regularization coefficient;is Fisher information matrix, is used for measuring the importance degree of network parameters,as a sign of the partial derivative,representing that the input x corresponds to the label y in the ith training; A parameter representing the network under the condition that the ith training input is x label y; for loss functionCalculating gradient value by deviator and parameterAll gradients are accumulated and divided by the number of samples N.

The cross entropy loss function added with the regular term of the synaptic intelligence (Synaptic Intelligence, SI) is shown as a formula (4) and a formula (5), wherein the synapse refers to the strength of the interconnection between neurons, namely the parameters of the neural network, when knowing which parameters have the largest contribution to the reduction of the loss function L of the current task T, the update of the parameters can be limited when a new task arrives, and the parameters can be used for maintaining the performance of an old model on the old task; the formula shows that when the absolute contribution of a parameter is large and the self update amplitude is small, the parameter is very important, and needs to be reserved.

（4）

（5）

In the method, in the process of the invention,to add the cross entropy loss function of SI regularization term,a loss function for the current task B;to trade-off the coefficients of the current task loss and proxy loss,as for the current kth parameter,optimizing the parameter position after finishing for the last task,for the importance of this parameter to past tasks,for measuring the current parameter as the importance of the parameter Absolute contribution of (2);is the inner product for measuring parametersThe update amplitude in the current task;is constant and is used for preventing unstable calculation caused by excessively small denominator.

Exemplary, the initial learning rate of the flight delay prediction method is 0.001, adam optimizer is used, EWC regularization penalty term coefficients100 EWC Fisher information matrix sample size is 4096, SI damping coefficient is 0.1, SI regularization penalty term coefficient c is 5, and task number is 10. The accuracy of two different incremental algorithms (EWC and SI) under different parameters was compared and analyzed. A flight weather data set of a certain International airport (ZBTJ) of Tianjin divided into 10 tasks is adopted, each task comprises 20000 data samples, and the grid search method is utilized to find the optimal parameters of two incremental algorithms. Regularization punishment term coefficient of EWC algorithm at different valuesThe average accuracy over this is shown in FIG. 12, which can be found whenAt 100, on 10 tasks91.8% is reached, and the online learning is improved by 0.4% compared with online learning without adopting an incremental algorithm. The EWC algorithm is described to effectively alleviate the catastrophic forgetting of the model on the new task. The average accuracy of the SI algorithm on different regularization penalty coefficients c is shown in FIG. 13, and it can be found that when c takes a value of 5, the algorithm takes on 10 tasks 92.0% is reached, which is improved by 0.6% compared with online learning without adopting incremental algorithm, and compared with EWC algorithm on optimal parameter valueThe improvement is 0.2 percent.

In step S3, the structure of extracting the structure of the flight information to be predicted using the online incremental MHHA-SRU network includes: simple loop unit (Multi-Head Hierarchical Attention mechanism based on Simple Recurrent Units, MHHA-SRU) extraction features using the add Multi-head hierarchical attention mechanism shown in fig. 7 include: a layered encoder module (primary, secondary encoder, fig. 10), a multi-headed attention mechanism module (fig. 11), and a Softmax fully connected layer classifier module built by SRU units (fig. 8, fig. 9). The input data is the data which is serialized and divided into a plurality of subsets; then, two sections of feature extraction is carried out through a layered encoder module built by an SRU unit, a multi-head attention mechanism is added after the two encoders respectively, more computing resources are allocated for the data with great influence on the flight delay prediction result, and the overall classification capacity of the model is improved; and finally, carrying out classification prediction of 5 levels of flight delay by a Softmax full connection layer, and selecting the level with the highest probability as the final output result.

The Bi-SRU unit is used for constructing the primary encoder and the secondary encoder, and a bidirectional structure is added, so that the connection in the hidden layer can flow in the reverse order, and the information between the flight weather data contexts can be more effectively utilized. The formulas for the forward and backward states of the Bi-SRU are described as:

（6）

（7）

（8）

in the method, in the process of the invention,for the hidden state of the forward SRU unit,for the hidden state of the backward SRU unit,concealment for both forward and backward SRU unitsThe state of the state splicing is that,as the final output of the Bi-SRU encoder;inputting a data set for a given set of flight weather, wherein the t-th element is。

As shown in fig. 12, the multi-head attention mechanism module trains different flight weather samples and different weights of single flight weather samples in a sequence through a neural network, finds the difference of influence degrees of various feature variables in single flight weather data on the flight delay level, and allocates more computing resources to the part with large influence degrees on the flight delay prediction result in the data, thereby improving the overall classification performance. Wherein each layer of attention mechanism employs a multi-headed self-attention mechanism, which is derived from the self-attention mechanism. Equation (9) describes the equation for the self-attention mechanism:

（9）

In the method, in the process of the invention,representing the output of the attention mechanism, whereinAre matrices, Q represents a query matrix; k represents a keyword matrix; v represents a matrix of values;for dot product calculation, calculating dot products between the keyword matrix K and the query vector Q;as the dimension of the key vector,is the vector length;refer to normalization operations using SoftMax functions.

The multi-head attention mechanism modifies the output to be the result of different head output splicing without changing the input, and the multi-head attention mechanism describes the formulas as (10) and (11):

（10）

（11）

in the method, in the process of the invention,representing the output of a multi-headed attentiveness mechanism, whereinThe meaning of the expressed matrix is the same as the expression of the attention mechanism;is an output matrix for converting the connected multi-headed attention output into a desired dimension;describing a formula for the original attention mechanism, the i subscript indicates the i-th attention header currently being the multi-head attention mechanism,are all linear mapping parameters for calculating a matrix of weight coefficients representing the distribution of attention, wherein、Respectively represent forLearning a unique property or feature associated with the head on the ith head;representing a separate attention mechanism module.

The online updating model method is as shown in fig. 14, in the system, an incremental learning algorithm based on regularization is adopted, an online accumulated untrained flight and weather new data or a part of trained historical flights and weather data are added on the basis of the new data to construct a new training data set, the new training data set is sent into an online incremental MHHA-SRU network, after back propagation, the loss function added with a regularized item is enabled to be minimum, a weight file with the minimum loss function is saved to replace an existing weight file, and the updating of parameters of the online incremental MHHA-SRU network is realized;

in step S5, the system may perform an update of online network model parameters. The traditional offline batch learning generally needs to prepare all data before model training, adopts preset super parameters to perform model training, fixes the model after the training is finished, and deploys the model into an actual test environment, which puts high requirements on the integrity of data collection. Under a certain scenario, data is continuously generated along with time, and may be destroyed due to storage limitation or privacy protection, so that the model is required to have learning ability on a new sample, and thus parameter updating and expansion are performed on the model itself, otherwise, the network model cannot adapt to the new data and is eliminated. The incremental learning algorithm based on regularization can effectively avoid the additional expense caused by repeated training of the model on all data sets. The flight and weather data are continuously updated in practical application, and the parameters of the online increment MHHA-SRU network can be updated by sending the online accumulated flight information data stream for a period of time into the online increment MHHA-SRU network for training.

In step S6, the prediction structure refers to that a full-connection layer of a Softmax activation function is finally adopted as a classifier, the flight delay situation is classified into 5 classes, a probability value is given to each class through the Softmax function, the probability of judging the class is expressed, and a class of label with the highest probability is taken as a final output result. Specifically, an online increment MHHA-SRU network after the flight information data stream to be predicted is input with updated parameters, the probability that the flights to be predicted belong to each category is obtained through a prediction structure after feature extraction, and the delay category of the flights to be predicted is obtained based on the probability, so that the prediction of the flight delay level after the updated parameters is realized.

In step S7, the system technical architecture required for the visual analysis is shown in fig. 15, and the system running environment is Windows, and mainly includes an access layer, a presentation layer, an access layer, a service layer and a data layer. The main functions of the system are as shown in fig. 16, and mainly comprise a user registration and login function, an airport delay prediction warning function, a next-day flight delay prediction function and a next-day flight delay quantity counting function of each airport. In a visual interface of a system required by visual analysis, from a system login interface, the system firstly skips to a login page for account login, the page automatically hides the password input by the current user in consideration of the safety of the user, and simultaneously the password input by the user is written into a database in an MD5 encryption mode; from each airport delay prediction warning interface, a map-type overview of the number of delay predictions of an airport group can be displayed, ripple points of different airports are marked in the map, and different ripple points represent different delay degrees. The upper left corner of the map can be marked with delay degree quantization indexes of different ripple points, the delay degree quantization indexes represent flight delay times of all airports, the ripple points represent that the delay times of the airports are between 0 and 50 frames, the ripple points represent that the delay times of the airports are between 50 and 200 frames, the ripple points represent that the delay times of the airports are between 200 and 500 frames, and the ripple points represent that the delay times of the airports are more than 500 frames. Moving the mouse to the responsive airport location will reveal the particular next-day delayed flight prediction for that airport. The ripple points indicate that the delay of the next-day flight of an international airport is 474 frames; meanwhile, the next-day flight delay information predicted by the algorithm subsystem is provided in an airport group next-day flight delay module, the data can be refreshed in real time through the module, the data update date can be displayed, and information such as flight numbers, departure airports, departure time and predicted delay grades can be provided. And jumping to a complete flight delay prediction information base by clicking a complete information button displayed by the module; the information such as the flight number, the model, the data updating date, the departure airport name, the landing airport name, the planned departure time, the planned arrival time, the delay prediction grade and the like of the delayed flights is provided through the system shell replacement needed by visual analysis, and the information of the flight prediction information of the last month is recorded in the database, so that the check and the verification of the historical flight data by the control personnel are facilitated. Meanwhile, fuzzy screening conditions are added in the navigation bar, and screening and filtering of display information can be performed rapidly according to flight dates, flight numbers and flight numbers; the number distribution histogram stack of 5 different flight delay levels of each airport of an airport group can be displayed in the next day flight delay condition module of the airport group, and the flight delay condition information of the airport is displayed.

Embodiment 2 as shown in fig. 17, the flight delay prediction system based on the online incremental MHHA-SRU network according to the embodiment of the present invention includes:

the data receiving module 1 acquires real-time flight and weather data from the office data system through an FTP service;

the data set processing module 2 is used for carrying out data preprocessing, data set fusion and data encoding on the real-time and historical flight meteorological data, and constructing training for an online initial model and subsequent online parameter learning;

the flight delay initial MHHA-SRU model training module 3 is used for setting initial parameters of an online increment MHHA-SRU network model, inputting flight meteorological data into the model with the set parameters for training, and obtaining a flight delay prediction model based on the online increment MHHA-SRU network;

the online learning module 4 of the online incremental MHHA-SRU network uses online data training models accumulated for a period of time for online updating of the initial models;

the flight delay category obtaining module 5 judges the category of the flight through the prediction structure, obtains the probability of each delay category of each flight, and takes the category corresponding to the maximum probability as a delay level prediction result;

and the delay information visual analysis module 6 uses Browser/Server (B/S) mode to build a front-end system and a back-end system, and the front end uses Echarts to realize visual analysis and display of big data.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

The content of the information interaction and the execution process between the devices/units and the like is based on the same conception as the method embodiment of the present invention, and specific functions and technical effects brought by the content can be referred to in the method embodiment section, and will not be described herein.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

The flight delay prediction algorithm provided by the embodiment of the invention is deployed on a server or is applied through a web terminal.

The embodiment of the invention provides computer equipment, which comprises: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, which when executed by the processor performs the steps of any of the various method embodiments described above.

Embodiments of the present invention also provide a computer readable storage medium storing a computer program which, when executed by a processor, performs the steps of the respective method embodiments described above.

The embodiment of the invention also provides a server, which is used for realizing the steps in the method embodiments when being executed on the electronic device and providing a user input interface.

Embodiments of the present invention provide a computer program product which, when run on an electronic device, causes the electronic device to perform the steps of the method embodiments described above.

The embodiment of the invention also provides an information visual terminal, wherein the information data processing terminal is used for providing a user interface to implement the steps in the method embodiments when the information data processing terminal is implemented on an electronic device and displaying delay information for analyzing flight delay through the visual terminal.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above-described embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and the computer program may implement the steps of the method embodiments described above when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc.

Based on the above examples, the invention further relates to experiments, which show that: the method for predicting the flight delay provided by the embodiment of the invention comprises the steps of firstly setting initial parameters of an online incremental MHHA-SRU network model of the method for predicting the flight delay, obtaining flight data and weather data, and constructing a data set by preprocessing historical flight and weather data, fusing data, encoding data and serializing data. Inputting the constructed data set into an online incremental MHHA-SRU network model with set parameters for training to obtain a flight delay prediction model based on deep learning; inputting the flight information to be predicted into a flight delay prediction model, and extracting features of the flight information through an MHHA-SRU network; judging delay categories of flights through a prediction structure, obtaining probability of the flights corresponding to each category, and predicting and obtaining flight delay category information; inputting online incremental MHHA-SRU network model through online accumulated data to train and update model parameters; and carrying out visual analysis on the predicted flight delay information through the front-end framework.

The invention has good prediction performance for the flight delay prediction, can realize the online update of model parameters, and has the online learning capability; the model is simple to operate and easy to train; easy to deploy. The results of the comparison of the prediction accuracy at the four airports with other existing methods are shown in table 1, and the prediction accuracy shows that the effect obtained by the method is optimal.

Table 1 comparison of prediction accuracy of different models at four airports

The incremental algorithm (EWC:、SI:) And the accuracy of online learning under ten continuous tasks without incremental online learning (None), the data set adopts a certain international airport of Tianjin (four-word: ZBTJ) flight weather dataset. Experimental results show that with the increase of task updating times, the model can be quickly adapted to the change of data sample distribution through the training of an online increment learning algorithm based on regularization, so that the prediction accuracy of the model on continuous tasks is improved. When the data sample changes, the offline batch learning is too fit with the data distribution in the old task and cannot capture the data rule on the new task well, so that performance degradation phenomenon occurs on some new tasks. After the 5 th task update, the model prediction accuracy by adopting the online learning algorithm is better than that of the offline model.

While the invention has been described with respect to what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Claims

1. An online increment based MHHA-SRU flight delay prediction method is characterized by comprising the following steps:

2. The online delta MHHA-SRU based flight delay prediction method of claim 1, wherein in step S1, the initial parameters of the online delta MHHA-SRU network include: the number of network layers of the SRU primary encoder and the SRU secondary encoder, the number of hidden layer nodes of the SRU primary encoder and the SRU secondary encoder, the number of total connection layer layers of the Softmax, the number of nodes of each layer of total connection layer of the Softmax, the number of attention heads and the sequence length;

initial parameters of the training regimen include: learning rate during training, optimizer during training, EWC regular term penalty coefficientSI regularization term penalty coefficient>SI damping coefficient.

3. The online delta MHHA-SRU based flight delay prediction method according to claim 1, wherein in step S2, the historical flight data and weather data are converted into online mode for use, and the data are divided into data subsets to simulate the process of continuously inputting online data streams into the system;

ordering data sets according to predicted take-off times in data serialization using lengthDividing the window of the input data, converting the input data into a time sequence to generate a time sequence data set;

4. The online delta MHHA-SRU based flight delay prediction method according to claim 1, wherein in step S2, the online delta MHHA-SRU network after the built data subset is sequentially input with the set initial parameters is trained to obtain the online delta MHHA-SRU based flight delay prediction model, which comprises the following steps:

5. The online delta MHHA-SRU based flight delay prediction method of claim 4, wherein the model training is performed using a regularized delta learning algorithm in performing the loss calculation using the cross entropy loss function of the online delta learning improved join regularization term and iterating the input data subset to complete the training, the delta learning algorithm comprising: the line increment MHHA-SRU network learns experience from the historical data set, and then utilizes the known experience to perform online learning on the newly arrived data set; the incremental learning is performed to perform line incremental MHHA-SRU network training by using a cross entropy loss function of an added regular term, wherein the cross entropy loss function of the added regular term comprises a cross entropy loss function of an EWC regular term and a cross entropy loss function of an added synaptic intelligent SI regular term;

；

in the method, in the process of the invention,cross entropy loss function for adding EWC regular term, < ->For the loss function of the current task B, +.>For all parameters of the neural network, +.>The parameters of the network of the current task B in the ith training are obtained; />For the ith training for old task A, < +. >Is a regularization coefficient; />Is Fisher information matrix,>is the sign of the partial derivative>Representing that the input x corresponds to the label y in the ith training; />A parameter representing the network under the condition that the ith training input is x label y; for loss function->Calculating gradient value by solving partial derivative, and calculating parameter +.>Accumulating all gradients and dividing by the number of samples N;

；

in the method, in the process of the invention,cross entropy loss function for adding SI regular term, < +.>A loss function for the current task B; />To trade-off the coefficients of the current task loss and the proxy loss, < >>For the current kth parameter,/->Optimizing the parameter position after the end for the last task,/->For the importance of this parameter to past tasks, +.>For the importance of the parameters->For internal accumulation, add>Is constant.

6. The online delta MHHA-SRU based flight delay prediction method according to claim 1, wherein in step S3, performing feature extraction on the flight information to be predicted through the online delta MHHA-SRU network after setting the initial parameters comprises: the simple circulation unit MHHA-SRU comprises a layered encoder module, a multi-head attention mechanism module and a Softmax full-connection layer classifier module, wherein the layered encoder module is constructed by taking an SRU as a unit, and comprises a primary encoder and a secondary encoder, and input data are data which are serialized and divided into a plurality of subsets; two sections of characteristic extraction are carried out through a layered encoder module built by Bi-SRU units, and a multi-head attention mechanism is added after the two encoders respectively; and finally, carrying out classification prediction of 5 levels of flight delay by a Softmax full connection layer, and selecting the level with the highest probability as the final output result.

7. The online delta MHHA-SRU based flight delay prediction method of claim 6, wherein the primary and secondary encoders built by Bi-SRU units add a Bi-directional structure such that connections in the hidden layer flow in reverse order; the formulas for the forward and backward states of the Bi-SRU cell are described as:

；

in the method, in the process of the invention,is the hidden state of the forward SRU unit, < >>For the hidden state of the backward SRU unit, < >>Splicing the two hidden states of the forward SRU unit and the backward SRU unit>As the final output of the Bi-SRU encoder; />Inputting a data set for a given set of flight weather, wherein the t-th element is +.>。

8. The online increment MHHA-SRU based flight delay prediction method of claim 6, wherein the multi-headed attention mechanism module gives different flight weather samples and different weights of single flight weather samples in a sequence through neural network training, and finds the difference between various feature variables in single flight weather data on the degree of influence on the flight delay level; wherein each layer of attention mechanism adopts a multi-head self-attention mechanism and is derived from the self-attention mechanism; the self-attention mechanism description formula is:

；

In the method, in the process of the invention,an output representing the mechanism of attention, wherein +.>Are matrices, Q represents a query matrix; k represents a keyword matrix; v represents a matrix of values; />For dot product calculation, calculating dot products between the keyword matrix K and the query vector Q; />For the dimension of the key vector, +.>Is the vector length; />Means performing normalization operation by using a SoftMax function;

；

in the method, in the process of the invention,an output representing a multi-headed attentiveness mechanism, wherein +.>The meaning of the expressed matrix is the same as the expression of the attention mechanism; />Is an output matrix for converting the connected multi-headed attention output into a desired dimension; />Describing a formula for the original attention mechanism, i subscript indicates the i-th attention head, which is currently the multi-head attention mechanism,>are all linear mapping parameters for calculating a matrix of weight coefficients representing the attention distribution, wherein +.>、/>Representing the unique properties or characteristics for learning on the ith attention head to be related to the head, respectively;representing a separate attention mechanism module.

9. The online incremental MHHA-SRU based flight delay prediction method according to claim 1, wherein in step S4, the prediction structure classifies the weather-bearing flight delay into five classes for the full-connection layer classifier that finally uses a Softmax activation function, and assigns a probability value to each output classification result by the Softmax activation function, indicating the likelihood of belonging to each class; then taking the grade label with the highest probability as a final prediction result;

10. An online delta MHHA-SRU based flight delay prediction system, wherein the online delta MHHA-SRU based flight delay prediction method of any one of claims 1 to 9 is implemented, the system comprising:

the data receiving module (1) acquires real-time flight and weather data from the local data system through the FTP service;

The data set processing module (2) is used for carrying out data preprocessing, data set fusion and data encoding on real-time and historical flight meteorological data, and constructing training for an online initial model and subsequent online parameter learning;

the flight delay initial MHHA-SRU model training module (3) is used for setting initial parameters of an online increment MHHA-SRU network model, inputting flight meteorological data into the model with the set parameters for training, and obtaining a flight delay prediction model based on the online increment MHHA-SRU network;

the online learning module (4) of the online incremental MHHA-SRU network uses online data training models accumulated for a period of time for online updating of the initial models;

the flight delay category obtaining module (5) judges the category of the flight through the prediction structure, obtains the probability of each delay category of each flight, and takes the category corresponding to the maximum probability as a delay level prediction result;

and the delay information visual analysis module (6) uses a browser and server architecture mode to build a front-end system and a back-end system, and the front end uses Echarts to realize visual analysis and display of big data.