CN116743646B - An anomaly detection method based on domain-adaptive deep autoencoder tunnel network - Google Patents

An anomaly detection method based on domain-adaptive deep autoencoder tunnel network Download PDF

Info

Publication number
CN116743646B
CN116743646B CN202311023612.8A CN202311023612A CN116743646B CN 116743646 B CN116743646 B CN 116743646B CN 202311023612 A CN202311023612 A CN 202311023612A CN 116743646 B CN116743646 B CN 116743646B
Authority
CN
China
Prior art keywords
network
network traffic
tunnel
anomaly detection
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311023612.8A
Other languages
Chinese (zh)
Other versions
CN116743646A (en
Inventor
李�浩
李朋
杨路
陆艳铭
陈志涛
李孜
胡皓
马伟任
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yunnan Communications Investment & Construction Group Co ltd
Yunnan Provincial Transportation Planning And Design Research Institute Co ltd
Original Assignee
Yunnan Provincial Transportation Planning And Design Research Institute Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yunnan Provincial Transportation Planning And Design Research Institute Co ltd filed Critical Yunnan Provincial Transportation Planning And Design Research Institute Co ltd
Priority to CN202311023612.8A priority Critical patent/CN116743646B/en
Publication of CN116743646A publication Critical patent/CN116743646A/en
Application granted granted Critical
Publication of CN116743646B publication Critical patent/CN116743646B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/50Testing arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/243Classification techniques relating to the number of classes
    • G06F18/2433Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • G06N3/0455Auto-encoder networks; Encoder-decoder networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/048Activation functions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0631Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/12Network monitoring probes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/16Threshold monitoring
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to a tunnel network anomaly detection method based on a domain self-adaptive depth self-encoder, and belongs to the technical field of tunnel network anomaly detection. The method comprises the steps of data acquisition and preprocessing, training and updating of an abnormal detection source domain model, dynamic threshold calculation of abnormal detection, abnormal data detection and the like. The invention can directly perform operations such as preprocessing, abnormality detection and the like on the tunnel network at the edge side, improves the processing speed of the monitoring system and effectively reduces the processing time delay. Meanwhile, based on the normal reference value of the current network state, the abnormal threshold range is reasonably set, and abnormal information missing report, false report and other conditions caused by fixed setting of the threshold are avoided.

Description

一种基于域自适应深度自编码器隧道网络异常检测方法A tunnel network anomaly detection method based on domain adaptive deep autoencoder

技术领域Technical Field

本发明属于隧道网络异常检测技术领域,具体涉及一种基于域自适应深度自编码器的隧道网络异常检测方法,具体涉及一种基于域自适应深度自编码器完成隧道网络异常检测的边缘计算方法。The present invention belongs to the technical field of tunnel network anomaly detection, and specifically relates to a tunnel network anomaly detection method based on a domain adaptive deep autoencoder, and specifically relates to an edge computing method for completing tunnel network anomaly detection based on a domain adaptive deep autoencoder.

背景技术Background Art

隧道内机电系统庞大,设备分布较为复杂,对高速公路隧道内机电设备网络状态的监控与管理尤为重要。目前在单个高速公路隧道内,每隔500米设置一个区域控制器,控制周边的现场设备,这些控制柜通过交换机互联,以洞口两个方向组成光纤环网。以太网交换机不仅需要处理视频监控系统的高带宽数据,同样也需要配置成冗余光纤环网连接区域控制器,以控制隧道内的通风、照明、交通信号灯等设备。随着工业以太网设备数量的增加,工业以太网网络的结构日益复杂化。在实际应用过程中,网络拓扑感知能力不足、误操作引发的网络风暴、以及病毒感染等问题已经成为影响网络稳定性和可靠性的重要因素。工业以太网发生问题时,往往瞬间就会蔓延至整个网络,影响范围较大。此外,隧道内的区域控制器通常负责执行数字量、模拟量输入输出以及串口通信等功能,但无法有效采集交换机等网络设备的运行状态信息。也就是说,尽管隧道网络实际上已经建立,但当网络流量发生异常时,系统无法准确定位故障位置,同时也无法生成相应的记录。The electromechanical system in the tunnel is huge, and the equipment distribution is relatively complex. It is particularly important to monitor and manage the network status of electromechanical equipment in highway tunnels. At present, in a single highway tunnel, a regional controller is set up every 500 meters to control the surrounding field equipment. These control cabinets are interconnected through switches to form a fiber optic ring network in two directions of the tunnel entrance. Ethernet switches not only need to process high-bandwidth data from the video surveillance system, but also need to be configured as redundant fiber optic ring networks to connect regional controllers to control ventilation, lighting, traffic lights and other equipment in the tunnel. With the increase in the number of industrial Ethernet devices, the structure of industrial Ethernet networks has become increasingly complex. In actual applications, problems such as insufficient network topology perception, network storms caused by misoperation, and virus infection have become important factors affecting network stability and reliability. When problems occur in industrial Ethernet, they often spread to the entire network in an instant, affecting a large range. In addition, the regional controllers in the tunnel are usually responsible for executing functions such as digital quantity, analog quantity input and output, and serial port communication, but cannot effectively collect the operating status information of network devices such as switches. In other words, although the tunnel network has actually been established, when network traffic is abnormal, the system cannot accurately locate the fault location, and cannot generate corresponding records.

因此,将边缘计算架构引入隧道内各机电系统中具有重大研究意义。边缘计算技术可直接在隧道机电设备端进行数据处理,避免云端或其他数据中心的中转,提高了响应速度,减少了对隧道网络带宽的需求。在隧道环境中部署边缘计算节点,管理环境中的大量前端设备,基于边缘计算检测隧道网络状态可避免机电设备在运行过程中出现故障,有助于提高隧道机电系统的可靠性、提升智慧化水平。Therefore, it is of great research significance to introduce edge computing architecture into various electromechanical systems in tunnels. Edge computing technology can process data directly on the tunnel electromechanical equipment side, avoiding the transfer of data to the cloud or other data centers, improving the response speed and reducing the demand for tunnel network bandwidth. Deploying edge computing nodes in the tunnel environment, managing a large number of front-end devices in the environment, and detecting the tunnel network status based on edge computing can avoid failures of electromechanical equipment during operation, which helps to improve the reliability of tunnel electromechanical systems and enhance the level of intelligence.

然而,为了使得边缘计算可以在隧道网络监测中发挥更好的作用,很有必要设计一种合理的隧道网络异常检测边缘计算方法。目前已经应用于实践中的异常检测方法大多仍然需要依赖人工的检测和分析,运用最多的理论方法为数理统计方法,通常以统计学中的统计分布作为异常判断的标准,计算样本间的统计特性,采用设定的阈值实现异常检测。第二类方法是基于分类模型的方法,但模型训练需要优质的训练数据,并有大量带标签的数据集进行模型训练。第三类是基于距离的方法,离群的样本视为异常,这类算法不太适用于数据量大、维度高的数据。然而,由于隧道内机电设备复杂多样导致采集到的网络数据网络流量特征维度较高及数据间呈现高度非线性等问题,很难建立有效的异常检测模型。此外,由于隧道网络运行环境随时间具有动态变化的特点,只依据固定的监测指标,并对所有运行样本用固定异常检测模型的方式,容易产生误检的问题。因此如何克服现有技术的不足是目前隧道网络异常检测技术领域亟需解决的问题。However, in order to make edge computing play a better role in tunnel network monitoring, it is necessary to design a reasonable edge computing method for tunnel network anomaly detection. Most of the anomaly detection methods currently used in practice still need to rely on manual detection and analysis. The most commonly used theoretical method is the mathematical statistics method. The statistical distribution in statistics is usually used as the standard for anomaly judgment, the statistical characteristics between samples are calculated, and the set threshold is used to achieve anomaly detection. The second type of method is based on the classification model, but model training requires high-quality training data and a large number of labeled data sets for model training. The third type is based on the distance method. Outliers are considered anomalies. This type of algorithm is not suitable for data with large data volume and high dimension. However, due to the complexity and diversity of electromechanical equipment in the tunnel, the network traffic characteristics of the collected network data have high dimensions and highly nonlinear data, making it difficult to establish an effective anomaly detection model. In addition, since the tunnel network operating environment has the characteristics of dynamic changes over time, it is easy to cause false detection problems based on fixed monitoring indicators and fixed anomaly detection models for all running samples. Therefore, how to overcome the shortcomings of the existing technology is an urgent problem to be solved in the field of tunnel network anomaly detection technology.

发明内容Summary of the invention

本发明的目的是为了解决现有技术的不足,提供一种基于域自适应深度自编码器隧道网络异常检测方法。The purpose of the present invention is to solve the deficiencies of the prior art and to provide a tunnel network anomaly detection method based on domain adaptive deep autoencoder.

为实现上述目的,本发明采用的技术方案如下:To achieve the above purpose, the technical solution adopted by the present invention is as follows:

一种基于域自适应深度自编码器的隧道网络异常检测方法,包括如下步骤:A tunnel network anomaly detection method based on domain adaptive deep autoencoder comprises the following steps:

步骤1:通过部署在隧道内的边缘计算节点收集隧道机电系统设备层中设备的历史网络流量数据,解析获取对应的网络原始数据流,并进行数据预处理,得到对应的网络流量特征,即预处理后的网络流量样本;Step 1: The edge computing nodes deployed in the tunnel collect the historical network traffic data of the equipment in the tunnel electromechanical system equipment layer, parse and obtain the corresponding network raw data stream, and perform data preprocessing to obtain the corresponding network traffic characteristics, that is, the preprocessed network traffic samples;

步骤2:采集到的历史网络流量数据经步骤1处理获得每个网络流量数据对应的网络流量特征后,将其作为源域数据集;以源域数据集作为训练集,基于深度自编码器算法训练异常检测源域模型,训练完成后将异常检测源域模型部署在隧道边缘计算节点中;Step 2: After the collected historical network traffic data is processed in step 1 to obtain the network traffic features corresponding to each network traffic data, it is used as the source domain data set; the source domain data set is used as the training set, and the anomaly detection source domain model is trained based on the deep autoencoder algorithm. After the training is completed, the anomaly detection source domain model is deployed in the tunnel edge computing node;

步骤3:实时采集的网络流量数据经过步骤1所述的预处理方式得到对应的网络流量特征后,构建自适应滑动窗口算法获取该网络流量对应的目标域数据集;依据所对应的目标域数据集对步骤2获得的异常检测源域模型进行更新;Step 3: After the real-time collected network traffic data is preprocessed in the manner described in step 1 to obtain the corresponding network traffic features, an adaptive sliding window algorithm is constructed to obtain the target domain data set corresponding to the network traffic; the anomaly detection source domain model obtained in step 2 is updated according to the corresponding target domain data set;

步骤4:计算用于异常检测的动态阈值;Step 4: Calculate the dynamic threshold for anomaly detection;

步骤5:将实时采集的待测网络流量数据预处理后的网络流量特征输入到更新后的异常检测源域模型,计算其重构误差;Step 5: Input the network traffic features after preprocessing of the real-time collected network traffic data to be tested into the updated anomaly detection source domain model, and calculate its reconstruction error;

步骤6:根据步骤4获得的动态阈值及步骤5获得的重构误差,以检测实时采集的待测网络流量数据是否为异常数据。Step 6: Based on the dynamic threshold obtained in step 4 and the reconstruction error obtained in step 5, detect whether the network traffic data to be tested collected in real time is abnormal data.

进一步,优选的是,步骤1中,利用边缘计算节点进行隧道网络异常检测的系统架构包括设备层、边缘计算层、网络层和云平台层;设备层、边缘计算层、网络层和云平台层顺序连接;设备层中的各个设备系统包括广播电话系统、隧道监控系统、隧道通风照明系统、隧道区域控制器、隧道消防系统、信息发布系统和隧道交通信号系统;所述的边缘计算层为隧道内部署的边缘计算节点。Furthermore, preferably, in step 1, the system architecture for tunnel network anomaly detection using edge computing nodes includes a device layer, an edge computing layer, a network layer, and a cloud platform layer; the device layer, the edge computing layer, the network layer, and the cloud platform layer are connected sequentially; each device system in the device layer includes a broadcast telephone system, a tunnel monitoring system, a tunnel ventilation and lighting system, a tunnel area controller, a tunnel fire protection system, an information release system, and a tunnel traffic signal system; and the edge computing layer is an edge computing node deployed in the tunnel.

进一步,优选的是,步骤1中,所述的数据预处理方式包括去除异常数据、去除无意义特征和数据归一化;Further, preferably, in step 1, the data preprocessing method includes removing abnormal data, removing meaningless features and normalizing data;

网络流量特征包括数据流持续时间、正向包的数量、反向包的数量、正向包的总字节数、反向包的总字节数、正向包头总字节数、反向包头总字节数、正向子流总字节数和反向子流总字节数。The network traffic characteristics include data flow duration, the number of forward packets, the number of reverse packets, the total bytes of forward packets, the total bytes of reverse packets, the total bytes of forward packet headers, the total bytes of reverse packet headers, the total bytes of forward subflows, and the total bytes of reverse subflows.

进一步,优选的是,步骤2中,所述的异常检测源域模型采用深度自动编码器,包括编码器和解码器;Further, preferably, in step 2, the anomaly detection source domain model adopts a deep autoencoder, including an encoder and a decoder;

所述的异常检测源域模型具有三层神经网络,分别为输入层、隐含层和输出层,输入为;其中,表示源域数据集,表示第个预处理后的网络流量样本;The anomaly detection source domain model has a three-layer neural network, namely, input layer, hidden layer and output layer. , ;in, represents the source domain dataset, Indicates Preprocessed network traffic samples;

异常检测源域模型的具体训练方法如下:The specific training method of the anomaly detection source domain model is as follows:

步骤2.1:编码器将源域数据经过激活函数映射得到隐含层数据:Step 2.1: Encoder converts source domain data After activation function Mapping gets the hidden layer data:

; ;

式中,表示隐含层向量,表示第个预处理后的网络流量样本的隐含层向量;In the formula, represents the hidden layer vector, Indicates The hidden layer vector of the preprocessed network traffic samples;

编码过程如式(1)所示:The encoding process is shown in formula (1):

(1) (1)

式中,分别表示编码器的网络权重和偏置向量,为激活函数,在本发明中为Sigmoid函数;In the formula, and Represent the network weights and bias vectors of the encoder respectively, Is the activation function, which is the Sigmoid function in the present invention;

步骤2.2:解码器通过激活函数将隐含层数据转化到输出层获得输出变量:Step 2.2: The decoder passes the activation function The hidden layer data Transform to the output layer to obtain the output variable:

; ;

式中,表示重构的输出变量,表示第个重构的网络流量样本;In the formula, represents the reconstructed output variable, Indicates reconstructed network traffic samples;

经隐含层向量重构了输入变量,解码过程如式(2)所示; The hidden layer vector The input variables are reconstructed and the decoding process is shown in formula (2);

(2) (2)

式中,分别表示解码器的网络权重和偏置向量,为激活函数;In the formula, and Represent the network weights and bias vectors of the decoder respectively, is the activation function;

步骤2.3:利用梯度下降算法对异常检测源域模型进行训练,以最小化重构的误差为目标,得到最佳网络参数;目标函数如式(3)所示:Step 2.3: Use the gradient descent algorithm to train the anomaly detection source domain model, with the goal of minimizing the reconstruction error and obtaining the optimal network parameters; the objective function is shown in formula (3):

(3) (3)

式中,网络参数集合分别表示编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,分别表示第个DAE网络的输入和重构输出变量;M表示预处理后的网络流量样本的总数;In the formula, the network parameter set Respectively represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder, and Respectively represent The input and reconstructed output variables of a DAE network; M represents the total number of preprocessed network traffic samples;

步骤2.4:保存训练好的异常检测源域模型的网络参数,将该模型部署在隧道边缘计算节点。Step 2.4: Save the network parameters of the trained anomaly detection source domain model and deploy the model on the tunnel edge computing node.

进一步,优选的是,为Sigmoid函数。Furthermore, it is preferred that is the Sigmoid function.

进一步,优选的是,步骤3的具体方法为:Further, preferably, the specific method of step 3 is:

步骤3.1:假设在时刻,边缘计算节点实时采集的网络流量数据经过步骤1所述的预处理得到对应的网络流量特征,即预处理后的网络流量样本,定义为待测样本;利用自适应滑动窗口算法构建目标域数据集;具体方法为:Step 3.1: Assume that At this moment, the network traffic data collected in real time by the edge computing node is preprocessed as described in step 1 to obtain the corresponding network traffic features, that is, the preprocessed network traffic sample , defined as the sample to be tested; the target domain dataset is constructed using the adaptive sliding window algorithm ; The specific method is:

步骤3.1.1:以时刻的预处理后的网络流量样本为滑动窗口的右边界,向前序的预处理后的网络流量样本进行扩张,将时序临近预处理后的网络流量样本归属到滑动窗口内,则自适应滑动窗口数据集表示为:Step 3.1.1: Preprocessed network traffic sample at time As the right boundary of the sliding window, the network traffic samples pre-processed in the previous order are expanded, and the network traffic samples pre-processed in the adjacent time sequence are attributed to the sliding window. Then the adaptive sliding window data set It is expressed as:

; ;

式中,表示长度为的滑动窗口,包含时刻到时刻内的预处理后的网络流量样本;为窗口的左边界预处理后的网络流量样本,即自适应滑动窗口以时刻向前序扩张个预处理后的网络流量样本;In the formula, Indicates the length is The sliding window contains Time has come Preprocessed network traffic samples within time; is the network traffic sample preprocessed at the left edge of the window, that is, the adaptive sliding window is Time forward expansion Preprocessed network traffic samples;

步骤3.1.2:自适应滑动窗口在确定是否扩张前序样本时,假设滑动窗口已扩张到时刻,待判断是否纳入窗口的样本为时刻;首先,依据如下所示的相似性函数计算该时刻样本与当前窗口内部所有样本的欧式距离平均值:Step 3.1.2: When the adaptive sliding window determines whether to expand the previous sample, it is assumed that the sliding window has been expanded to At time , the sample to be judged whether to be included in the window is Moment; first, the average Euclidean distance between the sample at that moment and all samples in the current window is calculated based on the similarity function shown below:

(4) (4)

其中,ED的计算方式为:The calculation method of ED is:

; ;

式中,表示当前滑动窗口内从时刻到时刻内的任意预处理后的网络流量样本,为当前窗口内的预处理后的网络流量样本个数,则为待判断是否纳入窗口的前序预处理后的网络流量样本;In the formula, Indicates that the current sliding window is from Time has come Any preprocessed network traffic sample within time, is the number of preprocessed network traffic samples in the current window, It is the network traffic sample after pre-processing to be judged whether to be included in the window;

步骤3.1.3:根据所述步骤3.1.2的相似性函数设定自适应滑动窗口的边界阈值,若,滑动窗口将纳入该预处理后的网络流量样本,即窗口的左边界预处理后的网络流量样本为,反之则停止扩张,此时的左边界样本为Step 3.1.3: Set the boundary threshold of the adaptive sliding window according to the similarity function of step 3.1.2 ,like , the sliding window will include the preprocessed network traffic sample, that is, the preprocessed network traffic sample at the left boundary of the window is , otherwise the expansion stops, and the left boundary sample is ;

令滑动窗口数据集作为的目标域数据,则目标域数据集表示为:Let the sliding window dataset As The target domain data is represented as:

; ;

由于中共有条预处理后的网络流量样本,因此目标域数据集表示为:because The CCP has preprocessed network traffic samples, so the target domain dataset It is expressed as:

; ;

式中,表示在包含条预处理后的网络流量样本的数据集内第个预处理后的网络流量样本;In the formula, Including A dataset of preprocessed network traffic samples Neidi Preprocessed network traffic samples;

步骤3.2:利用目标域数据集对步骤2训练的异常检测源域模型进行域自适应更新;具体步骤如下:Step 3.2: Leverage the target domain dataset Perform domain adaptive update on the anomaly detection source domain model trained in step 2; the specific steps are as follows:

步骤3.2.1:首先,将源域数据集输入到训练好的异常检测源域模型中,通过公式(1)前向传播获取源域数据的隐含层向量Step 3.2.1: First, the source domain dataset Input it into the trained anomaly detection source domain model, and obtain the hidden layer vector of the source domain data through forward propagation of formula (1) ;

步骤3.2.2:将目标域数据同样输入到异常检测源域模型中通过如下公式前向传播获取目标域数据的隐含层向量Step 3.2.2: Target domain data The same input is used to the anomaly detection source domain model and the hidden layer vector of the target domain data is obtained by forward propagation through the following formula: :

(5) (5)

步骤3.2.3:以最大平均差距离作为目标函数,计算公式如式(6)所示:Step 3.2.3: Take the maximum average difference distance as the objective function, and the calculation formula is shown in formula (6):

; ;

(6) (6)

式中,分别为内的样本个数,为求取最小上界函数,在式中指代数据集中的任意索引,即分别表示中第个样本,分别表示中第个样本;为高斯核函数,计算方式如下:In the formula, They are and The number of samples in To find the minimum upper bound function, In the formula, refers to any index in the data set, that is, and Respectively Middle and samples, and Respectively Middle and samples; is the Gaussian kernel function, The calculation is as follows:

(7) (7)

式中,表示带宽参数;In the formula, represents bandwidth parameter;

步骤3.2.4:根据公式(6)-(7)计算源域数据与目标域数据生成的隐含向量间的差异,以构建DADAE模型;以最小化DADAE模型目标函数为目标,DADAE模型目标函数如下:Step 3.2.4: Calculate the difference between the implicit vectors generated by the source domain data and the target domain data according to formulas (6)-(7) to build the DADAE model; the goal is to minimize the DADAE model objective function, which is as follows:

(8) (8)

式中,为式(3)所示的损失函数;为MMD距离损失函数;网络参数集合分别表示经过域自适应更新后编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,为平衡参数;In the formula, is the loss function shown in formula (3); is the MMD distance loss function; network parameter set They represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder after domain adaptive update, respectively. is the balance parameter;

步骤3.2.5:保存训练好的新的异常检测源域模型的网络参数,将该新的异常检测源域模型部署在隧道边缘计算节点。Step 3.2.5: Save the trained network parameters of the new anomaly detection source domain model and deploy the new anomaly detection source domain model on the tunnel edge computing node.

进一步,优选的是,步骤4中,计算用于异常检测的动态阈值,动态异常阈值的上限记为,下限记为;具体方法为:Furthermore, preferably, in step 4, a dynamic threshold for anomaly detection is calculated, and the upper limit of the dynamic anomaly threshold is recorded as , the lower limit is ; The specific method is:

步骤4.1:首先,将所述目标域数据集在更新后的异常检测源域模型中再次执行,得到经过编码器和解码器后的输出数据集,记为;接着,利用下式计算每条目标域数据的重构误差:Step 4.1: First, the target domain dataset Execute it again in the updated anomaly detection source domain model to obtain the output dataset after the encoder and decoder, denoted as ; Then, the reconstruction error of each target domain data is calculated using the following formula:

(9) (9)

式中,包含了个元素,表示为分别表示包含个网络流量的目标域数据集及其重构输出数据集;In the formula, Included elements, represented by , and Respectively include A target domain dataset of network traffic and its reconstructed output dataset;

步骤4.2:计算的平均值和标准差,计算方式如下:Step 4.2: Calculation The mean and standard deviation of are calculated as follows:

(10) (10)

(11) (11)

式中,表示的平均值,的标准差;则动态阈值范围为:In the formula, express The average value of for The standard deviation of the dynamic threshold is:

(12) (12)

(13) (13)

式中,为标准差系数。In the formula, is the standard deviation coefficient.

进一步,优选的是,为2。Furthermore, it is preferred that is 2.

进一步,优选的是,步骤5中,重构误差的计算方法如下:Further, preferably, in step 5, the calculation method of the reconstruction error is as follows:

; ;

式中,为所述待测预处理后网络流量样本经过更新后的异常检测源域模型编码后再解码的重构输出。In the formula, The reconstructed output is obtained by encoding and decoding the pre-processed network traffic sample to be tested through the updated anomaly detection source domain model.

进一步,优选的是,步骤6中的检测方法为:Further, preferably, the detection method in step 6 is:

时,标记为正常;当时,标记为异常。when When , it is marked as normal; when or , marked as abnormal.

本发明中,的取值可根据实际情况选择,本发明不对边界阈值做限制。In the present invention, The value of can be selected according to actual conditions, and the present invention does not limit the boundary threshold.

本发明所要解决的技术问题是:针对现有技术下以太网设备复杂、网络传输流量采样率较高导致的隧道网络异常检测困难问题,在隧道网络内引入边缘计算节点,直接在设备端对网络流量进行处理。然而,由于隧道内机电设备复杂多样导致采集到的网络数据网络流量特征维度较高及数据间呈现高度非线性等问题,很难建立有效的异常检测模型。此外,由于隧道网络运行环境随时间具有动态变化的特点,即时变性,只靠一成不变的模型容易出现随时间推移检测效果持续下降的情况,固定的异常阈值和检测模型对于隧道网络的异常检测任务没有较好的鲁棒性。The technical problem to be solved by the present invention is: in view of the difficulty in detecting anomalies in tunnel networks caused by the complexity of Ethernet devices and the high sampling rate of network transmission traffic under the existing technology, edge computing nodes are introduced in the tunnel network to process the network traffic directly on the device side. However, due to the complexity and diversity of electromechanical equipment in the tunnel, the network traffic characteristics of the collected network data have high dimensions and the data is highly nonlinear, making it difficult to establish an effective anomaly detection model. In addition, since the operating environment of the tunnel network has the characteristics of dynamic change over time and instant variability, relying solely on an unchanging model is prone to a continuous decline in detection effect over time. Fixed anomaly thresholds and detection models do not have good robustness for the anomaly detection task of the tunnel network.

针对以上情况,本发明提出一种基于域自适应深度自编码器(Domain AdaptiveDeep Autoencoder, DADAE)的隧道网络异常检测方法。本发明方法将边缘计算架构引入隧道机电系统,由边缘计算节点采集并获取不同业务对应的网络流量特征并完成异常检测任务。对于隧道网络异常检测模型构建困难的问题,本发明引入迁移学习的思想,设计一种域自适应深度自编码器算法实现实时更新检测网络状态。从迁移学习的角度来说,在本发明中,将隧道机电系统产生的一段历史网络流量视为源域,由于数据的时变特点,实时采集的待检测网络流量的分布在一定程度上不匹配历史网络流量,考虑到网络流量在相邻时间周期内具有很强的相关性,因此对于待检测流量样本,取一定时间窗口内的样本视为目标域。本发明的目的是在隧道边缘端直接对数据进行采集并处理,利用迁移学习的思想来提高异常检测模型对于时变流量样本的适应性,提高监控系统的处理速度、有效缩减处理时延,提高异常检测模型的鲁棒性及准确性。In view of the above situation, the present invention proposes a tunnel network anomaly detection method based on Domain Adaptive Deep Autoencoder (DADAE). The method of the present invention introduces the edge computing architecture into the tunnel electromechanical system, and the edge computing node collects and obtains the network traffic characteristics corresponding to different services and completes the anomaly detection task. For the problem of the difficulty in building a tunnel network anomaly detection model, the present invention introduces the idea of transfer learning and designs a domain adaptive deep autoencoder algorithm to realize real-time update of the detection network state. From the perspective of transfer learning, in the present invention, a section of historical network traffic generated by the tunnel electromechanical system is regarded as the source domain. Due to the time-varying characteristics of the data, the distribution of the network traffic to be detected collected in real time does not match the historical network traffic to a certain extent. Considering that the network traffic has a strong correlation in adjacent time periods, for the traffic samples to be detected, the samples within a certain time window are regarded as the target domain. The purpose of the present invention is to directly collect and process the data at the edge of the tunnel, and use the idea of transfer learning to improve the adaptability of the anomaly detection model to time-varying traffic samples, improve the processing speed of the monitoring system, effectively reduce the processing delay, and improve the robustness and accuracy of the anomaly detection model.

具体来说,首先,针对隧道网络特征维度高,数据间非线性特点,将历史正常网络流量视为源域数据建立源域自动编码器模型,以初步构建隧道网络的非线性拟合关系。接着,当边缘节点实时采集到的待检测网络样本到来后,通过滑动窗口确定待测样本对应的目标域数据,并以目标域数据对源域自编码器模型进行域自适应更新。最后待测样本输入到更新后的模型计算其重构损失,利用构建的异常检测模块,检测待测样本是否为异常网络流量样本。Specifically, firstly, considering the high dimension of tunnel network features and the nonlinear characteristics between data, the historical normal network traffic is regarded as the source domain data to establish the source domain autoencoder model, so as to preliminarily construct the nonlinear fitting relationship of the tunnel network. Then, when the network samples to be tested collected by the edge node in real time arrive, the target domain data corresponding to the samples to be tested is determined through the sliding window, and the source domain autoencoder model is domain-adaptively updated with the target domain data. Finally, the samples to be tested are input into the updated model to calculate their reconstruction loss, and the constructed anomaly detection module is used to detect whether the samples to be tested are abnormal network traffic samples.

本发明与现有技术相比,其有益效果为:Compared with the prior art, the present invention has the following beneficial effects:

(1)通过本发明提出的一种域自适应深度自编码器的隧道网络异常检测方法,可以在边缘侧直接对隧道网络进行预处理、异常检测等操作,提高监控系统的处理速度、有效缩减处理时延。(1) Through the tunnel network anomaly detection method of a domain adaptive deep autoencoder proposed in the present invention, preprocessing, anomaly detection and other operations can be performed directly on the edge side of the tunnel network, thereby improving the processing speed of the monitoring system and effectively reducing the processing delay.

(2)针对隧道网络环境随时间动态变化的特点,通过本发明提供的域自适应深度自编码器算法,可以令异常检测算法自适应匹配待检测网络流量样本,提高异常检测算法的鲁棒性及准确性。(2) In view of the fact that the tunnel network environment changes dynamically over time, the domain adaptive deep autoencoder algorithm provided by the present invention can enable the anomaly detection algorithm to adaptively match the network traffic samples to be detected, thereby improving the robustness and accuracy of the anomaly detection algorithm.

(3)本发明提供的动态确定异常阈值的方法,基于当前网络状态的正常基准值,合理设置异常阈值范围,避免了固定设置阈值导致的异常信息漏报、误报等情况。(3) The method for dynamically determining the abnormal threshold provided by the present invention reasonably sets the abnormal threshold range based on the normal baseline value of the current network status, thereby avoiding the omission and false alarm of abnormal information caused by the fixed setting of the threshold.

(4)通过应用本发明在高速公路隧道内部边缘计算节点中,可以提高网络流量异常检测任务的鲁棒性及准确性,降低高速公路隧道运营管理成本。(4) By applying the present invention to the edge computing nodes inside highway tunnels, the robustness and accuracy of network traffic anomaly detection tasks can be improved, and the operation and management costs of highway tunnels can be reduced.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1是本发明基于域自适应深度自编码器的隧道网络异常检测的各层架构图;FIG1 is a diagram of each layer architecture of tunnel network anomaly detection based on domain adaptive deep autoencoder of the present invention;

图2是本发明基于域自适应深度自编码器的隧道网络异常检测方法的流程图;FIG2 is a flow chart of a tunnel network anomaly detection method based on a domain adaptive deep autoencoder according to the present invention;

图3是本发明以本发明基于域自适应深度自编码器的隧道网络异常检测方法的算法图。FIG3 is an algorithm diagram of the tunnel network anomaly detection method based on the domain adaptive deep autoencoder of the present invention.

具体实施方式DETAILED DESCRIPTION

下面结合实施例对本发明作进一步的详细描述。The present invention is further described in detail below in conjunction with embodiments.

本领域技术人员将会理解,下列实施例仅用于说明本发明,而不应视为限定本发明的范围。实施例中未注明具体技术或条件者,按照本领域内的文献所描述的技术或条件或者按照产品说明书进行。所用材料或设备未注明生产厂商者,均为可以通过购买获得的常规产品。Those skilled in the art will appreciate that the following examples are only used to illustrate the present invention and should not be considered to limit the scope of the present invention. If no specific techniques or conditions are specified in the examples, the techniques or conditions described in the literature in the art or the product specifications are used. If the manufacturer of the materials or equipment used is not specified, they are all conventional products that can be purchased.

实施例1 一种基于域自适应深度自编码器的隧道网络异常检测方法,包括如下步骤:Embodiment 1 A method for detecting anomalies in a tunnel network based on a domain adaptive deep autoencoder comprises the following steps:

步骤1:通过部署在隧道内的边缘计算节点收集隧道机电系统设备层中设备的历史网络流量数据,解析获取对应的网络原始数据流,并进行数据预处理,得到对应的网络流量特征,即预处理后的网络流量样本;Step 1: The edge computing nodes deployed in the tunnel collect the historical network traffic data of the equipment in the tunnel electromechanical system equipment layer, parse and obtain the corresponding network raw data stream, and perform data preprocessing to obtain the corresponding network traffic characteristics, that is, the preprocessed network traffic samples;

步骤2:采集到的历史网络流量数据经步骤1处理获得每个网络流量数据对应的网络流量特征后,将其作为源域数据集;以源域数据集作为训练集,基于深度自编码器算法训练异常检测源域模型,训练完成后将异常检测源域模型部署在隧道边缘计算节点中;Step 2: After the collected historical network traffic data is processed in step 1 to obtain the network traffic features corresponding to each network traffic data, it is used as the source domain data set; the source domain data set is used as the training set, and the anomaly detection source domain model is trained based on the deep autoencoder algorithm. After the training is completed, the anomaly detection source domain model is deployed in the tunnel edge computing node;

步骤3:实时采集的网络流量数据经过步骤1所述的预处理方式得到对应的网络流量特征后,构建自适应滑动窗口算法获取该网络流量对应的目标域数据集;依据所对应的目标域数据集对步骤2获得的异常检测源域模型进行更新;Step 3: After the real-time collected network traffic data is preprocessed in the manner described in step 1 to obtain the corresponding network traffic features, an adaptive sliding window algorithm is constructed to obtain the target domain data set corresponding to the network traffic; the anomaly detection source domain model obtained in step 2 is updated according to the corresponding target domain data set;

步骤4:计算用于异常检测的动态阈值;Step 4: Calculate the dynamic threshold for anomaly detection;

步骤5:将实时采集的待测网络流量数据预处理后的网络流量特征输入到更新后的异常检测源域模型,计算其重构误差;Step 5: Input the network traffic features after preprocessing of the real-time collected network traffic data to be tested into the updated anomaly detection source domain model, and calculate its reconstruction error;

步骤6:根据步骤4获得的动态阈值及步骤5获得的重构误差,以检测实时采集的待测网络流量数据是否为异常数据。Step 6: Based on the dynamic threshold obtained in step 4 and the reconstruction error obtained in step 5, detect whether the network traffic data to be tested collected in real time is abnormal data.

实施例2 一种基于域自适应深度自编码器的隧道网络异常检测方法,包括如下步骤:Embodiment 2 A method for detecting anomalies in a tunnel network based on a domain adaptive deep autoencoder comprises the following steps:

步骤1:通过部署在隧道内的边缘计算节点收集隧道机电系统设备层中设备的历史网络流量数据,解析获取对应的网络原始数据流,并进行数据预处理,得到对应的网络流量特征,即预处理后的网络流量样本;Step 1: The edge computing nodes deployed in the tunnel collect the historical network traffic data of the equipment in the tunnel electromechanical system equipment layer, parse and obtain the corresponding network raw data stream, and perform data preprocessing to obtain the corresponding network traffic characteristics, that is, the preprocessed network traffic samples;

步骤2:采集到的历史网络流量数据经步骤1处理获得每个网络流量数据对应的网络流量特征后,将其作为源域数据集;以源域数据集作为训练集,基于深度自编码器算法训练异常检测源域模型,训练完成后将异常检测源域模型部署在隧道边缘计算节点中;Step 2: After the collected historical network traffic data is processed in step 1 to obtain the network traffic features corresponding to each network traffic data, it is used as the source domain data set; the source domain data set is used as the training set, and the anomaly detection source domain model is trained based on the deep autoencoder algorithm. After the training is completed, the anomaly detection source domain model is deployed in the tunnel edge computing node;

步骤3:实时采集的网络流量数据经过步骤1所述的预处理方式得到对应的网络流量特征后,构建自适应滑动窗口算法获取该网络流量对应的目标域数据集;依据所对应的目标域数据集对步骤2获得的异常检测源域模型进行更新;Step 3: After the real-time collected network traffic data is preprocessed in the manner described in step 1 to obtain the corresponding network traffic features, an adaptive sliding window algorithm is constructed to obtain the target domain data set corresponding to the network traffic; the anomaly detection source domain model obtained in step 2 is updated according to the corresponding target domain data set;

步骤4:计算用于异常检测的动态阈值;Step 4: Calculate the dynamic threshold for anomaly detection;

步骤5:将实时采集的待测网络流量数据预处理后的网络流量特征输入到更新后的异常检测源域模型,计算其重构误差;Step 5: Input the network traffic features after preprocessing of the real-time collected network traffic data to be tested into the updated anomaly detection source domain model, and calculate its reconstruction error;

步骤6:根据步骤4获得的动态阈值及步骤5获得的重构误差,以检测实时采集的待测网络流量数据是否为异常数据。Step 6: Based on the dynamic threshold obtained in step 4 and the reconstruction error obtained in step 5, detect whether the network traffic data to be tested collected in real time is abnormal data.

步骤1中,利用边缘计算节点进行隧道网络异常检测的系统架构包括设备层、边缘计算层、网络层和云平台层;设备层、边缘计算层、网络层和云平台层顺序连接;设备层中的各个设备系统包括广播电话系统、隧道监控系统、隧道通风照明系统、隧道区域控制器、隧道消防系统、信息发布系统和隧道交通信号系统;所述的边缘计算层为隧道内部署的边缘计算节点。In step 1, the system architecture for tunnel network anomaly detection using edge computing nodes includes a device layer, an edge computing layer, a network layer, and a cloud platform layer; the device layer, the edge computing layer, the network layer, and the cloud platform layer are connected sequentially; the various device systems in the device layer include a broadcast telephone system, a tunnel monitoring system, a tunnel ventilation and lighting system, a tunnel area controller, a tunnel fire protection system, an information release system, and a tunnel traffic signal system; the edge computing layer is an edge computing node deployed in the tunnel.

步骤1中,所述的数据预处理方式包括去除异常数据、去除无意义特征和数据归一化;In step 1, the data preprocessing method includes removing abnormal data, removing meaningless features and normalizing data;

网络流量特征包括数据流持续时间、正向包的数量、反向包的数量、正向包的总字节数、反向包的总字节数、正向包头总字节数、反向包头总字节数、正向子流总字节数和反向子流总字节数。The network traffic characteristics include data flow duration, the number of forward packets, the number of reverse packets, the total bytes of forward packets, the total bytes of reverse packets, the total bytes of forward packet headers, the total bytes of reverse packet headers, the total bytes of forward subflows, and the total bytes of reverse subflows.

步骤2中,所述的异常检测源域模型采用深度自动编码器,包括编码器和解码器;In step 2, the anomaly detection source domain model adopts a deep autoencoder, including an encoder and a decoder;

所述的异常检测源域模型具有三层神经网络,分别为输入层、隐含层和输出层,输入为;其中,表示源域数据集,表示第个预处理后的网络流量样本;The anomaly detection source domain model has a three-layer neural network, namely, input layer, hidden layer and output layer. , ;in, represents the source domain dataset, Indicates Preprocessed network traffic samples;

异常检测源域模型的具体训练方法如下:The specific training method of the anomaly detection source domain model is as follows:

步骤2.1:编码器将源域数据经过激活函数映射得到隐含层数据:Step 2.1: Encoder converts source domain data After activation function Mapping gets the hidden layer data:

; ;

式中,表示隐含层向量,表示第个预处理后的网络流量样本的隐含层向量;In the formula, represents the hidden layer vector, Indicates The hidden layer vector of the preprocessed network traffic samples;

编码过程如式(1)所示:The encoding process is shown in formula (1):

(1) (1)

式中,分别表示编码器的网络权重和偏置向量,为激活函数,在本发明中为Sigmoid函数;In the formula, and Represent the network weights and bias vectors of the encoder respectively, Is the activation function, which is the Sigmoid function in the present invention;

步骤2.2:解码器通过激活函数将隐含层数据转化到输出层获得输出变量:Step 2.2: The decoder passes the activation function The hidden layer data Transform to the output layer to obtain the output variable:

; ;

式中,表示重构的输出变量,表示第个重构的网络流量样本;In the formula, represents the reconstructed output variable, Indicates reconstructed network traffic samples;

经隐含层向量重构了输入变量,解码过程如式(2)所示; The hidden layer vector The input variables are reconstructed and the decoding process is shown in formula (2);

(2) (2)

式中,分别表示解码器的网络权重和偏置向量,为激活函数;In the formula, and Represent the network weights and bias vectors of the decoder respectively, is the activation function;

步骤2.3:利用梯度下降算法对异常检测源域模型进行训练,以最小化重构的误差为目标,得到最佳网络参数;目标函数如式(3)所示:Step 2.3: Use the gradient descent algorithm to train the anomaly detection source domain model, with the goal of minimizing the reconstruction error and obtaining the optimal network parameters; the objective function is shown in formula (3):

(3) (3)

式中,网络参数集合分别表示编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,分别表示第个DAE网络的输入和重构输出变量;M表示预处理后的网络流量样本的总数;In the formula, the network parameter set Respectively represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder, and Respectively represent The input and reconstructed output variables of a DAE network; M represents the total number of preprocessed network traffic samples;

步骤2.4:保存训练好的异常检测源域模型的网络参数,将该模型部署在隧道边缘计算节点。Step 2.4: Save the network parameters of the trained anomaly detection source domain model and deploy the model on the tunnel edge computing node.

为Sigmoid函数。 is the Sigmoid function.

步骤3的具体方法为:The specific method of step 3 is:

步骤3.1:假设在时刻,边缘计算节点实时采集的网络流量数据经过步骤1所述的预处理得到对应的网络流量特征,即预处理后的网络流量样本,定义为待测样本;利用自适应滑动窗口算法构建目标域数据集;具体方法为:Step 3.1: Assume that At this moment, the network traffic data collected in real time by the edge computing node is preprocessed as described in step 1 to obtain the corresponding network traffic features, that is, the preprocessed network traffic sample , defined as the sample to be tested; the target domain dataset is constructed using the adaptive sliding window algorithm ; The specific method is:

步骤3.1.1:以时刻的预处理后的网络流量样本为滑动窗口的右边界,向前序的预处理后的网络流量样本进行扩张,将时序临近预处理后的网络流量样本归属到滑动窗口内,则自适应滑动窗口数据集表示为:Step 3.1.1: Preprocessed network traffic sample at time As the right boundary of the sliding window, the network traffic samples pre-processed in the previous order are expanded, and the network traffic samples pre-processed in the adjacent time sequence are attributed to the sliding window. Then the adaptive sliding window data set It is expressed as:

; ;

式中,表示长度为的滑动窗口,包含时刻到时刻内的预处理后的网络流量样本;为窗口的左边界预处理后的网络流量样本,即自适应滑动窗口以时刻向前序扩张个预处理后的网络流量样本;In the formula, Indicates the length is The sliding window contains Time has come Preprocessed network traffic samples within time; is the network traffic sample preprocessed at the left edge of the window, that is, the adaptive sliding window is Time forward expansion Preprocessed network traffic samples;

步骤3.1.2:自适应滑动窗口在确定是否扩张前序样本时,假设滑动窗口已扩张到时刻,待判断是否纳入窗口的样本为时刻;首先,依据如下所示的相似性函数计算该时刻样本与当前窗口内部所有样本的欧式距离平均值:Step 3.1.2: When the adaptive sliding window determines whether to expand the previous sample, it is assumed that the sliding window has been expanded to At time , the sample to be judged whether to be included in the window is Moment; first, the average Euclidean distance between the sample at that moment and all samples in the current window is calculated based on the similarity function shown below:

(4) (4)

其中,ED的计算方式为:The calculation method of ED is:

; ;

式中,表示当前滑动窗口内从时刻到时刻内的任意预处理后的网络流量样本,为当前窗口内的预处理后的网络流量样本个数,则为待判断是否纳入窗口的前序预处理后的网络流量样本;In the formula, Indicates that the current sliding window is from Time has come Any preprocessed network traffic sample within time, is the number of preprocessed network traffic samples in the current window, It is the network traffic sample after pre-processing to be judged whether to be included in the window;

步骤3.1.3:根据所述步骤3.1.2的相似性函数设定自适应滑动窗口的边界阈值,若,滑动窗口将纳入该预处理后的网络流量样本,即窗口的左边界预处理后的网络流量样本为,反之则停止扩张,此时的左边界样本为Step 3.1.3: Set the boundary threshold of the adaptive sliding window according to the similarity function of step 3.1.2 ,like , the sliding window will include the preprocessed network traffic sample, that is, the preprocessed network traffic sample at the left boundary of the window is , otherwise the expansion stops, and the left boundary sample is ;

令滑动窗口数据集作为的目标域数据,则目标域数据集表示为:Let the sliding window dataset As The target domain data is represented as:

; ;

由于中共有条预处理后的网络流量样本,因此目标域数据集表示为:because The CCP has preprocessed network traffic samples, so the target domain dataset It is expressed as:

; ;

式中,表示在包含条预处理后的网络流量样本的数据集内第个预处理后的网络流量样本;In the formula, Including A dataset of preprocessed network traffic samples Neidi Preprocessed network traffic samples;

步骤3.2:利用目标域数据集对步骤2训练的异常检测源域模型进行域自适应更新;具体步骤如下:Step 3.2: Leverage the target domain dataset Perform domain adaptive update on the anomaly detection source domain model trained in step 2; the specific steps are as follows:

步骤3.2.1:首先,将源域数据集输入到训练好的异常检测源域模型中,通过公式(1)前向传播获取源域数据的隐含层向量Step 3.2.1: First, the source domain dataset Input it into the trained anomaly detection source domain model, and obtain the hidden layer vector of the source domain data through forward propagation of formula (1) ;

步骤3.2.2:将目标域数据同样输入到异常检测源域模型中通过如下公式前向传播获取目标域数据的隐含层向量Step 3.2.2: Target domain data The same input is used to the anomaly detection source domain model and the hidden layer vector of the target domain data is obtained by forward propagation through the following formula: :

(5) (5)

步骤3.2.3:以最大平均差距离作为目标函数,计算公式如式(6)所示:Step 3.2.3: Take the maximum average difference distance as the objective function, and the calculation formula is shown in formula (6):

; ;

(6) (6)

式中,分别为内的样本个数,为求取最小上界函数,在式中指代数据集中的任意索引,即分别表示中第个样本,分别表示中第个样本;为高斯核函数,计算方式如下:In the formula, They are and The number of samples in To find the minimum upper bound function, In the formula, refers to any index in the data set, that is, and Respectively Middle and samples, and Respectively Middle and samples; is the Gaussian kernel function, The calculation is as follows:

(7) (7)

式中,表示带宽参数;In the formula, represents bandwidth parameter;

步骤3.2.4:根据公式(6)-(7)计算源域数据与目标域数据生成的隐含向量间的差异,以构建DADAE模型;以最小化DADAE模型目标函数为目标,DADAE模型目标函数如下:Step 3.2.4: Calculate the difference between the implicit vectors generated by the source domain data and the target domain data according to formulas (6)-(7) to build the DADAE model; the goal is to minimize the DADAE model objective function, which is as follows:

(8) (8)

式中,为式(3)所示的损失函数;为MMD距离损失函数;网络参数集合分别表示经过域自适应更新后编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,为平衡参数;In the formula, is the loss function shown in formula (3); is the MMD distance loss function; network parameter set They represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder after domain adaptive update, respectively. is the balance parameter;

步骤3.2.5:保存训练好的新的异常检测源域模型的网络参数,将该新的异常检测源域模型部署在隧道边缘计算节点。Step 3.2.5: Save the trained network parameters of the new anomaly detection source domain model and deploy the new anomaly detection source domain model on the tunnel edge computing node.

步骤4中,计算用于异常检测的动态阈值,动态异常阈值的上限记为,下限记为;具体方法为:In step 4, the dynamic threshold for anomaly detection is calculated, and the upper limit of the dynamic anomaly threshold is recorded as , the lower limit is ; The specific method is:

步骤4.1:首先,将所述目标域数据集在更新后的异常检测源域模型中再次执行,得到经过编码器和解码器后的输出数据集,记为;接着,利用下式计算每条目标域数据的重构误差:Step 4.1: First, the target domain dataset Execute it again in the updated anomaly detection source domain model to obtain the output dataset after the encoder and decoder, denoted as ; Then, the reconstruction error of each target domain data is calculated using the following formula:

(9) (9)

式中,包含了个元素,表示为分别表示包含个网络流量的目标域数据集及其重构输出数据集;In the formula, Included elements, represented by , and Respectively include A target domain dataset of network traffic and its reconstructed output dataset;

步骤4.2:计算的平均值和标准差,计算方式如下:Step 4.2: Calculation The mean and standard deviation of are calculated as follows:

(10) (10)

(11) (11)

式中,表示的平均值,的标准差;则动态阈值范围为:In the formula, express The average value of for The standard deviation of the dynamic threshold is:

(12) (12)

(13) (13)

式中,为标准差系数。In the formula, is the standard deviation coefficient.

为2。 is 2.

步骤5中,重构误差的计算方法如下:In step 5, the reconstruction error is calculated as follows:

; ;

式中,为所述待测预处理后网络流量样本经过更新后的异常检测源域模型编码后再解码的重构输出。In the formula, The reconstructed output is obtained by encoding and decoding the pre-processed network traffic sample to be tested through the updated anomaly detection source domain model.

步骤6中的检测方法为:The detection method in step 6 is:

时,标记为正常;当时,标记为异常。when When , it is marked as normal; when or , marked as abnormal.

实施例3 如图1所示,由于隧道内网络流量异常识别困难、定位不准确,引入边缘计算架构,本发明提出一种基于域自适应深度自编码器的隧道网络异常检测方法。隧道机电系统的边缘计算架构分为设备层、边缘计算层、网络层及云平台层。其中,设备层主要为隧道感知设备和控制设备,如广播电话系统、隧道监控系统、隧道通风照明系统、隧道区域控制器、隧道消防系统、信息发布系统、隧道交通信号系统等。考虑现有技术下以太网设备复杂、网络传输流量采样率较高导致的隧道网络异常检测困难问题,在隧道内部署边缘计算节点,管理周边大量前端设备并进行数据采集、处理,包括模型训练、域自适应更新及网络异常检测等功能。由于隧道内机电设备复杂多样导致采集到的网络流量特征维度较高且数据间呈现高度非线性等问题,很难建立有效的异常检测模型,本发明构建一种基于域自适应深度自编码器的隧道网络异常检测方法。Embodiment 3 As shown in Figure 1, due to the difficulty in identifying network traffic anomalies in tunnels and inaccurate positioning, an edge computing architecture is introduced. The present invention proposes a tunnel network anomaly detection method based on a domain adaptive deep autoencoder. The edge computing architecture of the tunnel electromechanical system is divided into a device layer, an edge computing layer, a network layer, and a cloud platform layer. Among them, the device layer mainly includes tunnel sensing devices and control devices, such as broadcast telephone systems, tunnel monitoring systems, tunnel ventilation and lighting systems, tunnel area controllers, tunnel fire protection systems, information release systems, tunnel traffic signal systems, etc. Considering the difficulty in detecting tunnel network anomalies caused by the complexity of Ethernet devices and the high sampling rate of network transmission traffic under the prior art, edge computing nodes are deployed in the tunnel to manage a large number of surrounding front-end devices and perform data collection and processing, including model training, domain adaptive updates, and network anomaly detection functions. Due to the complexity and diversity of electromechanical equipment in the tunnel, the collected network traffic features have high dimensions and are highly nonlinear between data. It is difficult to establish an effective anomaly detection model. The present invention constructs a tunnel network anomaly detection method based on a domain adaptive deep autoencoder.

如图2的流程图和图3的算法图所示,以隧道监控系统网络流量异常检测为例,本实例涉及的一种基于域自适应深度自编码器的隧道网络异常检测方法,具体包括以下步骤:As shown in the flowchart of FIG2 and the algorithm diagram of FIG3, taking the network traffic anomaly detection of the tunnel monitoring system as an example, this example involves a tunnel network anomaly detection method based on a domain adaptive deep autoencoder, which specifically includes the following steps:

步骤1:以隧道监控系统网络流量异常检测为例,通过部署在隧道内的边缘计算节点收集隧道监控系统的历史网络流量数据,利用现有常规方式解析获取对应的网络原始数据流,并进行数据预处理得到该业务对应的网络流量特征。Step 1: Taking the network traffic anomaly detection of the tunnel monitoring system as an example, the historical network traffic data of the tunnel monitoring system is collected by the edge computing nodes deployed in the tunnel, and the corresponding network raw data stream is parsed using the existing conventional methods, and the data is preprocessed to obtain the network traffic characteristics corresponding to the business.

所述的数据预处理方式包括去除异常数据、去除无意义特征、数据归一化。其中,去除异常数据操作为所采集的监控系统网络流量中,经人工判断,保留正常的历史数据供后续建模使用。去除无意义特征操作包括去去除无意义特征,如IP地址、端口号、时间戳等,并将隧道网络的各种特征数据转换为可处理的数据。所述的网络流量特征包括数据流的基本特征、协议连接的内容特征、基于时间的流量统计特征、连接特征。可选的,包括但不限于数据流持续时间、正向包的数量、反向包的数量、正向包的总字节数、反向包的总字节数、正向包头总字节数、反向包头总字节数、正向子流总字节数和反向子流总字节数这几个特征用于建模及异常检测。所述的数据归一化为以流量特征的最大最小值为基准,对数据采用最大最小归一化,保证所有数据的取值范围在[0,1]区间。The data preprocessing method includes removing abnormal data, removing meaningless features, and normalizing data. Among them, the operation of removing abnormal data is to retain normal historical data for subsequent modeling after manual judgment in the collected monitoring system network traffic. The operation of removing meaningless features includes removing meaningless features, such as IP addresses, port numbers, timestamps, etc., and converting various feature data of the tunnel network into processable data. The network traffic features include basic features of data streams, content features of protocol connections, time-based traffic statistics features, and connection features. Optionally, several features including but not limited to data stream duration, the number of forward packets, the number of reverse packets, the total number of bytes of forward packets, the total number of bytes of reverse packets, the total number of bytes of forward headers, the total number of bytes of reverse headers, the total number of bytes of forward substreams, and the total number of bytes of reverse substreams are used for modeling and anomaly detection. The data normalization is based on the maximum and minimum values of the traffic features, and the maximum and minimum normalization is applied to the data to ensure that the value range of all data is in the interval [0,1].

步骤2:此步骤为离线部分,采集到的隧道监控系统历史网络流量数据经步骤1所述的预处理方法获得每个流量样本对应的特征后,将其作为源域数据集。以源域数据集作为训练集,基于深度自编码器(Deep Autoencoder, DAE)算法训练异常检测源域模型,训练完成后将DAE部署在隧道边缘计算节点中。其中,深度自动编码器是一种包括编码器和解码器的无监督神经网络模型,它可以学习到隧道监控系统网络流量输入数据的隐含特征(编码器),并利用学习到的隐含特征重构输入特征(解码器),DAE的原理就是使得解码器的输出尽可能的还原输入。假设用于训练的源域数据集表示为:Step 2: This step is the offline part. The collected historical network traffic data of the tunnel monitoring system is used as the source domain data set after the features corresponding to each traffic sample are obtained by the preprocessing method described in step 1. The source domain data set is used as the training set, and the anomaly detection source domain model is trained based on the Deep Autoencoder (DAE) algorithm. After the training is completed, the DAE is deployed in the tunnel edge computing node. Among them, the deep autoencoder is an unsupervised neural network model including an encoder and a decoder. It can learn the implicit features of the network traffic input data of the tunnel monitoring system (encoder) and use the learned implicit features to reconstruct the input features (decoder). The principle of DAE is to make the output of the decoder restore the input as much as possible. Assume that the source domain data set used for training is expressed as:

;

其中,表示源域数据集,表示第个预处理后的网络流量样本。in, represents the source domain dataset, Indicates A sample of network traffic after preprocessing.

本发明对DAE算法的隐含层数目和隐含层神经元的个数不做限制,如图3算法图中的自编码器结构图,以上述隧道监控系统网络流量为例,基于DAE算法的异常检测源域模型具有三层神经网络,分别为输入层、隐含层和输出层,输入为,DAE的具体训练方式如下:The present invention does not limit the number of hidden layers and neurons in the hidden layers of the DAE algorithm. As shown in the structure diagram of the autoencoder in the algorithm diagram of FIG3 , taking the network traffic of the tunnel monitoring system as an example, the anomaly detection source domain model based on the DAE algorithm has a three-layer neural network, namely, an input layer, a hidden layer and an output layer. The input is , the specific training method of DAE is as follows:

步骤2.1:编码器将源域数据逐一经过激活函数映射得到隐含层数据:Step 2.1: Encoder converts source domain data Pass through the activation function one by one Mapping gets the hidden layer data:

; ;

式中,表示DAE的隐含层向量,表示第个网络流量样本的隐含层向量;In the formula, represents the hidden layer vector of DAE, Indicates Hidden layer vector of network traffic samples;

编码过程如式(1)所示:The encoding process is shown in formula (1):

(1) (1)

式中,分别表示编码器的网络权重和偏置向量,为激活函数,在本发明中为Sigmoid函数。In the formula, and Represent the network weights and bias vectors of the encoder respectively, is an activation function, which is a Sigmoid function in the present invention.

步骤2.2:解码器通过激活函数将隐含层数据转化到输出层获得输出变量:Step 2.2: The decoder passes the activation function The hidden layer data Transform to the output layer to obtain the output variable:

; ;

式中,表示重构的输出变量,表示第个重构的网络流量样本;在此步骤中经隐含层向量重构了输入变量,解码过程如式(2)所示;In the formula, represents the reconstructed output variable, Indicates reconstructed network traffic samples; in this step The hidden layer vector The input variables are reconstructed, and the decoding process is shown in formula (2);

(2) (2)

式中,分别表示解码器的网络权重和偏置向量,为激活函数,在本发明中为Sigmoid函数。In the formula, and Represent the network weights and bias vectors of the decoder respectively, is an activation function, which is a Sigmoid function in the present invention.

步骤2.3:利用梯度下降算法对DAE进行训练,通过最小化重构的误差来实现,得到最佳网络参数;训练过程中所需优化的目标损失函数如式(3)所示:Step 2.3: Use the gradient descent algorithm to train the DAE by minimizing the reconstruction error to obtain the optimal network parameters; the target loss function to be optimized during the training process is shown in formula (3):

(3) (3)

式中,网络参数集合分别表示编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,分别表示第个DAE网络的输入和重构输出变量;M表示网络流量样本的总数;In the formula, the network parameter set Respectively represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder, and Respectively represent The input and reconstructed output variables of a DAE network; M represents the total number of network traffic samples;

步骤2.4:保存训练好的DAE网络参数,将该模型部署在隧道边缘计算节点,作为异常检测的源域模型。Step 2.4: Save the trained DAE network parameters and deploy the model on the tunnel edge computing node as the source domain model for anomaly detection.

需要说明的是,此步骤2中利用源域数据训练的DAE模型,便可以对隧道监控系统实时产生的网络流量进行异常检测,但是,在隧道内终端设备网络环境中,网络流量随时间动态变化,采用一成不变的模型在一定程度上会使得异常检测的鲁棒性较低。It should be noted that the DAE model trained with source domain data in step 2 can perform anomaly detection on the network traffic generated in real time by the tunnel monitoring system. However, in the network environment of terminal equipment in the tunnel, the network traffic changes dynamically over time. Using a fixed model will make the robustness of anomaly detection lower to a certain extent.

步骤3:此步骤为在线部分,基于所述部署在隧道边缘计算节点上的DAE模型进行域自适更新。在本实例中,当边缘计算节点实时采集到隧道监控系统产生的网络流量后,需要检测异常时,通过本发明构建的域自适应更新策略对所述步骤2建立的DAE模型进行更新,该算法被定义为域自适应自编码器(Domain Adaptive Deep Autoencoder, DADAE)。具体的,DADAE的更新步骤如下:Step 3: This step is the online part, and domain adaptive update is performed based on the DAE model deployed on the tunnel edge computing node. In this example, when the edge computing node collects the network traffic generated by the tunnel monitoring system in real time and needs to detect anomalies, the DAE model established in step 2 is updated by the domain adaptive update strategy constructed by the present invention. The algorithm is defined as Domain Adaptive Deep Autoencoder (DADAE). Specifically, the update steps of DADAE are as follows:

步骤3.1:步骤3.1:假设在时刻,边缘计算节点实时采集的隧道监控系统产生的网络流量样本经过步骤1所述的预处理得到对应的网络流量特征,该样本预处理后表示为,也就是说为隧道监控系统的待异常检测的预处理后的网络流量样本,在本实例中定义为待测样本。Step 3.1: Assume that At this moment, the network traffic sample generated by the tunnel monitoring system collected in real time by the edge computing node is preprocessed as described in step 1 to obtain the corresponding network traffic characteristics. The sample is expressed as , that is to say The preprocessed network traffic samples to be detected for anomalies in the tunnel monitoring system are defined as samples to be tested in this example.

由于域自适应更新需要利用目标域对源域模型进行更新,而目标域要求与目标(即)在数据结构、特性等方面具有较高相似性,才能使得更新后的模型与目标匹配。根据网络流量在相邻时间周期内的强相关性特性,在本发明中,引入滑动窗口的理念,构建自适应滑动窗口算法获取对应的目标域数据集,具体步骤如下:Since domain adaptation requires updating the source domain model using the target domain, and the target domain requires the same ) have high similarity in data structure, characteristics, etc., so that the updated model can match the target. According to the strong correlation characteristics of network traffic in adjacent time periods, in this invention, the concept of sliding window is introduced to construct an adaptive sliding window algorithm to obtain The corresponding target domain dataset, the specific steps are as follows:

步骤3.1.1:以时刻的隧道监控系统网络流量样本为滑动窗口的右边界,向前序的预处理后的网络流量样本进行扩张,找到合适的时序临近网络流量归属到滑动窗口内,则自适应滑动窗口数据集可以表示为:Step 3.1.1: Tunnel monitoring system network traffic samples at the moment As the right boundary of the sliding window, expand the pre-processed network traffic samples in the previous order, find the appropriate time sequence of adjacent network traffic to belong to the sliding window, then the adaptive sliding window data set It can be expressed as:

; ;

式中,表示长度为的滑动窗口,包含时刻到时刻内的网络流量样本;需要说明的是,这些样本都是经过异常检测后的正常数据。为窗口的左边界预处理后的网络流量样本,即自适应滑动窗口以时刻向前序扩张个网络流量样本。In the formula, Indicates the length is The sliding window contains Time has come Network traffic samples within a certain time period; it should be noted that these samples are normal data after anomaly detection. is the network traffic sample preprocessed at the left edge of the window, that is, the adaptive sliding window is Time forward expansion A sample of network traffic.

步骤3.1.2:自适应滑动窗口在确定是否扩张前序样本时,假设滑动窗口已扩张到时刻,待判断是否纳入窗口的样本为时刻;首先,依据如下所示的相似性函数计算该时刻样本与当前窗口内部所有样本的欧式距离(Euclidean Distance, ED)平均值:Step 3.1.2: When the adaptive sliding window determines whether to expand the previous sample, it is assumed that the sliding window has been expanded to At time , the sample to be judged whether to be included in the window is Moment; first, the average Euclidean distance (ED) between the sample at that moment and all samples in the current window is calculated based on the similarity function shown below:

(4) (4)

其中,ED的计算方式为:The calculation method of ED is:

; ;

式中,表示当前滑动窗口内从时刻到时刻内的任意网络流量样本,为当前窗口内的网络流量样本个数,则为待判断是否纳入窗口的前序样本;In the formula, Indicates that the current sliding window is from Time has come Any network traffic sample within a certain time period, is the number of network traffic samples in the current window, It is the previous sample to be determined whether to be included in the window;

步骤3.1.3:根据所述步骤3.1.2的相似性函数设定自适应滑动窗口的边界阈值,若,滑动窗口将纳入该网络流量样本,即窗口的左边界样本为,反之则停止扩张,此时的左边界样本为Step 3.1.3: Set the boundary threshold of the adaptive sliding window according to the similarity function of step 3.1.2 ,like , the sliding window will include the network traffic sample, that is, the left boundary sample of the window is , otherwise the expansion stops, and the left boundary sample is .

通过所述步骤3.1,所得到的自适应滑动窗口内网络流量与所述待测样本之间有着强相关特性,令滑动窗口数据集作为的目标域数据,则目标域数据集表示为:The network traffic in the adaptive sliding window obtained through step 3.1 is consistent with the sample to be tested. There is a strong correlation between them, making the sliding window dataset As The target domain data is represented as:

; ;

由于中共有条网络流量样本,因此目标域数据集也可以表示为:because The CCP has network traffic samples, so the target domain dataset It can also be expressed as:

; ;

式中,表示在包含条样本的数据集内第个网络流量样本。In the formula, Including Dataset of samples Neidi A sample of network traffic.

步骤3.2:利用目标域数据集对步骤2训练的DAE进行域自适应更新。域自适应可以简单描述为源域和目标域之间模型相似性的域间知识转移,目的是为了发现并削弱两域之间的差异。因此,在动态变化的隧道终端设备网络环境下,所构建的DADAE可以自适应匹配待检测网络流量样本,提高异常检测的准确性和鲁棒性。具体步骤如下:Step 3.2: Leverage the target domain dataset Perform domain adaptation update on the DAE trained in step 2. Domain adaptation can be simply described as the inter-domain knowledge transfer of model similarity between the source domain and the target domain, with the purpose of discovering and weakening the differences between the two domains. Therefore, in the dynamically changing network environment of tunnel terminal equipment, the constructed DADAE can adaptively match the network traffic samples to be detected, improving the accuracy and robustness of anomaly detection. The specific steps are as follows:

步骤3.2.1:如图3所示,首先,将源域数据集输入到训练好的DAE中,通过公式(1)前向传播获取源域数据的隐含层向量Step 3.2.1: As shown in Figure 3, first, the source domain dataset Input into the trained DAE, and forward propagate through formula (1) to obtain the hidden layer vector of the source domain data ;

步骤3.2.2:将目标域数据同样输入到DAE中通过如下公式前向传播获取目标域数据的隐含层向量Step 3.2.2: Target domain data The same input is sent to DAE and the hidden layer vector of the target domain data is obtained by forward propagation through the following formula: :

(5) (5)

步骤3.2.3:将最大平均差(Maximum Mean Difference, MMD)距离引入DAE的目标函数中,以计算源域与目标域之间的数据差异。其中,之间的MMD计算如式(6)所示:Step 3.2.3: Introduce the Maximum Mean Difference (MMD) distance into the objective function of DAE to calculate the data difference between the source domain and the target domain. and The MMD calculation between is shown in formula (6):

; ;

(6) (6)

式中,分别为内的样本个数,为求取最小上界函数,在式中指代数据集中的任意索引。MMD旨在测量再生希尔伯特空间(Reproducing KernelHilbert Space)下两个域之间的距离,这是一种核学习的方法,MMD距离越小,两个数据域之间的相似性越高。为高斯核函数,计算方式如下:In the formula, They are and The number of samples in To find the minimum upper bound function, In the formula, it refers to any index in the data set. MMD aims to measure the distance between two domains in the reproducing Kernel Hilbert Space, which is a kernel learning method. The smaller the MMD distance, the higher the similarity between the two data domains. is the Gaussian kernel function, The calculation is as follows:

(7) (7)

式中,表示带宽参数,其取值与高斯核函数的宽度成正比,其取值常取为1。In the formula, Represents the bandwidth parameter, whose value is proportional to the width of the Gaussian kernel function and is usually 1.

步骤3.2.4:将目标域数据集直接输入步骤2离线训练好的DAE中,将MMD距离引入DAE的目标函数中,根据公式(6)-(7)计算源域数据与目标域数据生成的隐含向量间的差异,以构建DADAE模型。Step 3.2.4: Target domain dataset Directly input the DAE trained offline in step 2, introduce the MMD distance into the objective function of DAE, and calculate the difference between the implicit vectors generated by the source domain data and the target domain data according to formulas (6)-(7) to build the DADAE model.

由于基于前述步骤训练的网络参数,在此步骤中仅需少量迭代(即网络权重微调)便可实现模型的域自适应更新。本发明所构建的DADAE模型目标函数(以最小化目标函数为目标)如下:Due to the network parameters trained based on the previous steps, only a small number of iterations (i.e., fine-tuning of network weights) are required in this step to achieve domain adaptive update of the model. The objective function of the DADAE model constructed by the present invention (with the goal of minimizing the objective function) is as follows:

(8) (8)

式中,目标函数由两部分损失组成,分别是DAE的目标函数损失和源域数据的隐含向量与目标域数据的隐含向量之间的MMD距离损失;网络参数集合分别表示经过域自适应更新后编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,为DAE损失和域间MMD距离损失之间的平衡参数,其取值一般取0.5,在实施过程中可以上下微调,本发明不对的取值做约束。练方式仍为梯度下降算法。In the formula, the objective function consists of two parts of loss, namely the objective function loss of DAE and the MMD distance loss between the latent vector of the source domain data and the latent vector of the target domain data; the network parameter set They represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder after domain adaptive update, respectively. It is a balance parameter between DAE loss and inter-domain MMD distance loss. Its value is generally 0.5. It can be fine-tuned up or down during implementation. The present invention does not The training method is still the gradient descent algorithm.

步骤3.2.5:保存训练好的新的异常检测源域模型的网络参数,将该新的异常检测源域模型部署在隧道边缘计算节点。Step 3.2.5: Save the trained network parameters of the new anomaly detection source domain model and deploy the new anomaly detection source domain model on the tunnel edge computing node.

可以看出,本发明所构建的目标函数不仅充分利用了源域模型信息,且通过最小化目标函数,令更新后的网络权重和偏置趋向于目标域数据的特性,在一定程度上解决了静态模型无法适应动态变化的隧道机电设备网络环境的问题。It can be seen that the objective function constructed by the present invention not only makes full use of the source domain model information, but also makes the updated network weights and biases tend to the characteristics of the target domain data by minimizing the objective function, which to a certain extent solves the problem that the static model cannot adapt to the dynamically changing network environment of tunnel electromechanical equipment.

步骤4中基于所述待检测网络流量的目标域数据集计算用于异常检测的动态阈值,动态异常阈值的上限记为,下限记为In step 4, a dynamic threshold for anomaly detection is calculated based on the target domain data set of the network traffic to be detected. The upper limit of the dynamic anomaly threshold is recorded as , the lower limit is .

由于在隧道机电系统网络环境中,网络流量随时间动态变化,正常的流量的状态也将随着网络环境等相关因素不断更新。因此,对于边缘计算节点采集的隧道监控系统网络流量异常判断,应基于当前网络状态的正常基准值。根据所述步骤3.1基于自适应滑动窗口确定的目标域数据集具有时间强相关性,其网络状态受时间变化影响较小。因此,基于待检测网络流量的目标域数据确定动态阈值范围的具体步骤如下:Since the network traffic in the tunnel electromechanical system network environment changes dynamically over time, the state of normal traffic will also be continuously updated with relevant factors such as the network environment. Therefore, the judgment of abnormal network traffic of the tunnel monitoring system collected by the edge computing node should be based on the normal baseline value of the current network state. According to step 3.1, the target domain data set determined based on the adaptive sliding window has a strong time correlation, and its network state is less affected by time changes. Therefore, the specific steps for determining the dynamic threshold range based on the target domain data of the network traffic to be detected are as follows:

步骤4.1:首先,将所述目标域数据集在更新后的DADAE模型中再次执行,得到经过编码器和解码器后的输出数据集,记为。接着,利用下式计算每条目标域数据的重构误差:Step 4.1: First, the target domain dataset Execute it again in the updated DADAE model to get the output data set after the encoder and decoder, denoted as Next, the reconstruction error of each target domain data is calculated using the following formula:

(9) (9)

式中,包含了个元素,可表示为分别表示包含个网络流量的目标域数据集及其重构输出数据集。In the formula, Included elements, which can be expressed as , and Respectively include The target domain dataset of network traffic and its reconstructed output dataset.

步骤4.2:计算的平均值和标准差,计算方式如下:Step 4.2: Calculation The mean and standard deviation of are calculated as follows:

(10) (10)

(11) (11)

式中,表示的平均值,的标准差。则本发明所设定的动态阈值范围为:In the formula, express The average value of for The standard deviation of . Then the dynamic threshold range set by the present invention is:

(12) (12)

(13) (13)

式中,为标准差系数,本发明不限定的取值,例如可以为2。In the formula, is the standard deviation coefficient, and the present invention does not limit The value of, for example It can be 2.

步骤5中,对所述的待测样本输入到DADAE模型中进行推理,计算其重构误差,计算方式如下:In step 5, the sample to be tested is input into the DADAE model for inference, and its reconstruction error is calculated as follows:

; ;

式中,为所述待测样本经过DADAE编码后再解码的重构输出。In the formula, It is the reconstructed output of the sample to be tested after being encoded by DADAE and then decoded.

步骤6:根据所述待测样本的重构误差与所述动态误差阈值范围,判断所述待检测的实时隧道监控系统网络流量是否为异常数据。判断准则为:Step 6: Reconstruct the error of the sample to be tested and the dynamic error threshold range, to determine whether the network traffic of the real-time tunnel monitoring system to be detected is abnormal data. The judgment criteria are:

时,标记为正常;当时,标记为异常。when When , it is marked as normal; when or , marked as abnormal.

步骤7:当前待测样本结束异常检测后,下一时刻边缘计算节点采集到新的待检测网络流量后,按照前述步骤进行数据预处理、重新确定滑动窗口数据集、利用目标域数据对DAE模型进行域自适应更新、计算动态阈值范围、检测是否异常等操作。Step 7: After the anomaly detection of the current sample to be tested is completed, the edge computing node collects new network traffic to be tested at the next moment, and performs data preprocessing according to the above steps, re-determines the sliding window data set, uses the target domain data to perform domain adaptive update on the DAE model, calculates the dynamic threshold range, detects whether it is abnormal, and other operations.

以上显示和描述了本发明的基本原理、主要特征和本发明的优点。本行业的技术人员应该了解,本发明不受上述实施例的限制,上述实施例和说明书中描述的只是说明本发明的原理,在不脱离本发明精神和范围的前提下,本发明还会有各种变化和改进,这些变化和改进都落入要求保护的本发明范围内。本发明要求保护范围由所附的权利要求书及其等效物界定。The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims (8)

1.一种基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,包括如下步骤:1. A tunnel network anomaly detection method based on a domain adaptive deep autoencoder, characterized in that it comprises the following steps: 步骤1:通过部署在隧道内的边缘计算节点收集隧道机电系统设备层中设备的历史网络流量数据,解析获取对应的网络原始数据流,并进行数据预处理,得到对应的网络流量特征,即预处理后的网络流量样本;Step 1: The edge computing nodes deployed in the tunnel collect the historical network traffic data of the equipment in the tunnel electromechanical system equipment layer, parse and obtain the corresponding network raw data stream, and perform data preprocessing to obtain the corresponding network traffic characteristics, that is, the preprocessed network traffic samples; 步骤2:采集到的历史网络流量数据经步骤1处理获得每个网络流量数据对应的网络流量特征后,将其作为源域数据集;以源域数据集作为训练集,基于深度自编码器算法训练异常检测源域模型,训练完成后将异常检测源域模型部署在隧道边缘计算节点中;Step 2: After the collected historical network traffic data is processed in step 1 to obtain the network traffic features corresponding to each network traffic data, it is used as the source domain data set; the source domain data set is used as the training set, and the anomaly detection source domain model is trained based on the deep autoencoder algorithm. After the training is completed, the anomaly detection source domain model is deployed in the tunnel edge computing node; 步骤3:实时采集的网络流量数据经过步骤1所述的预处理方式得到对应的网络流量特征后,构建自适应滑动窗口算法获取该网络流量对应的目标域数据集;依据所对应的目标域数据集对步骤2获得的异常检测源域模型进行更新;Step 3: After the real-time collected network traffic data is preprocessed in the manner described in step 1 to obtain the corresponding network traffic features, an adaptive sliding window algorithm is constructed to obtain the target domain data set corresponding to the network traffic; the anomaly detection source domain model obtained in step 2 is updated according to the corresponding target domain data set; 步骤4:计算用于异常检测的动态阈值;Step 4: Calculate the dynamic threshold for anomaly detection; 步骤5:将实时采集的待测网络流量数据预处理后的网络流量特征输入到更新后的异常检测源域模型,计算其重构误差;Step 5: Input the network traffic features after preprocessing of the real-time collected network traffic data to be tested into the updated anomaly detection source domain model, and calculate its reconstruction error; 步骤6:根据步骤4获得的动态阈值及步骤5获得的重构误差,以检测实时采集的待测网络流量数据是否为异常数据;Step 6: According to the dynamic threshold obtained in step 4 and the reconstruction error obtained in step 5, whether the network traffic data to be tested collected in real time is abnormal data is detected; 步骤3的具体方法为:The specific method of step 3 is: 步骤3.1:假设在q时刻,边缘计算节点实时采集的网络流量数据经过步骤1所述的预处理得到对应的网络流量特征,即预处理后的网络流量样本xq,定义为待测样本;利用自适应滑动窗口算法构建目标域数据集Xt;具体方法为:Step 3.1: Assume that at time q, the network traffic data collected in real time by the edge computing node is preprocessed as described in step 1 to obtain the corresponding network traffic features, that is, the preprocessed network traffic sample xq is defined as the sample to be tested; the target domain data set Xt is constructed using the adaptive sliding window algorithm; the specific method is: 步骤3.1.1:以q-1时刻的预处理后的网络流量样本xq-1为滑动窗口的右边界,向前序的预处理后的网络流量样本进行扩张,将时序临近预处理后的网络流量样本归属到滑动窗口内,则自适应滑动窗口数据集DW表示为:Step 3.1.1: Take the preprocessed network traffic sample xq-1 at time q-1 as the right boundary of the sliding window, expand it to the preprocessed network traffic sample in the previous order, and attribute the preprocessed network traffic samples with close time sequence to the sliding window. Then the adaptive sliding window data set DW is expressed as: DW={xq-N,xt-N+1,...,xq-1};D W = {x qN , x t-N+1 ,..., x q-1 }; 式中,DW表示长度为N的滑动窗口,包含q-N时刻到q-1时刻内的预处理后的网络流量样本;xq-N为窗口的左边界预处理后的网络流量样本,即自适应滑动窗口以q-1时刻向前序扩张N个预处理后的网络流量样本;Where D W represents a sliding window of length N, which contains the preprocessed network traffic samples from time qN to time q-1; x qN is the preprocessed network traffic sample at the left edge of the window, that is, the adaptive sliding window expands N preprocessed network traffic samples from time q-1 to the forward order; 步骤3.1.2:自适应滑动窗口在确定是否扩张前序样本时,假设滑动窗口已扩张到q-n时刻,待判断是否纳入窗口的样本为q-n-1时刻;首先,依据如下所示的相似性函数计算该时刻样本与当前窗口内部所有样本的欧式距离平均值:Step 3.1.2: When the adaptive sliding window determines whether to expand the previous sample, it is assumed that the sliding window has been expanded to the q-n moment, and the sample to be included in the window is the q-n-1 moment; first, the average Euclidean distance between the sample at that moment and all samples in the current window is calculated according to the similarity function shown below: 其中,ED的计算方式为:The calculation method of ED is: ED(xq-n-1,xw)=||xq-n-1-xw||2ED (x qn-1 , x w )=||x qn-1 -x w || 2 ; 式中,xw表示当前滑动窗口内从q-n时刻到q-1时刻内的任意预处理后的网络流量样本,n为当前窗口内的预处理后的网络流量样本个数,xq-n-1则为待判断是否纳入窗口的前序预处理后的网络流量样本;In the formula, xw represents any preprocessed network traffic sample from time qn to time q-1 in the current sliding window, n is the number of preprocessed network traffic samples in the current window, and xqn-1 is the previous preprocessed network traffic sample to be determined whether to be included in the window; 步骤3.1.3:根据所述步骤3.1.2的相似性函数设定自适应滑动窗口的边界阈值δw,若Sw≥δw,滑动窗口将纳入该预处理后的网络流量样本,即窗口的左边界预处理后的网络流量样本为xq-n-1,反之则停止扩张,此时的左边界样本为xq-nStep 3.1.3: According to the similarity function of step 3.1.2, the boundary threshold δ w of the adaptive sliding window is set. If S w ≥ δ w , the sliding window will include the preprocessed network traffic sample, that is, the left boundary preprocessed network traffic sample of the window is x qn-1 . Otherwise, the expansion is stopped, and the left boundary sample at this time is x qn . 令滑动窗口数据集DW作为xq的目标域数据,则目标域数据集表示为:Let the sliding window dataset D W be the target domain data of x q , then the target domain dataset is expressed as: Xt=DW={xq-N,xt-N+1,...,xq-1};X t = D W = {x qN , x t-N+1 ,..., x q-1 }; 由于DW中共有N条预处理后的网络流量样本,因此目标域数据集Xt表示为:Since there are N preprocessed network traffic samples in D W , the target domain dataset X t is expressed as: 式中,xti表示在包含N条预处理后的网络流量样本的数据集Xt内第i个预处理后的网络流量样本;Where xti represents the i-th preprocessed network traffic sample in the data set Xt containing N preprocessed network traffic samples; 步骤3.2:利用目标域数据集Xt对步骤2训练的异常检测源域模型进行域自适应更新;具体步骤如下:Step 3.2: Use the target domain dataset Xt to perform domain adaptive update on the anomaly detection source domain model trained in step 2; the specific steps are as follows: 步骤3.2.1:首先,将源域数据集Xs输入到训练好的异常检测源域模型中,通过如下公式前向传播获取源域数据的隐含层向量HsStep 3.2.1: First, input the source domain data set Xs into the trained anomaly detection source domain model, and obtain the hidden layer vector Hs of the source domain data through forward propagation using the following formula; Hs=f(WXs+b) Hs =f( WXs +b) 其中,W和b分别表示编码器的网络权重和偏置向量,f为激活函数;Where W and b represent the network weight and bias vector of the encoder respectively, and f is the activation function; 步骤3.2.2:将目标域数据Xt同样输入到异常检测源域模型中通过如下公式前向传播获取目标域数据的隐含层向量HtStep 3.2.2: Input the target domain data Xt into the anomaly detection source domain model and obtain the hidden layer vector Ht of the target domain data through the following formula: Ht=f(WXt+b)H t = f ( WX t + b ) 步骤3.2.3:以最大平均差距离作为目标函数,计算公式如下公式所示:Step 3.2.3: Take the maximum average difference distance as the objective function, and the calculation formula is as follows: 式中,N,M分别为Ht和Hs内的样本个数,sup{·}为求取最小上界函数,i,j在式中指代数据集中的任意索引,即hti和htj分别表示Ht中第i和j个样本,hsi和hsj分别表示Hs中第i和j个样本;G(·)为高斯核函数,G(hsi,hsj)计算方式如下:Where N and M are the number of samples in H t and H s, respectively. sup{·} is the function for finding the minimum upper bound. i and j in the formula refer to any index in the data set, that is, h ti and h tj represent the i-th and j-th samples in H t, respectively. h si and h sj represent the i-th and j-th samples in H s, respectively. G(·) is the Gaussian kernel function. G(h si ,h sj ) is calculated as follows: 式中,σ表示带宽参数;Where, σ represents the bandwidth parameter; 步骤3.2.4:根据步骤3.2.3所示公式计算源域数据与目标域数据生成的隐含向量间的差异,以构建DADAE模型;以最小化DADAE模型目标函数为目标,DADAE模型目标函数如下:Step 3.2.4: Calculate the difference between the implicit vectors generated by the source domain data and the target domain data according to the formula shown in step 3.2.3 to build the DADAE model; the goal is to minimize the DADAE model objective function, which is as follows: 式中,JDAE为上述公式所示的损失函数;λMMD(Hs,Ht)为MMD距离损失函数;网络参数集合分别表示经过域自适应更新后编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,λ为平衡参数;xsi分别表示第i个DAE网络的输入和重构输出变量;M表示预处理后的网络流量样本的总数;Wherein, JDAE is the loss function shown in the above formula; λMMD( Hs , Ht ) is the MMD distance loss function; the network parameter set is Respectively represent the network weights of the encoder, the bias vector of the encoder, the network weights of the decoder, and the bias vector of the decoder after domain adaptive update, λ is the balance parameter; x si and They represent the input and reconstructed output variables of the i-th DAE network respectively; M represents the total number of network traffic samples after preprocessing; 步骤3.2.5:保存训练好的新的异常检测源域模型的网络参数,将该新的异常检测源域模型部署在隧道边缘计算节点。Step 3.2.5: Save the trained network parameters of the new anomaly detection source domain model, and deploy the new anomaly detection source domain model on the tunnel edge computing node. 2.根据权利要求1所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,步骤1中,利用边缘计算节点进行隧道网络异常检测的系统架构包括设备层、边缘计算层、网络层和云平台层;设备层、边缘计算层、网络层和云平台层顺序连接;设备层中的各个设备系统包括广播电话系统、隧道监控系统、隧道通风照明系统、隧道区域控制器、隧道消防系统、信息发布系统和隧道交通信号系统;所述的边缘计算层为隧道内部署的边缘计算节点。2. According to the method for tunnel network anomaly detection based on domain adaptive deep autoencoder according to claim 1, it is characterized in that in step 1, the system architecture of tunnel network anomaly detection using edge computing nodes includes a device layer, an edge computing layer, a network layer and a cloud platform layer; the device layer, the edge computing layer, the network layer and the cloud platform layer are connected sequentially; each device system in the device layer includes a broadcast telephone system, a tunnel monitoring system, a tunnel ventilation and lighting system, a tunnel area controller, a tunnel fire protection system, an information release system and a tunnel traffic signal system; the edge computing layer is an edge computing node deployed in the tunnel. 3.根据权利要求1所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,步骤1中,所述的数据预处理方式包括去除异常数据、去除无意义特征和数据归一化;3. The method for detecting anomalies in a tunnel network based on a domain adaptive deep autoencoder according to claim 1, characterized in that in step 1, the data preprocessing method includes removing abnormal data, removing meaningless features and normalizing data; 网络流量特征包括数据流持续时间、正向包的数量、反向包的数量、正向包的总字节数、反向包的总字节数、正向包头总字节数、反向包头总字节数、正向子流总字节数和反向子流总字节数。The network traffic characteristics include data flow duration, the number of forward packets, the number of reverse packets, the total bytes of forward packets, the total bytes of reverse packets, the total bytes of forward packet headers, the total bytes of reverse packet headers, the total bytes of forward subflows, and the total bytes of reverse subflows. 4.根据权利要求1所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,步骤2中,所述的异常检测源域模型采用深度自动编码器,包括编码器和解码器;4. The method for tunnel network anomaly detection based on domain adaptive deep autoencoder according to claim 1, characterized in that, in step 2, the anomaly detection source domain model adopts a deep autoencoder, including an encoder and a decoder; 所述的异常检测源域模型具有三层神经网络,分别为输入层、隐含层和输出层,输入为Xs,Xs={xs1,xs2,...,xsM};其中,Xs表示源域数据集,xsM表示第M个预处理后的网络流量样本;The anomaly detection source domain model has a three-layer neural network, which includes an input layer, a hidden layer and an output layer. The input is Xs , Xs = { xs1 , xs2 , ..., xsM }; wherein Xs represents the source domain data set, and xsM represents the Mth preprocessed network traffic sample; 异常检测源域模型的具体训练方法如下:The specific training method of the anomaly detection source domain model is as follows: 步骤2.1:编码器将源域数据Xs经过激活函数f映射得到隐含层数据:Step 2.1: The encoder maps the source domain data Xs through the activation function f to obtain the hidden layer data: Hs={hs1,hs2,...,hsM};H s = {h s1 , h s2 ,..., h sM }; 式中,Hs表示隐含层向量,hsM表示第M个预处理后的网络流量样本的隐含层向量;Where Hs represents the hidden layer vector, hsM represents the hidden layer vector of the Mth preprocessed network traffic sample; 编码过程如下公式所示:The encoding process is shown in the following formula: Hs=f(WXs+b) Hs =f( WXs +b) 式中,W和b分别表示编码器的网络权重和偏置向量,f为激活函数,在本发明中为Sigmoid函数;Wherein, W and b represent the network weight and bias vector of the encoder respectively, and f is the activation function, which is the Sigmoid function in the present invention; 步骤2.2:解码器通过激活函数f将隐含层数据Hs转化到输出层获得输出变量:Step 2.2: The decoder transforms the hidden layer data Hs to the output layer through the activation function f to obtain the output variable: 式中,表示重构的输出变量,表示第M个重构的网络流量样本;In the formula, represents the reconstructed output variable, represents the Mth reconstructed network traffic sample; 经隐含层向量Hs重构了输入变量,解码过程如下公式所示; The input variable is reconstructed by the hidden layer vector Hs , and the decoding process is shown in the following formula; 式中,分别表示解码器的网络权重和偏置向量,f为激活函数;In the formula, and They represent the network weight and bias vector of the decoder respectively, and f is the activation function; 步骤2.3:利用梯度下降算法对异常检测源域模型进行训练,以最小化重构的误差为目标,得到最佳网络参数;目标函数如下公式所示:Step 2.3: Use the gradient descent algorithm to train the anomaly detection source domain model, with the goal of minimizing the reconstruction error and obtaining the optimal network parameters; the objective function is shown in the following formula: 式中,网络参数集合分别表示编码器的网络权重、编码器的偏置向量、解码器的网络权重、解码器的偏置向量,xsi分别表示第i个DAE网络的输入和重构输出变量;M表示预处理后的网络流量样本的总数;In the formula, the network parameter set Respectively represent the network weight of the encoder, the bias vector of the encoder, the network weight of the decoder, the bias vector of the decoder, x si and They represent the input and reconstructed output variables of the i-th DAE network respectively; M represents the total number of network traffic samples after preprocessing; 步骤2.4:保存训练好的异常检测源域模型的网络参数,将该模型部署在隧道边缘计算节点。Step 2.4: Save the network parameters of the trained anomaly detection source domain model and deploy the model on the tunnel edge computing node. 5.根据权利要求1所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,步骤4中,计算用于异常检测的动态阈值,动态异常阈值的上限记为Ue,下限记为Le;具体方法为:5. The method for detecting anomalies in a tunnel network based on a domain adaptive deep autoencoder according to claim 1 is characterized in that, in step 4, a dynamic threshold for anomaly detection is calculated, the upper limit of the dynamic anomaly threshold is denoted as U e , and the lower limit is denoted as L e ; the specific method is: 步骤4.1:首先,将所述目标域数据集Xt在更新后的异常检测源域模型中再次执行,得到经过编码器和解码器后的输出数据集,记为接着,利用下式计算每条目标域数据的重构误差:Step 4.1: First, the target domain dataset Xt is re-executed in the updated anomaly detection source domain model to obtain the output dataset after the encoder and decoder, which is recorded as Next, the reconstruction error of each target domain data is calculated using the following formula: 式中,Et包含了N个元素,表示为Xt分别表示包含N个网络流量的目标域数据集及其重构输出数据集;In the formula, Et contains N elements, expressed as Xt and They represent the target domain dataset containing N network flows and its reconstructed output dataset respectively; 步骤4.2:计算Et的平均值和标准差,计算方式如下:Step 4.2: Calculate the mean and standard deviation of E t as follows: 式中,表示Et的平均值,St为Et的标准差;则动态阈值范围为:In the formula, represents the average value of E t , and St is the standard deviation of E t ; then the dynamic threshold range is: 式中,β为标准差系数。In the formula, β is the standard deviation coefficient. 6.根据权利要求5所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,β为2。6. According to the tunnel network anomaly detection method based on domain adaptive deep autoencoder according to claim 5, it is characterized in that β is 2. 7.根据权利要求5所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,步骤5中,重构误差的计算方法如下:7. The method for detecting anomalies in a tunnel network based on a domain adaptive deep autoencoder according to claim 5, characterized in that in step 5, the calculation method of the reconstruction error is as follows: 式中,为所述待测预处理后网络流量样本经过更新后的异常检测源域模型编码后再解码的重构输出。In the formula, The reconstructed output is obtained by encoding and decoding the pre-processed network traffic sample to be tested through the updated anomaly detection source domain model. 8.根据权利要求6所述的基于域自适应深度自编码器的隧道网络异常检测方法,其特征在于,步骤6中的检测方法为:8. The method for detecting anomalies in a tunnel network based on a domain adaptive deep autoencoder according to claim 6, wherein the detection method in step 6 is: 当Le≤eq≤Ue时,标记为正常;当eq>Ue或eq<Le时,标记为异常。When Le ≤e q ≤U e , it is marked as normal; when e q >U e or e q <L e , it is marked as abnormal.
CN202311023612.8A 2023-08-15 2023-08-15 An anomaly detection method based on domain-adaptive deep autoencoder tunnel network Active CN116743646B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311023612.8A CN116743646B (en) 2023-08-15 2023-08-15 An anomaly detection method based on domain-adaptive deep autoencoder tunnel network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311023612.8A CN116743646B (en) 2023-08-15 2023-08-15 An anomaly detection method based on domain-adaptive deep autoencoder tunnel network

Publications (2)

Publication Number Publication Date
CN116743646A CN116743646A (en) 2023-09-12
CN116743646B true CN116743646B (en) 2023-12-19

Family

ID=87904783

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311023612.8A Active CN116743646B (en) 2023-08-15 2023-08-15 An anomaly detection method based on domain-adaptive deep autoencoder tunnel network

Country Status (1)

Country Link
CN (1) CN116743646B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109858509A (en) * 2018-11-05 2019-06-07 杭州电子科技大学 Based on multilayer stochastic neural net single classifier method for detecting abnormality
CN109948117A (en) * 2019-03-13 2019-06-28 南京航空航天大学 A Satellite Anomaly Detection Method for Adversarial Network Autoencoders
CN110992354A (en) * 2019-12-13 2020-04-10 华中科技大学 Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism
CN111585997A (en) * 2020-04-27 2020-08-25 国家计算机网络与信息安全管理中心 Network flow abnormity detection method based on small amount of labeled data
CN112994940A (en) * 2019-05-29 2021-06-18 华为技术有限公司 Network anomaly detection method and device
CN114372530A (en) * 2022-01-11 2022-04-19 北京邮电大学 A method and system for abnormal traffic detection based on deep self-encoding convolutional network
CN114742165A (en) * 2022-04-15 2022-07-12 哈尔滨工业大学 Anomaly detection system of aero-engine gas path performance based on deep autoencoder
CN114783524A (en) * 2022-06-17 2022-07-22 之江实验室 Path Anomaly Detection System Based on Adaptive Resampling Deep Encoder Network
CN115169430A (en) * 2022-04-27 2022-10-11 北京理工大学 Cloud network end resource multidimensional time sequence anomaly detection method based on multi-scale decoding
CN115242556A (en) * 2022-09-22 2022-10-25 中国人民解放军战略支援部队航天工程大学 Network anomaly detection method based on incremental self-encoder
KR102510060B1 (en) * 2022-07-28 2023-03-14 주식회사 어니언소프트웨어 An obtaining method abnormality data through deep learning pump simulation and an abnormality detection model establishment method based on auto-encoder and a system thereof
CN116055413A (en) * 2023-03-07 2023-05-02 云南省交通规划设计研究院有限公司 Tunnel network anomaly identification method based on cloud edge cooperation
CN116385935A (en) * 2023-04-08 2023-07-04 苏州海裕鸿智能科技有限公司 Abnormal event detection algorithm based on unsupervised domain self-adaption

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3885989A1 (en) * 2020-03-26 2021-09-29 Another Brain Anomaly detection based on an autoencoder and clustering
US20230082899A1 (en) * 2021-09-14 2023-03-16 Eduardo CORRAL-SOTO Devices, systems, methods, and media for domain adaptation using hybrid learning

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109858509A (en) * 2018-11-05 2019-06-07 杭州电子科技大学 Based on multilayer stochastic neural net single classifier method for detecting abnormality
CN109948117A (en) * 2019-03-13 2019-06-28 南京航空航天大学 A Satellite Anomaly Detection Method for Adversarial Network Autoencoders
CN112994940A (en) * 2019-05-29 2021-06-18 华为技术有限公司 Network anomaly detection method and device
CN110992354A (en) * 2019-12-13 2020-04-10 华中科技大学 Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism
CN111585997A (en) * 2020-04-27 2020-08-25 国家计算机网络与信息安全管理中心 Network flow abnormity detection method based on small amount of labeled data
CN114372530A (en) * 2022-01-11 2022-04-19 北京邮电大学 A method and system for abnormal traffic detection based on deep self-encoding convolutional network
CN114742165A (en) * 2022-04-15 2022-07-12 哈尔滨工业大学 Anomaly detection system of aero-engine gas path performance based on deep autoencoder
CN115169430A (en) * 2022-04-27 2022-10-11 北京理工大学 Cloud network end resource multidimensional time sequence anomaly detection method based on multi-scale decoding
CN114783524A (en) * 2022-06-17 2022-07-22 之江实验室 Path Anomaly Detection System Based on Adaptive Resampling Deep Encoder Network
KR102510060B1 (en) * 2022-07-28 2023-03-14 주식회사 어니언소프트웨어 An obtaining method abnormality data through deep learning pump simulation and an abnormality detection model establishment method based on auto-encoder and a system thereof
CN115242556A (en) * 2022-09-22 2022-10-25 中国人民解放军战略支援部队航天工程大学 Network anomaly detection method based on incremental self-encoder
CN116055413A (en) * 2023-03-07 2023-05-02 云南省交通规划设计研究院有限公司 Tunnel network anomaly identification method based on cloud edge cooperation
CN116385935A (en) * 2023-04-08 2023-07-04 苏州海裕鸿智能科技有限公司 Abnormal event detection algorithm based on unsupervised domain self-adaption

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Anomaly Detection Using LSTM-Based Variational Autoencoder in Unsupervised Data in Power Grid;Dibyajyoti Guha等;《IEEE Systems Journal》;第17卷(第03期);4313-4323 *
基于生成对抗网络与自编码器的网络流量异常检测模型;郭森森等;《信息网络安全》(第12期);7-15 *
基于自编码器的异常检测算法研究;蔚焘;《中国优秀硕士学位论文全文数据库》;I140-985 *
结合二次特征提取和LSTM-Autoencoder的网络流量异常检测方法;孙旭日等;《北京交通大学学报》(第02期);21-30 *

Also Published As

Publication number Publication date
CN116743646A (en) 2023-09-12

Similar Documents

Publication Publication Date Title
CN113158543B (en) Intelligent prediction method for software defined network performance
CN114422381B (en) Communication network traffic prediction method, system, storage medium and computer equipment
CN113271225B (en) Network reliability evaluation method based on in-band network telemetry technology
CN105825271B (en) Satellite failure diagnosis and prediction method based on evidential reasoning
CN109547431A (en) A kind of network security situation evaluating method based on CS and improved BP
WO2015158198A1 (en) Fault recognition method and system based on neural network self-learning
CN118133203A (en) A method for fault diagnosis of electric energy metering detection information
CN102496284A (en) Method for acquiring and predicting road traffic flow
CN106961349A (en) A kind of sensor fault identifying system and method based on data fusion
CN102868224B (en) Secondary network measurement and multimode decision-making method and device for intelligent substation
CN105574604B (en) Monitoring pre-judging analysis system for power grid operation event
CN111884874B (en) Programmable data plane-based ship network real-time anomaly detection method
CN113033772A (en) Multi-equipment state monitoring method based on federal learning
CN117110748A (en) A method for detecting abnormal operating status of substation main equipment based on converged terminals
CN119046883A (en) Control signal acquisition and transmission system for host
CN118659978A (en) Artificial intelligence-based low-altitude communication network organization method
CN101751018B (en) Distributing data monitoring and prealarming system under test environment and method therefor
CN116743646B (en) An anomaly detection method based on domain-adaptive deep autoencoder tunnel network
CN113642403A (en) Crowd abnormal intelligent safety detection system based on edge calculation
CN112801815A (en) Power communication network fault early warning method based on federal learning
CN118569081A (en) Dynamic evolution and credibility assessment method for equipment digital twin data drive
CN116094758B (en) Large-scale network flow acquisition method and system
CN114745253B (en) Traffic early warning method based on transducer and federal learning
CN117879907A (en) A network environment anomaly detection method based on graph convolution behavior feature extraction
CN104168599B (en) Wireless sensor network fault diagnosis method based on time weight K- nearest neighbour methods

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 650000 Yunnan province Kunming City Road No. 9 Xiang Shi Tuo

Applicant after: Yunnan Provincial Transportation Planning and Design Research Institute Co.,Ltd.

Address before: 650041 No. 9 Shijiaxiang, Tuodong Road, Kunming City, Yunnan Province

Applicant before: BROADVISION ENGINEERING CONSULTANTS

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20241230

Address after: 650000 Yunnan province Kunming City Road No. 9 Xiang Shi Tuo

Patentee after: Yunnan Provincial Transportation Planning and Design Research Institute Co.,Ltd.

Country or region after: China

Patentee after: YUNNAN COMMUNICATIONS INVESTMENT & CONSTRUCTION GROUP CO.,LTD.

Address before: 650000 Yunnan province Kunming City Road No. 9 Xiang Shi Tuo

Patentee before: Yunnan Provincial Transportation Planning and Design Research Institute Co.,Ltd.

Country or region before: China

TR01 Transfer of patent right