WO2022083353A1 - 异常网络数据检测方法、装置、计算机设备和存储介质 - Google Patents

异常网络数据检测方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2022083353A1
WO2022083353A1 PCT/CN2021/117975 CN2021117975W WO2022083353A1 WO 2022083353 A1 WO2022083353 A1 WO 2022083353A1 CN 2021117975 W CN2021117975 W CN 2021117975W WO 2022083353 A1 WO2022083353 A1 WO 2022083353A1
Authority
WO
WIPO (PCT)
Prior art keywords
data packet
fingerprint
target
confidence
network data
Prior art date
Application number
PCT/CN2021/117975
Other languages
English (en)
French (fr)
Inventor
彭婧
杨勇
甘祥
郑兴
许艾斯
华珊珊
郭晶
常优
范宇河
唐文韬
申军利
何澍
王悦
刘羽
Original Assignee
腾讯科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技(深圳)有限公司 filed Critical 腾讯科技(深圳)有限公司
Publication of WO2022083353A1 publication Critical patent/WO2022083353A1/zh
Priority to US17/994,944 priority Critical patent/US20230089187A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1416Event detection, e.g. attack signature detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/02Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227Filtering policies
    • H04L63/0236Filtering by address, protocol, port number or service, e.g. IP-address or URL
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425Traffic logging, e.g. anomaly detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/20Network architectures or network communication protocols for network security for managing network security; network security policies in general

Definitions

  • the present application relates to the field of computer technology, and in particular, to a method, apparatus, computer equipment and storage medium for detecting abnormal network data.
  • the detection of network attack events is mainly by collecting public blacklist IPs. If it is detected that the IP of the current network data packet is a public blacklist IP, it is determined that the network data packet is an abnormal network data packet, and the occurrence of cyber attack incident.
  • the number of IPs on the network is very large, and the public blacklisted IPs only include some IPs used by hackers, which is prone to missed judgments, resulting in low detection accuracy of abnormal network data packets and low detection efficiency of abnormal network data packets.
  • a method, apparatus, computer device, and storage medium for detecting abnormal network data are provided.
  • a method for detecting abnormal network data comprising:
  • the target data packet fingerprint set includes at least one target data packet fingerprint
  • the reference confidence level is obtained, and the abnormality detection result of the network data packet to be detected is determined based on the reference confidence level and the target confidence level.
  • a device for detecting abnormal network data comprising:
  • the data packet acquisition module is used to acquire the network data packets to be detected
  • the data packet fingerprint generation module is used for extracting features of the network data packet to be detected according to the target protocol type corresponding to the network data packet to be detected, and generating a corresponding target data packet fingerprint set; the target data packet fingerprint set includes at least one target data packet fingerprint;
  • the data packet fingerprint matching module is used to match the target data packet fingerprint with the reference data packet fingerprint in the reference data packet fingerprint database
  • a confidence level calculation module configured to calculate the target confidence level corresponding to the network data packet to be detected based on the confidence level correlation information corresponding to the fingerprint of the successfully matched target data packet;
  • the detection result determination module is used to obtain the reference confidence, and determine the abnormal detection result of the network data packet to be detected based on the reference confidence and the target confidence.
  • a computer device comprising a memory and one or more processors, the memory storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors
  • the processor executes the steps of the above-mentioned abnormal network data detection method.
  • One or more non-volatile computer-readable storage media having computer-readable instructions stored thereon that, when executed by one or more processors, cause the one or multiple processors execute the steps of the above-mentioned abnormal network data detection method.
  • a computer program product or computer program comprising computer readable instructions stored in a computer readable storage medium from which a processor of a computer device readable storage The medium reads the computer-readable instructions, and the processor executes the computer-readable instructions, so that the computer device performs the steps of the abnormal network data detection method.
  • 1 is an application environment diagram of a method for detecting abnormal network data in one embodiment
  • FIG. 2 is a schematic flowchart of a method for detecting abnormal network data in one embodiment
  • 3 is a schematic diagram of classifying data packet fingerprints in one embodiment
  • FIG. 4 is a schematic flowchart of establishing a reference data packet fingerprint library in one embodiment
  • FIG. 5 is a schematic flowchart of calculating target confidence in one embodiment
  • FIG. 6 is a schematic flowchart of calculating target confidence in another embodiment
  • FIG. 7 is a schematic flowchart of determining an abnormality detection result of a network data packet to be detected in one embodiment
  • FIG. 8 is an application environment diagram of a method for detecting abnormal network data in another embodiment
  • FIG. 9 is a schematic flowchart of a method for detecting abnormal network data in another embodiment
  • FIG. 10 is a structural block diagram of an apparatus for detecting abnormal network data in one embodiment
  • FIG. 11 is a structural block diagram of a device for detecting abnormal network data in another embodiment
  • Figure 12 is a diagram of the internal structure of a computer device in one embodiment.
  • Cloud security refers to the general term for security software, hardware, users, institutions, and security cloud platforms based on cloud computing business model applications. Cloud security integrates emerging technologies and concepts such as parallel processing, grid computing, and unknown virus behavior judgment. Through the abnormal monitoring of software behavior in the network by a large number of mesh clients, the latest information on Trojans and malicious programs in the Internet is obtained and sent. To the server for automatic analysis and processing, and then distribute the virus and Trojan solutions to each client.
  • Cloud computing security which mainly studies how to ensure the security of the cloud itself and various applications on the cloud, including cloud computer system security, secure storage and isolation of user data, user access authentication, and information transmission security , network attack protection, compliance audit, etc.
  • Cloudization of security infrastructure mainly to study how to use cloud computing to build and integrate security infrastructure resources, and optimize security protection mechanisms, including the construction of ultra-large-scale security events and information through cloud computing technology
  • the collection and processing platform realizes the collection and correlation analysis of massive information, and improves the ability to control security events and risk control of the entire network
  • Cloud security services mainly research various security services provided to users based on cloud computing platforms, such as Antivirus service, etc.
  • the abnormal network data detection method provided in this application can be applied to the application environment shown in FIG. 1 .
  • the terminal 102 communicates with the service server 104 through the network, the terminal 102 communicates with the detection server 106 through the network, and the service server 104 communicates with the detection server 106 through the network.
  • the terminal 102 and the service server 104 may communicate by sending network data packets.
  • the detection server 106 may obtain the network data packets to be detected, extract features of the network data packets to be detected according to the target protocol type corresponding to the network data packets to be detected, and generate a corresponding target data packet fingerprint set, where the target data packet fingerprint set includes at least one target data packet. fingerprint.
  • the detection server 106 can match the target data packet fingerprint with the reference data packet fingerprint in the reference data packet fingerprint database, and calculate the target confidence corresponding to the network data packet to be detected based on the confidence level associated information corresponding to the successfully matched target data packet fingerprint.
  • the reference confidence level is obtained, and the abnormal detection result of the network data packet to be detected is determined based on the reference confidence level and the target confidence level.
  • the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, and cloud communications. , middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms.
  • the terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. There may be more than one terminal, and the terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited in this application.
  • a method for detecting abnormal network data is provided, and the method is applied to the detection server 106 in FIG. 1 as an example for description, including the following steps:
  • Step S202 acquiring network data packets to be detected.
  • the network data packet to be detected refers to the network data packet to be detected whether it is abnormal.
  • the network data packets to be detected may be network data packets obtained in real time, that is, real-time traffic. Network packets are used to transmit data between the terminal and the service server.
  • the network data packet may specifically include information such as a destination IP address, a source IP address, and payload data.
  • the destination IP address refers to the address of the receiver corresponding to the network data packet
  • the source IP address refers to the address of the sender corresponding to the network data packet
  • the payload data refers to the specific request content or response content.
  • the terminal can send to the service server.
  • the service server may return the network data packet corresponding to the current news webpage to the terminal.
  • the requested content includes at least one of text, pictures, and videos.
  • the terminal and the service server can communicate by sending network data packets.
  • the detection server may acquire the current network data packet sent by the terminal or the service server as the network data packet to be detected.
  • network data packets can be transmitted between the terminal and the service server by means of a switch.
  • the detection server can listen for network packets from the switch in a bypass mode.
  • Step S204 extract features of the network data packets to be detected according to the target protocol type corresponding to the network data packets to be detected, and generate a corresponding target data packet fingerprint set; the target data packet fingerprint set includes at least one target data packet fingerprint.
  • the target protocol type refers to the protocol type corresponding to the network data packet to be detected.
  • Protocol type refers to the type of network protocol that transmits network packets.
  • the protocol type includes at least one of the current layer protocol type and the association layer protocol type.
  • the current layer protocol type refers to the application layer protocol type corresponding to the network data packet
  • the association layer protocol type refers to the other layer protocol type corresponding to the network data packet, which may specifically include the transport layer protocol type corresponding to the network data packet, the network layer protocol type, At least one of the data link layer protocol types.
  • the network data packet is an HTTP protocol (Hyper Text Transfer Protocol) network data packet
  • the current layer protocol type corresponding to the network data packet to be detected can be the HTTP protocol
  • the transport layer protocol type corresponding to the HTTP protocol is TCP protocol (Transmission Control Protocol, Transmission Control Protocol)
  • the association layer protocol type corresponding to the network data packet can be the TCP protocol.
  • the target data packet fingerprint refers to the data packet fingerprint corresponding to the network data packet to be detected.
  • the data packet fingerprint is to extract the corresponding feature information from the network data packet according to the network protocol type, and generate an identifier according to the extracted feature information, which is used to identify the network data packet.
  • a network packet may correspond to at least one packet fingerprint.
  • One protocol type can generate at least one data packet fingerprint.
  • the protocol types corresponding to network data packets include HTTP protocol and TCP protocol.
  • At least one data packet fingerprint can be generated based on HTTP protocol, and at least one data packet fingerprint can be generated based on TCP protocol.
  • Different data packet fingerprints corresponding to a network data packet can represent the characteristic information of the network data packet from different dimensions.
  • Different network data packets correspond to different data packet fingerprint sets, but different data packet fingerprint sets may include at least one different data packet fingerprint, for example, network data packet A and network data packet B sent from the same terminal, due to network data
  • the senders corresponding to the packet A and the network packet B are the same.
  • the network packet A and the network packet B may have the same packet fingerprint, and the same packet fingerprint may be generated according to the relevant information of the sender in the feature information.
  • the detection server may obtain the target protocol type corresponding to the network data packet to be detected from the network data packet to be detected, extract feature information from the network data packet to be detected according to the target protocol type, and generate corresponding at least one corresponding feature information according to the extracted feature information.
  • Target data packet fingerprint each target data packet fingerprint constitutes a target data packet fingerprint set.
  • the detection server can generate a corresponding target data packet fingerprint set according to the current layer protocol type corresponding to the to-be-detected network data packet to-be-detected network data packet extraction feature, or can also use the to-be-detected network data packet to be detected according to the association layer protocol type corresponding to the to-be-detected network data packet Extract features to generate corresponding target data packet fingerprint sets, and can also extract features to generate corresponding target data packet fingerprint sets according to the current layer protocol type and association layer protocol type corresponding to the to-be-detected network data packets.
  • the detection server may obtain the current layer protocol type corresponding to the network data packet to be detected from the network data packet to be detected, and obtain the fingerprint generation algorithm corresponding to the current layer protocol type locally or from other terminals or servers, based on the fingerprint
  • the generation algorithm extracts the feature field matching the fingerprint generation algorithm from the network data packet to be detected, and obtains the target data packet fingerprint generated by the fingerprint generation algorithm corresponding to the current layer protocol type according to the extracted feature field.
  • the fingerprint generation algorithm corresponding to the current layer protocol type may be at least one. If there are multiple fingerprint generation algorithms corresponding to the protocol type of the current layer, multiple target data packet fingerprints corresponding to the protocol type of the current layer can be generated.
  • the detection server can obtain the association layer protocol type corresponding to the network data packet to be detected from the network data packet to be detected, and obtain the fingerprint generation algorithm corresponding to the association layer protocol type locally or from other terminals and servers.
  • a feature field matching the fingerprint generation algorithm is extracted from the network data packet to be detected, and a target data packet fingerprint generated by the fingerprint generation algorithm corresponding to the association layer protocol type is obtained according to the extracted feature field.
  • the fingerprint generation algorithm corresponding to the association layer protocol type may be at least one. When there are multiple fingerprint generation algorithms corresponding to the association layer protocol type, multiple target data packet fingerprints corresponding to the association layer protocol type can be generated.
  • It can be a combination of target data packet fingerprints corresponding to the current layer protocol type to obtain a target data packet fingerprint set, or a combination of target data packet fingerprints corresponding to an association layer protocol type to obtain a target data packet fingerprint set, or the current layer protocol type.
  • the corresponding target data packet fingerprints are combined with each target data packet fingerprint corresponding to the association layer protocol type to obtain a target data packet fingerprint set.
  • the network data packet includes multiple fields, and different fields may represent different characteristic information of the network data packet.
  • the fields of the network data packet include destination IP address, source IP address, protocol type, sending time, packet size, payload data, and the like.
  • Generating the data packet fingerprint according to the extracted characteristic fields may specifically be to arrange and combine the characteristic fields according to the field order specified by the fingerprint generation algorithm to obtain target characteristic information, and calculate the hash value of the target characteristic information to obtain the data packet fingerprint.
  • the field positions of fields of the same type in the network data packets corresponding to different protocol types may be the same or different, and the target feature information may further include the field positions of each feature field in the network data packet.
  • each protocol may correspond to multiple fingerprint generation algorithms, respectively, and then the data packet fingerprints may be classified according to the protocol type and the fingerprint generation algorithm.
  • the network protocols include HTTP protocol, DHCP protocol (Dynamic Host Configuration Protocol, Dynamic Host Configuration Protocol), SSH protocol (Secure Shell, Secure Shell Protocol), SSL protocol (Secure Sockets Layer, Secure Socket Protocol) , TLS protocol (Transport Layer Security, security transport layer protocol), RDP protocol (Remote Display Protocol, remote display protocol), TCP protocol and UDP protocol (User Datagram Protocol, user datagram protocol).
  • the DHCP protocol is transmitted based on the UDP protocol
  • the SSH protocol is transmitted based on the TCP protocol
  • the SSL protocol/TLS protocol is transmitted based on the TCP protocol
  • the RDP protocol is transmitted based on the TCP protocol.
  • the fingerprint generation algorithm corresponding to the HTTP protocol may include fatt, pmercury, and the like.
  • fatt(Fingerprint All The Things) is a pyshark based script for extracting packet fingerprints from pcap(Packet Capture Data) files and network packets.
  • pmercury is a python implementation of mercury, a Linux application for extracting packet fingerprints from network packets.
  • the fingerprint generation algorithm corresponding to the DHCP protocol may include pmercury and the like.
  • the fingerprint generation algorithm corresponding to the SSH protocol may include HASSH-SSH_MSG_KEXINIT, pmercury, and the like.
  • HASSH is an open source algorithm
  • SSH_MSG_KEXINIT is the key exchange packet header
  • HASSH-SSH_MSG_KEXINIT is a method to identify the fingerprint of the SSH client.
  • the fingerprint generation algorithms corresponding to the SSL protocol and the TLS protocol may include JA3, pmercury, and the like. JA3 is a way to identify TLS client fingerprints online.
  • the fingerprint generation algorithm corresponding to the RDP protocol may include fatt and the like.
  • the fingerprint generation algorithms corresponding to the TCP protocol and the UDP protocol may include pmercury and the like.
  • a fingerprint generation algorithm corresponding to a protocol is used as a category, so that when calculating the packet fingerprint, a network packet can correspond to multiple protocols, or it can correspond to multiple fingerprint generation algorithms.
  • the data packet fingerprint corresponding to a network data packet of the HTTP protocol may include the data packet fingerprint generated by the fingerprint generation algorithm pmercury corresponding to the TCP protocol, and the fingerprint generated by the fingerprint corresponding to the HTTP protocol.
  • the data packet fingerprint generated by the algorithm pmercury and the data packet fingerprint generated by the fingerprint generation algorithm fatt corresponding to the HTTP protocol.
  • the feature fields extracted from network packets by different fingerprint generation algorithms can be the same or different.
  • the data packet fingerprints generated by the same fingerprint generation algorithm corresponding to different protocols are different, because for different protocols, the same fingerprint generation algorithm extracts different feature fields from the network data packet.
  • the fingerprint generation algorithm pmercury extracts different feature fields from network packets.
  • Step S206 matching the target data packet fingerprint with the reference data packet fingerprint in the reference data packet fingerprint database.
  • the reference data packet fingerprint database includes a plurality of reference data packet fingerprints.
  • the reference packet fingerprint database is generated after clustering analysis of the packet fingerprints corresponding to multiple candidate network packets.
  • Each candidate network data packet refers to a network data packet within a set historical time period, for example, a network data packet collected within 5 minutes before the current moment is obtained as a candidate network data packet.
  • the length of the historical time period may be determined according to the actual situation, which is not limited in this embodiment of the present invention.
  • the attack behavior of hackers is a continuous and common attack behavior. For example, hackers mainly attack from the same terminal, hackers mainly attack from the same browser, and hackers mainly attack from the same account.
  • the data packet fingerprint can be considered as an abnormal data packet fingerprint, and the data packet fingerprint is added to the reference data packet fingerprint database as a reference data packet fingerprint.
  • the detection server can match the target data packet fingerprint with the reference data packet fingerprint in the reference data packet fingerprint database, and further determine the network to be detected according to the matching result. Anomaly detection result of the packet.
  • the fingerprint association information includes at least one of a protocol type corresponding to the data packet fingerprint and a fingerprint generation algorithm.
  • a network data packet to be detected can correspond to multiple target data packet fingerprints, each target data packet fingerprint can be matched with each reference data packet fingerprint in the reference data packet fingerprint database, and each target data packet fingerprint can be successfully matched. , or all matches fail, or at least one match succeeds.
  • the reference data packet fingerprint library may be generated by the detection server, or may be generated by other terminals or servers and then sent to the detection server.
  • Step S208 Calculate the target confidence level corresponding to the network data packet to be detected based on the confidence level correlation information corresponding to the fingerprint of the successfully matched target data packet.
  • the confidence level correlation information refers to the correlation information used to calculate the confidence level corresponding to the network data packet.
  • the confidence level associated information of the data packet fingerprint includes at least one of the protocol confidence level of the protocol type corresponding to the data packet fingerprint, and the algorithm confidence level of the fingerprint generation algorithm corresponding to the data packet fingerprint under the protocol type. For example, if the successfully matched target data packet fingerprint is the data packet fingerprint generated by the fingerprint generation algorithm 1 corresponding to the HTTP protocol, then the confidence associated information corresponding to the target data packet fingerprint includes the protocol confidence corresponding to the HTTP protocol and the information under the HTTP protocol.
  • the protocol confidence and algorithm confidence can be determined according to the actual situation, for example, manually set according to security experience, and calculated according to a custom formula.
  • the algorithm confidence levels corresponding to the same fingerprint generation algorithm under different protocols may be the same or different.
  • the target confidence level is used to determine the abnormality degree of the network data packet. The greater the target confidence degree, the greater the abnormality degree of the network data packet
  • the detection server can filter out the successfully matched target data packet fingerprints according to the matching result, obtain the confidence level correlation information corresponding to the successfully matched target data packet fingerprints, and calculate and obtain the target corresponding to the network data packet to be detected according to the confidence level correlation information Confidence.
  • the detection server may perform weighted summation of the protocol confidence and algorithm confidence corresponding to each successfully matched target data packet fingerprint to obtain the target confidence.
  • the detection server may also weight and multiply each protocol confidence degree and each algorithm confidence degree to obtain the target confidence degree.
  • the intermediate confidence levels corresponding to each protocol type can be calculated separately, and the target confidence degree can be obtained by weighting and summing up each intermediate confidence degree.
  • the confidence level of each algorithm corresponding to the same protocol type can be weighted and summed to obtain the algorithm confidence level statistical value, and the algorithm confidence level statistical value and the corresponding protocol confidence level are weighted and multiplied. Get the corresponding intermediate confidence.
  • the intermediate confidence level corresponding to each protocol type may also be used as the target confidence level corresponding to the network data packet to be detected.
  • step S210 a reference confidence level is obtained, and an abnormality detection result of the network data packet to be detected is determined based on the reference confidence level and the target confidence level.
  • the reference confidence may be determined according to the actual situation, for example, manually set according to security experience, and calculated according to a custom formula.
  • the detection server may obtain the reference confidence level, compare the reference confidence level with the target confidence level, and determine the abnormal detection result of the network data packet to be detected according to the comparison result.
  • Anomaly detection results include abnormal network packets and normal network packets.
  • the detection server can send alarm information to the operation and maintenance personnel, so that the operation and maintenance personnel can perform security maintenance in a timely manner.
  • the detection server can also directly block the network data packet to be detected to prevent hacker attacks.
  • all network data packets to be detected may use the same reference confidence level, that is, there is only one reference confidence level.
  • a protocol type corresponds to a reference confidence level.
  • the protocol types corresponding to the successfully matched target data packet fingerprints include http protocol and tcp protocol, http protocol corresponds to reference confidence level 1, and tcp protocol corresponds to reference confidence level 2, then when The confidence statistical value calculated based on the confidence correlation information of the target data packet fingerprint corresponding to the http protocol is greater than the reference confidence 1, and the confidence statistical value calculated based on the confidence correlation information of the target data packet fingerprint corresponding to the tcp protocol is greater than With reference to the confidence level of 2, it is determined that the abnormality detection result of the network data packet to be detected is the abnormality of the network data packet.
  • the characteristics of the network data packets to be detected are extracted according to the target protocol type corresponding to the network data packets to be detected, and the corresponding target data packet fingerprint set is generated, and the target data packet fingerprint set includes: At least one destination packet fingerprint.
  • the generated target data packet fingerprint can represent the characteristic information of the network data packet to be detected
  • multiple target data packet fingerprints can represent the characteristic information of the network data packet to be detected from different dimensions, thereby enriching the detection dimension of abnormal network data packets.
  • the detection accuracy of abnormal network packets has been improved.
  • the reference data packet fingerprint database integrates multiple abnormal reference data packet fingerprints, so the successfully matched target data packet fingerprints can represent the characteristic information of abnormal network data packets to be detected, and the corresponding confidence level based on the successfully matched target data packet fingerprints
  • the target confidence calculated by the correlation information can represent the abnormality of the network data packet to be detected. Based on the reference confidence and target confidence, the abnormal detection result of the network data packet to be detected can be quickly determined, which improves the detection accuracy of abnormal network data packets. and detection efficiency.
  • the method before acquiring the network data packet to be detected, the method further includes:
  • Step S402 Obtain a candidate network data packet set; the candidate network data packet set includes multiple candidate network data packets within the same time window.
  • Step S404 extracting features from the candidate network data packets according to the candidate protocol types corresponding to the candidate network data packets, and generating a corresponding candidate data packet fingerprint set, where the candidate data packet fingerprint set includes the candidate data packet fingerprints corresponding to each candidate network data packet.
  • the detection server may acquire multiple candidate network data packets within a time window to form a candidate network data packet set.
  • the detection server can extract features from the candidate network data packets according to the candidate protocol types corresponding to the candidate network data packets, and generate at least one candidate data packet fingerprint corresponding to each candidate network data packet, and the candidate data packet fingerprints corresponding to each candidate network data packet form a candidate network data packet.
  • the length of the time window can be set as required. For example, it is set to 5 minutes.
  • the detection server obtains each network data packet collected within 5 minutes before the current moment as candidate network data packets, and forms each candidate network data packet into A collection of candidate network packets.
  • the detection server can obtain the current layer protocol type corresponding to the candidate network data packet from the candidate network data packet, obtain the fingerprint generation algorithm corresponding to the current layer protocol type locally or from other terminals or servers, based on the fingerprint generation algorithm
  • the feature fields matching the fingerprint generation algorithm are extracted from the candidate network data packets, and the candidate data packet fingerprints generated by the fingerprint generation algorithm corresponding to the current layer protocol type are obtained according to the extracted feature fields.
  • the fingerprint generation algorithm corresponding to the current layer protocol type may be at least one. If there are multiple fingerprint generation algorithms corresponding to the protocol type of the current layer, multiple candidate data packet fingerprints corresponding to the protocol type of the current layer can be generated.
  • the detection server can obtain the association layer protocol type corresponding to the candidate network data packet from the candidate network data packet, obtain the fingerprint generation algorithm corresponding to the association layer protocol type locally or from other terminals or servers, and obtain the fingerprint generation algorithm from the candidate network based on the fingerprint generation algorithm.
  • Feature fields matching the fingerprint generation algorithm are extracted from the data packets, and candidate data packet fingerprints generated by the fingerprint generation algorithm corresponding to the protocol type of the association layer are obtained according to the extracted feature fields.
  • the fingerprint generation algorithm corresponding to the association layer protocol type may be at least one. When there are multiple fingerprint generation algorithms corresponding to the association layer protocol type, multiple candidate data packet fingerprints corresponding to the association layer protocol type can be generated.
  • each candidate data packet fingerprint corresponding to the current layer protocol type of each candidate network data packet can be the combination of each candidate data packet fingerprint corresponding to the current layer protocol type of each candidate network data packet to obtain the candidate data packet fingerprint set, or the combination of each candidate data packet fingerprint corresponding to the association layer protocol type of each candidate network data packet to obtain the candidate data packet fingerprint set.
  • the data packet fingerprint set may also be a combination of each candidate data packet fingerprint corresponding to the current layer protocol type of each candidate network data packet and each candidate data packet fingerprint corresponding to the associated layer protocol type to obtain the candidate data packet fingerprint set.
  • Step S406 Cluster the candidate data packet fingerprints in the candidate data packet fingerprint set based on the fingerprint association information to obtain a clustering result.
  • the fingerprint association information refers to association information of data packet fingerprints.
  • the fingerprint associated information of the data packet fingerprint includes the protocol type and the fingerprint generation algorithm used to generate the data packet fingerprint.
  • the detection server may cluster the candidate data packet fingerprints in the candidate data packet fingerprint set based on the fingerprint association information, and may specifically cluster the same candidate data packet fingerprints generated based on the same fingerprint association information together to obtain multiple different clusters.
  • clustering the candidate data packet fingerprints in the candidate data packet fingerprint set based on fingerprint association information to obtain a clustering result includes: clustering the same candidate data packets generated based on the same fingerprint generation algorithm corresponding to the same protocol type The fingerprints are clustered to obtain a plurality of different clusters, the number of candidate data packet fingerprints in the same cluster is counted, and the statistical value corresponding to each cluster is obtained.
  • the detection server may cluster the fingerprints of the same candidate data packets generated based on the same fingerprint generation algorithm corresponding to the same protocol type to obtain a plurality of different clusters.
  • a cluster represents a class of candidate packet fingerprints.
  • the detection server can count the number of candidate data packet fingerprints in the same cluster, and obtain the statistical value corresponding to each cluster. That is, the detection server can count the number of fingerprints of the same candidate data packets generated under the same protocol type and the same fingerprint generation algorithm within a time window, and obtain multiple statistical values.
  • count the number of candidate packet fingerprints a generated by the fingerprint generation algorithm 1 corresponding to the HTTP protocol count the number of candidate packet fingerprints b generated by the fingerprint generation algorithm 2 corresponding to the HTTP protocol, and count the fingerprints generated by the SSH protocol.
  • the number of candidate data packet fingerprints c generated by algorithm 3 is counted by the number of candidate data packet fingerprints d generated by fingerprint generation algorithm 1 corresponding to the SSH protocol.
  • Step S408 Determine a reference data packet fingerprint from the candidate data packet fingerprint set according to the clustering result, and combine the reference data packet fingerprints to obtain a reference data packet fingerprint database.
  • the detection server can determine the reference data packet fingerprint from the set of candidate data packet fingerprints according to the clustering result. Specifically, when the statistical value corresponding to the cluster is greater than the statistical threshold, the candidate data packet fingerprint corresponding to the cluster is selected. As a reference data packet fingerprint, each reference data packet fingerprint is combined to obtain a reference data packet fingerprint database.
  • the reference data packet fingerprint database may further include fingerprint correlation information and statistical values corresponding to each reference data packet fingerprint.
  • a reference data packet fingerprint is determined from a set of candidate data packet fingerprints according to a clustering result, and each reference data packet fingerprint is combined to obtain a reference data packet fingerprint database, including: The candidate packet fingerprint is used as the reference packet fingerprint.
  • the detection server can The candidate packet fingerprints in the cluster are used as reference packet fingerprints.
  • the statistical threshold can be set according to the size of the time window. The larger the time window, the larger the statistical threshold.
  • the time window can be dynamically changed, and the reference data packet fingerprint database is also dynamically updated. For example, if the time window is always kept 5 minutes before the current time, the detection server can cluster the candidate data packet fingerprints corresponding to each candidate network data packet obtained within 5 minutes every 5 minutes, and classify the fingerprints according to the clustering results. Refer to the Packet Fingerprint Library for updates.
  • the reference packet fingerprint library includes reference packet fingerprint 1 generated based on the fingerprint generation algorithm fatt corresponding to the HTTP protocol, reference packet fingerprint 2 generated based on the fingerprint generation algorithm fatt corresponding to the HTTP protocol, and fingerprint generation algorithm based on the HTTP protocol.
  • matching the target data packet fingerprint with the reference data packet fingerprint in the reference data packet fingerprint database includes: comparing the target data packet fingerprint with the current reference data packet fingerprint to obtain a data packet fingerprint comparison result; The fingerprint correlation information corresponding to the fingerprint of the target data packet is compared with the fingerprint correlation information corresponding to the fingerprint of the current reference data packet, and the comparison result of the fingerprint correlation information is obtained; when the data packet fingerprint comparison result and the fingerprint correlation information comparison result are consistent, the target data is determined. The packet fingerprint matches the current reference packet fingerprint successfully.
  • the detection server may compare the target data packet fingerprint with the current reference data packet fingerprint to obtain a data packet fingerprint comparison result.
  • the packet fingerprint comparison result includes consistency and inconsistency.
  • the fingerprint association information corresponding to the target data packet fingerprint and the fingerprint association information corresponding to the current reference data packet fingerprint are compared to obtain the fingerprint association information comparison result.
  • the fingerprint association information comparison result includes consistency and inconsistency.
  • the protocol type corresponding to the target data packet fingerprint a and the protocol type corresponding to the current reference data packet fingerprint b are both HTTP protocols, and the target data packet fingerprint a corresponds to the protocol type.
  • both the fingerprint generation algorithm and the fingerprint generation algorithm corresponding to the current reference data packet fingerprint b are pmercury, it is determined that the target data packet fingerprint a and the current reference data packet fingerprint b match successfully.
  • the matching result of the target data packet fingerprint and the reference data packet fingerprint can be quickly determined through the data packet fingerprint comparison result and the fingerprint correlation information, thereby helping to improve the detection efficiency of the network data packet to be detected.
  • the target confidence corresponding to the network data packet to be detected is calculated based on the confidence correlation information corresponding to the fingerprint of the successfully matched target data packet, including:
  • Step S502 Obtain the target protocol confidence level corresponding to the target protocol type.
  • Step S504 Obtain the corresponding target algorithm confidence based on the fingerprint generation algorithm corresponding to the successfully matched target data packet fingerprint.
  • step S506 the target confidence is obtained based on the target protocol confidence and the target algorithm confidence.
  • the detection server can obtain the confidence of the target protocol corresponding to the target protocol type, obtain the confidence of the target algorithm corresponding to the fingerprint generation algorithm corresponding to the fingerprint of each successfully matched target data packet, and compare the confidence of the target protocol with the confidence of the target algorithm. Fusion to get the target confidence.
  • the detection server may multiply the target protocol confidence and target algorithm confidence to obtain the target confidence, or may first obtain the algorithm confidence statistical value by performing statistics on each target algorithm confidence, and then calculate the algorithm confidence The statistical value and the confidence of the target protocol are fused to obtain the confidence of the target.
  • Counting the confidence degrees of each target algorithm may specifically be adding the confidence degrees of each target algorithm to obtain the statistical value of the algorithm confidence degree, and calculating the average value of the confidence degrees of each target algorithm to obtain the statistical value of the algorithm confidence degree, or it can be calculated according to a preset formula.
  • the confidence of each target algorithm is fused to obtain the statistical value of algorithm confidence.
  • the confidence of the target protocol corresponding to the target protocol type is obtained, the confidence of the corresponding target algorithm is obtained based on the fingerprint generation algorithm corresponding to the fingerprint of the successfully matched target data packet, and the confidence of the target algorithm is obtained based on the confidence of the target protocol and the confidence of the target algorithm. Confidence.
  • the calculation of the target confidence degree comprehensively considers the target protocol type and the fingerprint generation algorithm corresponding to the fingerprint of the successfully matched target data packet, and the calculated target confidence degree is more accurate and reliable.
  • the target protocol type includes the current layer protocol type and the associated layer protocol type
  • the target confidence degree is obtained based on the target protocol confidence degree and the target algorithm confidence degree, including:
  • step S602 the confidence levels of each target algorithm corresponding to the protocol type of the same layer are sorted from large to small, and the sorting result corresponding to the protocol type of each layer is obtained.
  • Step S604 Obtain the first confidence level corresponding to the protocol type of each layer according to the confidence level of the target algorithm ranked first and ranked second in the sorting result corresponding to the protocol type of the same layer.
  • Step S606 sort the remaining target algorithm confidence levels and the corresponding first confidence levels in the sorting results corresponding to the same layer protocol types from large to small, obtain the updated sorting results corresponding to the protocol types of each layer, and return the above according to the same.
  • the confidence of the target algorithm ranked first and the second ranked is the step of obtaining the first confidence degree corresponding to each layer protocol type, until the confidence degree of each target algorithm corresponding to the same layer protocol type is involved in data processing. , to obtain the second confidence level corresponding to the protocol type of each layer.
  • Step S608 obtaining intermediate confidences corresponding to the protocol types of each layer based on the target protocol confidences and the second confidences corresponding to the protocol types of the same layer, and obtaining the target confidences according to the intermediate confidences.
  • the target protocol type includes the current layer protocol type and the associated layer protocol type, and the protocol types of different layers need to be calculated differently.
  • the detection server can sort the confidence levels of each target algorithm corresponding to the protocol type of the same layer from large to small, and obtain the sorting result corresponding to the protocol type of each layer. Obtain the confidences of the first and second ranked target algorithms in the ranking results corresponding to the same layer protocol type, and perform the weighted summation of the confidences of the first and second ranked target algorithms corresponding to the same layer protocol type to obtain the protocols of each layer.
  • the first confidence level corresponding to the type is
  • the remaining target algorithm confidences and the corresponding first confidences in the sorting results corresponding to the same layer protocol type are sorted from large to small, and the update sorting results corresponding to each layer protocol type are obtained, and the update sorting results corresponding to the same layer protocol type are obtained.
  • the confidences of the target algorithms ranked first and second in the ranking are weighted and summed to obtain the first update confidence corresponding to the protocol type of each layer by weighted summation of the confidences of the first and second ranked target algorithms corresponding to the same layer protocol type,
  • the final calculation result corresponding to each layer protocol type is used as the second confidence level corresponding to each layer protocol type.
  • Each intermediate confidence level may be weighted and summed to obtain the target confidence level, or each intermediate confidence level may be used as the target confidence level respectively.
  • obtaining the first confidence level corresponding to the protocol type of each layer according to the confidence level of the target algorithm ranked first and ranked second in the sorting result corresponding to the same layer protocol type includes: obtaining the target algorithm currently ranked first The fusion weight corresponding to the confidence degree and the confidence degree of the currently ranked second target algorithm; the fusion weight corresponding to the confidence degree of the currently ranked second target algorithm is determined based on the confidence degree of the current ranked first target algorithm; based on the same layer protocol type corresponding The fusion weight of , fuses the confidences of the first and second target algorithms in the ranking results corresponding to the protocol types of the same layer, and obtains the first confidences corresponding to the protocol types of each layer.
  • the fusion weight corresponding to the confidence level of the target algorithm ranked second may be determined based on the confidence level of the target algorithm ranked first in the ranking results corresponding to the protocol types of the same layer. The difference between the confidence of the first target algorithm and the confidence of the preset algorithm is used as the fusion weight corresponding to the confidence of the second target algorithm currently ranked.
  • the preset algorithm confidence level can be set as required, for example, set to 1.
  • the fusion weight corresponding to the confidence level of the target algorithm ranked first in the ranking results corresponding to the same layer protocol type may be a preset weight, and the preset weight may be set as required, for example, set to 1.
  • the detection server can sort the corresponding protocol types of the same layer based on the fusion weights corresponding to the protocol types of the same layer. In the result, the confidences of the target algorithms ranked first and second are fused to obtain the first confidences corresponding to the protocol types of each layer.
  • the value range of M i is [0, 1], and the larger the M i is, the greater the abnormality of the corresponding network data packet.
  • the value range of N k is [0, 1], and the larger N k is, the greater the abnormal degree of the corresponding network data packet.
  • ⁇ pro i , alg j , hash ij > represents the target data packet fingerprint array
  • ⁇ l_pro i , l_alg j , l_hash ij , l_count ij > represents the reference data packet fingerprint array
  • pro i represents the protocol type i corresponding to the target data packet fingerprint
  • alg j represents the fingerprint generation algorithm j corresponding to the target data packet fingerprint
  • hash ij represents the target data packet fingerprint generated based on the fingerprint generation algorithm j corresponding to the protocol type i
  • l_pro i represents The protocol type i corresponding to the reference data packet fingerprint
  • l_alg j represents the fingerprint generation algorithm j corresponding to the reference data packet fingerprint
  • l_hash ij represents the reference data packet fingerprint generated based on the fingerprint generation algorithm j corresponding to the protocol type i
  • l_count ij represents the reference data packet The statistics corresponding to the fingerprint.
  • each successfully matched target data packet fingerprint can be determined, and then the intermediate confidence of the protocol type corresponding to each successfully matched target data packet fingerprint can be cyclically accumulated to obtain the target confidence corresponding to the network data packet to be detected.
  • Score. Target confidence score+ sum(M i *sumN), i++.
  • the network data packet to be detected is considered to be an abnormal network data packet; otherwise, the network data packet to be detected is considered to be a normal network data packet.
  • an alarm message can be sent to the operation and maintenance personnel, or the to-be-detected network data packet can be blocked.
  • the confidence of each target algorithm corresponding to the same layer protocol type is sorted in reverse order from large to small, that is, sort(score( ⁇ P, alg k >), greater), and then according to the accumulation formula Accumulate.
  • the accumulation formula can ensure that when the number of target data packet fingerprints are successfully matched, the algorithm confidence value obtained by accumulation under the same layer protocol type is closer to 1, but not more than 1, which can guarantee the algorithm confidence value obtained by accumulation under the same layer protocol type A value greater than the maximum target algorithm confidence.
  • score pro (http) 0.9
  • score alg ( ⁇ http, fatt>) 0.8
  • score alg ( ⁇ http, pmercury>) 0.6
  • mb_THR 0.7.
  • mb_THR represents the reference confidence.
  • a target data packet fingerprint of the network data packet to be detected matches a reference data packet fingerprint generated by the fingerprint generation algorithm fatt corresponding to the HTTP protocol in the reference data packet fingerprint database, and another target data packet fingerprint and the reference data packet fingerprint database.
  • the fingerprint of a reference packet generated by pmercury, the fingerprint generation algorithm corresponding to the HTTP protocol matches.
  • the target data packet fingerprint of the network data packet to be detected does not match the reference data packet fingerprint generated based on the TCP protocol in the reference data packet fingerprint database, there is no need to calculate the intermediate confidence corresponding to the TCP protocol, and the HTTP protocol can be directly used.
  • the corresponding intermediate confidence is used as the target confidence corresponding to the network data packet to be detected.
  • determining the abnormality detection result of the network data packet to be detected based on the reference confidence level and the target confidence level includes: when the target confidence level is greater than the reference confidence level, determining that the abnormality detection result is a network data packet abnormality; When the degree is less than or equal to the reference confidence degree, it is determined that the abnormal detection result is that the network data packet is normal.
  • the reference confidence may be a comprehensive confidence threshold. It can also be that one protocol type corresponds to one confidence threshold, that is, there are multiple reference confidences. When the target confidence is greater than the reference confidence, the abnormal detection result is determined to be an abnormal network data packet, and when the target confidence is less than or equal to the reference confidence, the abnormal detection result is determined to be a normal network data packet.
  • the confidence thresholds corresponding to different protocol types may be the same or different. For example, the confidence threshold corresponding to the HTTP protocol may be 0.7, and the confidence threshold corresponding to the TCP protocol may be 0.65.
  • the abnormal detection result of the network data packet to be detected can be quickly determined.
  • the method further includes: when the abnormality detection result is that the network data packet is abnormal, acquiring the blocking data packet; sending the blocking data packet to the receiver corresponding to the network data packet to be detected, so that the receiver Stop establishing the communication connection of the sender corresponding to the network data packet to be detected.
  • the detection server may obtain the blocking data packet, and send the blocking data packet to the receiver corresponding to the network data packet to be detected.
  • the detection server can obtain the relevant information of the sender from the network data packet to be detected, and carry the relevant information of the sender into the blocking data packet.
  • the communication connection of the sender to block the network packets to be detected.
  • the blocking data packet may be an RST (Reset) packet.
  • the RST packet is used to disconnect the communication connection between the communicating parties.
  • the blocking data packet is sent to the receiver corresponding to the network data packet to be detected, and the corresponding network data packet to be detected can be quickly disconnected.
  • the communication connection between the sender and the receiver can effectively prevent hacker attacks.
  • the method further includes: regularly counting the matching success rate corresponding to each reference data packet fingerprint; filtering out the reference data packet fingerprint whose matching success rate is less than a preset threshold from the reference data packet fingerprint database.
  • the reference data packet fingerprints in the reference data packet fingerprint database may be filtered periodically to filter out the reference data packet fingerprints that have not been matched for a long time. It can be understood that if the reference data packet fingerprint does not match successfully for a long time, it can be considered that the hacker attack corresponding to the reference data packet fingerprint has stopped. Because the reference data packet fingerprint database is constantly expanding, the reference data packet fingerprints that have not been matched for a long time can be filtered out, so as to improve the space utilization of the reference data packet fingerprint database.
  • the detection server may periodically count the matching success rate corresponding to each reference data packet fingerprint in the reference data packet fingerprint database, and filter out the reference data packet fingerprints whose matching success rate is less than a preset threshold from the reference data packet fingerprint database. Specifically, the detection server may obtain the matching success rate corresponding to the reference data packet fingerprint according to the number of successful matching of the reference data packet fingerprint and the storage time of the reference data packet fingerprint in the reference data packet fingerprint database. Different storage times correspond to different reference matching times, and the ratio of the successful matching times to the reference matching times is used as the matching success rate.
  • the preset threshold can be set according to actual needs.
  • the present application also provides an application scenario where the above-mentioned abnormal network data detection method is applied.
  • the application of the abnormal network data detection method in this application scenario is as follows:
  • the abnormal network data detection method can be used for second dial attack detection.
  • Second dialing refers to the principle of using domestic broadband dial-up Internet access. Every time the connection is disconnected and reconnected, a new IP will be obtained, and the IP jump of dial-by-second is realized.
  • Hackers can use second-dial technology to attack computer equipment to steal core data.
  • both normal users and second dial users can send network data packets to the service server through the terminal.
  • the terminal can use the switch to communicate with the service server.
  • the detection server can obtain candidate network data packets within the same time window from the switch by bypassing traffic, generate candidate data packet fingerprints corresponding to each candidate network data packet, and establish a second dial black database according to the candidate data packet fingerprints.
  • the detection server can obtain the target network data packet (real-time traffic) from the switch by bypassing the traffic, generate the target data packet fingerprint corresponding to the target network data packet, match the target data packet fingerprint with the reference data packet fingerprint, and calculate the matching success
  • the target confidence level corresponding to the target data packet fingerprint, and the second dial IP is identified according to the comparison result of the target confidence level and the confidence threshold.
  • the target network data packet is determined to be an abnormal network data packet, it is determined that it matches the second dial IP attack, an alarm is issued, and the abnormal network data packet is blocked in real time.
  • the target confidence is less than or equal to the confidence threshold, it is determined that the target network data packet is a normal network data packet.
  • At least one candidate data packet fingerprint corresponding to each candidate network data packet is generated, and each candidate data packet fingerprint is combined to obtain a candidate data packet fingerprint set.
  • the current layer protocol type and the association layer protocol type corresponding to the candidate network data packet are obtained, the matching feature field is obtained from the candidate network data packet based on at least one fingerprint generation algorithm corresponding to the current layer protocol type, and the corresponding feature field is generated according to the feature field.
  • the candidate data packet fingerprint based on at least one fingerprint generation algorithm corresponding to the association layer protocol type, obtains the matching characteristic field from the candidate network data packet, and generates the corresponding candidate data packet fingerprint according to the characteristic field.
  • Each candidate network data packet corresponds to multiple candidate data packets.
  • the packet fingerprints are combined to obtain a set of candidate packet fingerprints.
  • the current layer protocol type corresponding to the HTTP protocol network data packet is the HTTP protocol
  • the corresponding association layer protocol type is the TCP protocol
  • the fingerprint generation algorithm corresponding to the HTTP protocol includes fatt and pmercury
  • the fingerprint generation algorithm corresponding to the TCP protocol includes pmercury.
  • the matching feature fields are obtained from the candidate network data packets, and the corresponding candidate data packet fingerprint hash1 is generated according to the feature fields.
  • the fingerprint generation algorithm pmercury corresponding to the HTTP protocol the matching feature fields are obtained from the candidate network data packets, and the corresponding candidate data packet fingerprint hash2 is generated according to the feature fields.
  • the matching feature fields are obtained from the candidate network data packets, and the corresponding candidate data packet fingerprint hash3 is generated according to the feature fields.
  • the candidate data packet fingerprints corresponding to the HTTP protocol network data packets include the candidate data packet fingerprint hash1, the candidate data packet fingerprint hash2, and the candidate data packet fingerprint hash3.
  • clustering is performed based on the same candidate data packet fingerprints generated by the same fingerprint generation algorithm corresponding to the same protocol type to obtain multiple different clusters, and the number of candidate data packet fingerprints in the same cluster is counted to obtain each cluster.
  • the statistics corresponding to the clusters are used as the reference data packet fingerprints, and each reference data packet fingerprint is combined to obtain the second dial black library (that is, the reference data packet fingerprint library).
  • the candidate packet fingerprint hash1 can be added as a reference packet fingerprint to the second dial black database .
  • the second dial black library can store candidate data packet fingerprint hash1 according to the array of ⁇ HTTP protocol, fingerprint generation algorithm fatt, hash1, 55>.
  • 2-1 Generate at least one target data packet fingerprint corresponding to a network data packet to be detected (ie, real-time traffic), and combine each target data packet fingerprint to obtain a target data packet fingerprint set.
  • a network data packet to be detected ie, real-time traffic
  • the current layer protocol type and the association layer protocol type corresponding to the network data packet to be detected are obtained, and the matching feature field is obtained from the network data packet to be detected based on at least one fingerprint generation algorithm corresponding to the current layer protocol type , and generate the corresponding target packet fingerprint according to the feature field.
  • the matching feature field is obtained from the network data packet to be detected based on at least one fingerprint generation algorithm corresponding to the current layer protocol type , and generate the corresponding target packet fingerprint according to the feature field.
  • Obtain matching feature fields from the network data packets to be detected based on at least one fingerprint generation algorithm corresponding to the type of the association layer protocol generate the corresponding target data packet fingerprints according to the feature fields, and combine the fingerprints of the target data packets to obtain the corresponding target data packets.
  • a collection of target packet fingerprints are obtained from the network data packet to be detected based on at least one fingerprint generation algorithm corresponding to the type of the association layer protocol.
  • the target data packet fingerprint is matched with the reference data packet fingerprint in the second dial black library.
  • the protocol type corresponding to the target data packet fingerprint is the same as the protocol type corresponding to the reference data packet fingerprint
  • the fingerprint generation algorithm corresponding to the target data packet fingerprint and the fingerprint generation algorithm corresponding to the reference data packet fingerprint When they are consistent, it is determined that the fingerprint of the target data packet and the fingerprint of the reference data packet match successfully.
  • the corresponding target algorithm confidence is obtained, and the confidence of each target algorithm corresponding to the same layer protocol type is sorted from large to small, according to the sorting order, according to the cumulative
  • the formula accumulates the confidence degrees of each target algorithm corresponding to the same layer protocol type to obtain the algorithm confidence degree statistical value (ie the second confidence degree) corresponding to each layer protocol type. Multiply the target protocol confidence corresponding to the same layer protocol type and the algorithm confidence statistical value to obtain the intermediate confidence corresponding to each layer protocol type, and add each intermediate confidence to obtain the target confidence.
  • the detection server can send alarm information to the terminal corresponding to the operation and maintenance personnel, or directly block the network data packet to be detected.
  • a target data packet fingerprint of the network data packet to be detected matches a reference data packet fingerprint generated by the fingerprint generation algorithm fatt corresponding to the HTTP protocol in the reference data packet fingerprint database, and another target data packet fingerprint matches the reference data packet fingerprint database.
  • the fingerprint of a reference packet generated by the fingerprint generation algorithm pmercury corresponding to the HTTP protocol matches, and the fingerprint of another target packet matches the fingerprint of a reference packet generated by the fingerprint generation algorithm pmercury corresponding to the HTTP protocol in the reference packet fingerprint library.
  • the above abnormal network data detection method avoids the dependence of traditional detection methods on IP, and focuses on the request content, the client and even the attacker itself, that is, the characteristic information of network data packets, thereby reducing the error of the second dial attack.
  • the judgment rate is improved, and the detection accuracy of the second dial attack is improved.
  • the set of data packet fingerprints corresponding to the same network data packet includes data packet fingerprints corresponding to various protocol types and various fingerprint generation algorithms, which enriches the data packet fingerprints, can cover more attack types, and can further improve the attack rate of the second dial attack. Detection accuracy.
  • steps in the flowcharts of FIGS. 2 , 4 , 5 , and 6 are displayed in sequence according to the arrows, these steps are not necessarily executed in the sequence indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2, 4, 5, and 6 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but may be executed at different times. These steps Alternatively, the order of execution of the stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or stages in the other steps.
  • a device for detecting abnormal network data is provided.
  • the device can use software modules or hardware modules, or a combination of the two to become a part of computer equipment.
  • the device specifically includes: data Packet acquisition module 1002, data packet fingerprint generation module 1004, data packet fingerprint matching module 1006, confidence calculation module 1008 and detection result determination module 1010, wherein:
  • a data packet obtaining module 1002 configured to obtain a network data packet to be detected
  • the data packet fingerprint generation module 1004 is configured to extract features from the network data packets to be detected according to the target protocol type corresponding to the network data packets to be detected, and generate a corresponding target data packet fingerprint set; the target data packet fingerprint set includes at least one target data packet fingerprint;
  • the data packet fingerprint matching module 1006 is used to match the target data packet fingerprint with the reference data packet fingerprint in the reference data packet fingerprint database
  • a confidence calculation module 1008 configured to calculate the target confidence corresponding to the network data packet to be detected based on the confidence correlation information corresponding to the fingerprint of the successfully matched target data packet;
  • the detection result determination module 1010 is configured to obtain a reference confidence level, and determine an abnormal detection result of the network data packet to be detected based on the reference confidence level and the target confidence level.
  • the apparatus further includes:
  • the reference data packet fingerprint library establishment module 1001 is used to obtain a candidate network data packet set; the candidate network data packet set includes a plurality of candidate network data packets within the same time window; Extracting features from data packets to generate a corresponding set of candidate data packet fingerprints.
  • the candidate data packet fingerprint set includes candidate data packet fingerprints corresponding to each candidate network data packet; the candidate data packet fingerprints in the candidate data packet fingerprint set are aggregated based on fingerprint association information.
  • the clustering result is obtained; the reference data packet fingerprint is determined from the candidate data packet fingerprint set according to the clustering result, and the reference data packet fingerprint database is obtained by combining the reference data packet fingerprints.
  • the current network data packet is the network data packet to be detected, and the data packet fingerprint generation module is further configured to obtain the current protocol type corresponding to the current network data packet, and the current protocol type includes the current layer protocol type and the association layer protocol type; Obtain at least one fingerprint generation algorithm corresponding to the current protocol type; extract the matching feature field from the current network data packet based on the fingerprint generation algorithm, and obtain the current packet fingerprint generated by each fingerprint generation algorithm corresponding to the current protocol type according to the feature field; The data packet fingerprint combination obtains the data packet fingerprint set corresponding to the current network data packet.
  • the current network data packet is a candidate network data packet
  • the reference data packet fingerprint library establishment module is further configured to obtain the current protocol type corresponding to the current network data packet, and the current protocol type includes the current layer protocol type and the association layer protocol type Obtain at least one fingerprint generation algorithm corresponding to the current protocol type; extract the matching feature field from the current network data packet based on the fingerprint generation algorithm, and obtain the current data packet fingerprint generated by each fingerprint generation algorithm corresponding to the current protocol type according to the feature field; each The current data packet fingerprint combination obtains the data packet fingerprint set corresponding to the current network data packet.
  • the fingerprint association information includes a candidate protocol type and a fingerprint generation algorithm corresponding to the candidate data packet fingerprint
  • the reference data packet fingerprint library establishment module is further configured to generate the same candidate data based on the same fingerprint generation algorithm corresponding to the same protocol type.
  • the packet fingerprints are clustered to obtain a plurality of different clusters, the number of candidate data packet fingerprints in the same cluster is counted, and the statistical value corresponding to each cluster is obtained.
  • the reference data packet fingerprint library establishment module is also used to use the candidate data packet fingerprints in the clusters whose statistical values are greater than the statistical threshold as the reference data packet fingerprints.
  • the data packet fingerprint matching module is further configured to compare the target data packet fingerprint with the current reference data packet fingerprint to obtain a data packet fingerprint comparison result; compare the fingerprint correlation information corresponding to the target data packet fingerprint with the current reference data packet The fingerprint correlation information corresponding to the fingerprint is compared to obtain the fingerprint correlation information comparison result; when the data packet fingerprint comparison result and the fingerprint correlation information comparison result are consistent, it is determined that the target data packet fingerprint and the current reference data packet fingerprint match successfully.
  • the confidence calculation module is further configured to obtain the confidence of the target protocol corresponding to the target protocol type; obtain the confidence of the corresponding target algorithm based on the fingerprint generation algorithm corresponding to the fingerprint of the successfully matched target data packet; based on the confidence of the target protocol and the target algorithm confidence to get the target confidence.
  • the target protocol type includes the current layer protocol type and the associated layer protocol type
  • the confidence level calculation module is further configured to sort the confidence levels of each target algorithm corresponding to the same layer protocol type from large to small to obtain protocols of each layer
  • the ranking result corresponding to the type of the same layer protocol; the first confidence score corresponding to the protocol type of each layer is obtained according to the confidence of the first and second target algorithms in the ranking result corresponding to the same layer protocol type; the ranking result corresponding to the same layer protocol type is The remaining target algorithm confidences and the corresponding first confidences are sorted from large to small, and the updated sorting results corresponding to the protocol types of each layer are obtained, and the sorting results corresponding to the protocol types of the same layer are returned.
  • the target protocol confidence and the second confidence corresponding to the layer protocol type are used to obtain the intermediate confidence corresponding to each layer protocol type, and the target confidence is obtained according to each intermediate confidence.
  • the confidence calculation module is further configured to obtain the fusion weight corresponding to the confidence of the currently ranked first target algorithm and the confidence of the currently ranked second target algorithm, and the fusion weight corresponding to the confidence of the currently ranked second target algorithm The weight is determined based on the confidence of the target algorithm that is currently ranked first. Based on the fusion weight corresponding to the protocol type of the same layer, the confidence of the target algorithm ranked first and second in the ranking results corresponding to the protocol type of the same layer is fused. The first confidence level corresponding to the protocol type of each layer is obtained.
  • the detection result determination module is further configured to, when the target confidence is greater than the reference confidence, determine that the abnormal detection result is a network packet abnormality; when the target confidence is less than or equal to the reference confidence, determine that the abnormal detection result is The network packets are normal.
  • the apparatus further includes:
  • the communication connection blocking module 1011 is used to obtain the blocking data packet when the abnormal detection result is that the network data packet is abnormal; send the blocking data packet to the receiver corresponding to the network data packet to be detected, so that the receiver stops establishing the connection with the network data packet to be detected.
  • the apparatus further includes:
  • the reference data packet fingerprint database updating module 1012 is configured to regularly count the matching success rate corresponding to each reference data packet fingerprint; and filter out the reference data packet fingerprints whose matching success rate is less than a preset threshold from the reference data packet fingerprint database.
  • the generated target data packet fingerprint can represent the characteristic information of the network data packet to be detected, and multiple target data packet fingerprints can represent the characteristic information of the network data packet to be detected from different dimensions, thereby enriching the abnormal network data.
  • the detection dimension of packets improves the detection accuracy of abnormal network data packets.
  • the reference data packet fingerprint library integrates multiple abnormal reference data packet fingerprints, so the successfully matched target data packet fingerprints can represent the abnormal characteristic information of the network data packets to be detected, based on the confidence corresponding to the successfully matched target data packet fingerprints.
  • the target confidence level calculated from the degree correlation information can represent the abnormal degree of the network data packet to be detected. Based on the reference confidence degree and the target confidence degree, the abnormal detection result of the network data packet to be detected can be quickly determined, which improves the detection accuracy of abnormal network data packets. performance and detection efficiency.
  • All or part of the modules in the abnormal network data detection apparatus can be implemented by software, hardware and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 12 .
  • the computer device includes a processor, memory, and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions and a database.
  • the internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the database of the computer equipment is used to store data such as reference packet fingerprints, protocol confidence, algorithm confidence, and fingerprint generation algorithm.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer-readable instructions when executed by a processor, implement a method for detecting abnormal network data.
  • FIG. 12 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device including a memory and one or more processors, where computer-readable instructions are stored in the memory, and the one or more processors implement the above methods when executing the computer-readable instructions steps in the examples.
  • one or more non-transitory computer-readable storage media are provided having computer-readable instructions stored thereon, the computer-readable instructions when executed by one or more processors are provided. The steps in the above method embodiments are implemented.
  • a computer program product or computer program comprising computer readable instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computer device performs the steps in the foregoing method embodiments.
  • Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, or optical memory, and the like.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • the RAM may be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本申请涉及一种异常网络数据检测方法、装置、计算机设备和存储介质。所述方法包括:获取待检测网络数据包;根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合;目标数据包指纹集合包括至少一个目标数据包指纹;将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配;基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度;获取参考置信度,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果。

Description

异常网络数据检测方法、装置、计算机设备和存储介质
本申请要求于2020年10月22日提交中国专利局,申请号为2020111365052,申请名称为“异常网络数据检测方法、装置、计算机设备和存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域,特别是涉及一种异常网络数据检测方法、装置、计算机设备和存储介质。
背景技术
计算机技术的快速发展,为信息的传播带来了极大便利,但与此同时,人们也面临着巨大的信息安全挑战,信息安全问题日益突出,例如,黑客可以在网络上发起攻击,以窃取网络上的机密信息。
传统技术中,对网络攻击事件的检测主要是通过搜集公开的黑名单IP,若检测到当前网络数据包的IP为公开的黑名单IP,则确定该网络数据包为异常网络数据包,确定出现网络攻击事件。然而,网络上IP的数量非常庞大,公开的黑名单IP只是包括黑客使用的部分IP,容易出现漏判,从而导致异常网络数据包的检测准确性低,异常网络数据包的检测效率低。
发明内容
根据本申请提供的各种实施例,提供一种异常网络数据检测方法、装置、计算机设备和存储介质。
一种异常网络数据检测方法,所述方法包括:
获取待检测网络数据包;
根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合;目标数据包指纹集合包括至少一个目标数据包指纹;
将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配;
基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度;及
获取参考置信度,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果。
一种异常网络数据检测装置,所述装置包括:
数据包获取模块,用于获取待检测网络数据包;
数据包指纹生成模块,用于根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合;目标数据包指纹集合包括至少一个目标数据包指纹;
数据包指纹匹配模块,用于将目标数据包指纹和参考数据包指纹库中的参考数据包指 纹进行匹配;
置信度计算模块,用于基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度;及
检测结果确定模块,用于获取参考置信度,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果。
一种计算机设备,包括存储器和一个或多个处理器,所述存储器存储有计算机可读指令,所述计算机可读指令被所述一个或多个处理器执行时,使得所述一个或多个处理器执行上述异常网络数据检测方法的步骤。
一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,其上存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行上述异常网络数据检测方法的步骤。
一种计算机程序产品或计算机程序,所述计算机程序产品或计算机程序包括计算机可读指令,所述计算机可读指令存储在计算机可读存储介质中,计算机设备的处理器从所述计算机可读存储介质读取所述计算机可读指令,所述处理器执行所述计算机可读指令,使得所述计算机设备执行上述异常网络数据检测方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1为一个实施例中异常网络数据检测方法的应用环境图;
图2为一个实施例中异常网络数据检测方法的流程示意图;
图3为一个实施例中对数据包指纹进行分类的示意图;
图4为一个实施例中建立参考数据包指纹库的流程示意图;
图5为一个实施例中计算目标置信度的流程示意图;
图6为另一个实施例中计算目标置信度的流程示意图;
图7为一个实施例中确定待检测网络数据包的异常检测结果的流程示意图;
图8为另一个实施例中异常网络数据检测方法的应用环境图;
图9为另一个实施例中异常网络数据检测方法的流程示意图;
图10为一个实施例中异常网络数据检测装置的结构框图;
图11为另一个实施例中异常网络数据检测装置的结构框图;
图12为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。
云安全(Cloud Security)是指基于云计算商业模式应用的安全软件、硬件、用户、机构、安全云平台的总称。云安全融合了并行处理、网格计算、未知病毒行为判断等新兴技术和概念,通过网状的大量客户端对网络中软件行为的异常监测,获取互联网中木马、恶意程序的最新信息,并发送到服务端进行自动分析和处理,再把病毒和木马的解决方案分发到每一个客户端。
云安全主要研究方向包括:1.云计算安全,主要研究如何保障云自身及云上各种应用的安全,包括云计算机系统安全、用户数据的安全存储与隔离、用户接入认证、信息传输安全、网络攻击防护、合规审计等;2.安全基础设施的云化,主要研究如何采用云计算新建与整合安全基础设施资源,优化安全防护机制,包括通过云计算技术构建超大规模安全事件、信息采集与处理平台,实现对海量信息的采集与关联分析,提升全网安全事件把控能力及风险控制能力;3.云安全服务,主要研究各种基于云计算平台为用户提供的安全服务,如防病毒服务等。
本申请提供的异常网络数据检测方法,可以应用于如图1所示的应用环境中。其中,终端102通过网络与业务服务器104进行通信,终端102通过网络与检测服务器106进行通信,业务服务器104通过网络与检测服务器106进行通信。终端102和业务服务器104可以通过发送网络数据包进行通信。检测服务器106可以获取待检测网络数据包,根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合,目标数据包指纹集合包括至少一个目标数据包指纹。检测服务器106可以将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配,基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度,获取参考置信度,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果。
其中,服务器可以是独立的物理服务器,也可以是多个物理服务器构成的服务器集群或者分布式系统,还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人工智能平台等基础云计算服务的云服务器。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等,但并不局限于此。终端可以不止一个,终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接,本申请在此不做限制。
在一个实施例中,如图2所示,提供了一种异常网络数据检测方法,以该方法应用于图1中的检测服务器106为例进行说明,包括以下步骤:
步骤S202,获取待检测网络数据包。
其中,待检测网络数据包是指待检测是否为异常的网络数据包。待检测网络数据包可以是实时获取的网络数据包,即实时流量。网络数据包用于在终端和业务服务器之间传输数据。网络数据包具体可以包括目的IP地址、源IP地址、净载数据等信息。目的IP地址是指网络数据包对应的接收方的地址,源IP地址是指网络数据包对应的发送方的地址,净载数据是指具体的请求内容或应答内容,例如终端可以向业务服务器发送请求打开新闻网页的网络数据包,业务服务器可以向终端返回当前新闻网页对应的网络数据包。请求内容包括文字、图片、视频等中的至少一种。
具体地,终端和业务服务器之间可以通过发送网络数据包进行通信。在通信过程中,检测服务器可以获取终端或业务服务器发送的当前网络数据包作为待检测网络数据包。
在一个实施例中,终端和业务服务器之间可以借助交换机传输网络数据包。检测服务器可以采用旁路的方式从交换机中侦听网络数据包。
步骤S204,根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合;目标数据包指纹集合包括至少一个目标数据包指纹。
其中,目标协议类型是指待检测网络数据包对应的协议类型。协议类型是指传输网络数据包的网络协议类型。协议类型包括当前层协议类型和关联层协议类型中的至少一种。当前层协议类型是指网络数据包对应的应用层协议类型,关联层协议类型是指网络数据包对应的其他层协议类型,具体可以包括网络数据包对应的传输层协议类型、网络层协议类型、数据链路层协议类型中的至少一种。例如,网络数据包为HTTP协议(Hyper Text Transfer Protocol,超文本传输协议)网络数据包,那么待检测网络数据包对应的当前层协议类型可以为HTTP协议,由于HTTP协议对应的传输层协议类型为TCP协议(Transmission Control Protocol,传输控制协议),那么该网络数据包对应的关联层协议类型可以为TCP协议。
目标数据包指纹是指待检测网络数据包对应的数据包指纹。数据包指纹是根据网络协议类型从网络数据包中提取对应的特征信息,根据提取到的特征信息生成的标识,用于标识网络数据包。一个网络数据包可以对应至少一个数据包指纹。一种协议类型可以生成至少一个数据包指纹,例如,网络数据包对应的协议类型包括HTTP协议和TCP协议,基于HTTP协议可以生成至少一个数据包指纹,基于TCP协议可以生成至少一个数据包指纹。一个网络数据包对应的不同数据包指纹可以从不同的维度表征网络数据包的特征信息。不同的网络数据包对应不同的数据包指纹集合,但是不同的数据包指纹集合可以包括至少一个不同的数据包指纹,例如,从同一终端发送的网络数据包A和网络数据包B,由于网络数据包A和网络数据包B对应的发送方相同,网络数据包A和网络数据包B可以存在相同的数据包指纹,该相同的数据包指纹可以是根据特征信息中发送方的相关信息生成的。
具体地,检测服务器可以从待检测网络数据包中获取待检测网络数据包对应的目标协议类型,根据目标协议类型从待检测网络数据包中提取特征信息,根据提取的特征信息生成对应的至少一个目标数据包指纹,各个目标数据包指纹组成目标数据包指纹集合。检测服务器可以根据待检测网络数据包对应的当前层协议类型对待检测网络数据包提取特征生成对应的目标数据包指纹集合,也可以根据待检测网络数据包对应的关联层协议类型对待检测网络数据包提取特征生成对应的目标数据包指纹集合,还可以根据待检测网络数据包对应的当前层协议类型和关联层协议类型对待检测网络数据包提取特征生成对应的目标数据包指纹集合。
在一个实施例中,检测服务器可以从待检测网络数据包中获取待检测网络数据包对应的当前层协议类型,在本地或从其他终端、服务器获取当前层协议类型对应的指纹生成算法,基于指纹生成算法从待检测网络数据包中提取与该指纹生成算法匹配的特征字段,根据提取到的特征字段得到由当前层协议类型对应的指纹生成算法生成的目标数据包指纹。当前层协议类型对应的指纹生成算法可以为至少一个。若当前层协议类型对应的指纹生成算法为多个,可以生成当前层协议类型对应的多个目标数据包指纹。同理,检测服务器可以从待检测网络数据包中获取待检测网络数据包对应的关联层协议类型,在本地或从其他终端、服务器获取关联层协议类型对应的指纹生成算法,基于指纹生成算法从待检测网络 数据包中提取与该指纹生成算法匹配的特征字段,根据提取到的特征字段得到由关联层协议类型对应的指纹生成算法生成的目标数据包指纹。关联层协议类型对应的指纹生成算法可以为至少一个。当关联层协议类型对应的指纹生成算法为多个时,可以生成关联层协议类型对应的多个目标数据包指纹。可以是当前层协议类型对应的各个目标数据包指纹组合得到目标数据包指纹集合,也可以是关联层协议类型对应的各个目标数据包指纹组合得到目标数据包指纹集合,还可以是当前层协议类型对应的各个目标数据包指纹和关联层协议类型对应的各个目标数据包指纹组合得到目标数据包指纹集合。
在一个实施例中,网络数据包包括多个字段,不同的字段可以表征网络数据包不同的特征信息。例如,网络数据包的字段包括目的IP地址、源IP地址、协议类型、发送时间、包大小、净载数据等。根据提取到的特征字段生成数据包指纹具体可以是将特征字段按照指纹生成算法规定的字段顺序进行排列组合得到目标特征信息,计算目标特征信息的哈希值得到数据包指纹。可以理解,不同协议类型对应的网络数据包中相同类型字段的字段位置可以相同可以不同,目标特征信息还可以进一步包括各个特征字段在网络数据包中的字段位置。
在一个实施例中,每一种协议可以分别对应多种指纹生成算法,那么根据协议类型和指纹生成算法可以对数据包指纹进行分类。如图3所示,网络协议包括HTTP协议、DHCP协议(Dynamic Host Configuration Protocol,动态主机配置协议)、SSH协议(Secure Shell,安全外壳协议)、SSL协议(Secure Sockets Layer,安全套接字协议)、TLS协议(Transport Layer Security,安全传输层协议)、RDP协议(Remote Display Protocol,远程显示协议)、TCP协议和UDP协议(User Datagram Protocol,用户数据报协议)。DHCP协议是基于UDP协议传输的,SSH协议是基于TCP协议传输的,SSL协议/TLS协议是基于TCP协议传输的,RDP协议是基于TCP协议传输的。HTTP协议对应的指纹生成算法可以包括fatt、pmercury等。fatt(Fingerprint All The Things)是一个基于pyshark的脚本,用于从pcap(Packet Capture Data)文件和网络数据包中提取数据包指纹。pmercury是mercury的python实现,mercury是一个Linux应用程序,用于从网络数据包中提取数据包指纹。DHCP协议对应的指纹生成算法可以包括pmercury等。SSH协议对应的指纹生成算法可以包括HASSH-SSH_MSG_KEXINIT、pmercury等。HASSH是一个开源的算法,SSH_MSG_KEXINIT是密钥交换数据包头,HASSH-SSH_MSG_KEXINIT是一种识别SSH客户端指纹的方法。SSL协议和TLS协议对应的指纹生成算法可以包括JA3、pmercury等。JA3是一种在线识别TLS客户端指纹的方法。RDP协议对应的指纹生成算法可以包括fatt等。TCP协议和UDP协议对应的指纹生成算法可以包括pmercury等。一种协议对应的一种指纹生成算法作为一个类别,这样分类使得在计算数据包指纹的时候,一个网络数据包可以对应多个协议,也可以对应多重指纹生成算法,在分类网络数据包的时候,可以从不同维度进行分类,做出更加准确的判断。例如,对于HTTP协议,其本身是基于TCP协议的,所以一个HTTP协议网络数据包对应的数据包指纹可以包括由TCP协议对应的指纹生成算法pmercury生成的数据包指纹、由HTTP协议对应的指纹生成算法pmercury生成的数据包指纹、由HTTP协议对应的指纹生成算法fatt生成的数据包指纹。不同的指纹生成算法从网络数据包中提取的特征字段可以相同可以不同。对于同一网络数据包,通过不同协议对应的同一指纹生成算法生成的数据包指纹是不同的,因为针对不同的协议,同一 指纹生成算法从网络数据包中提取的特征字段不同。例如,对于HTTP协议和TCP协议,指纹生成算法pmercury从网络数据包中提取的特征字段不同。
步骤S206,将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配。
其中,参考数据包指纹库包括多个参考数据包指纹。参考数据包指纹库是对多个候选网络数据包对应的数据包指纹进行聚类分析后生成的。各个候选网络数据包是指设定历史时间段内的网络数据包,例如获取在当前时刻之前的5分钟内采集到的网络数据包作为候选网络数据包。其中,历史时间段的长短可以根据实际情况确定,本发明实施例对此不做限定。可以理解,黑客的攻击行为是一个持续性且有共性的攻击行为,例如,黑客主要从同一终端发起攻击,黑客主要从同一浏览器发起攻击,黑客主要从同一账号发起攻击等,那么在进行聚类分析时,若同一数据包指纹的数量大于阈值时,可以认为该数据包指纹为异常的数据包指纹,将该数据包指纹作为参考数据包指纹加入参考数据包指纹库。
具体地,在计算得到待检测网络数据包对应的目标数据包指纹后,检测服务器可以将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配,根据匹配结果进一步确定待检测网络数据包的异常检测结果。当目标数据包指纹和参考数据包指纹相同,并且目标数据包指纹和参考数据包指纹对应的指纹关联信息也相同时,可以确定该目标数据包指纹和参考数据包指纹匹配成功。指纹关联信息包括数据包指纹对应的协议类型和指纹生成算法中的至少一种。可以理解,一个待检测网络数据包可以对应多个目标数据包指纹,各个目标数据包指纹可以分别与参考数据包指纹库中的各个参考数据包指纹进行匹配,各个目标数据包指纹可以均匹配成功,也可以均匹配失败,还可以是有至少一个匹配成功。
在一个实施例中,参考数据包指纹库可以是由检测服务器生成的,也可以是由其他终端或服务器生成后发送至检测服务器。
步骤S208,基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度。
其中,置信度关联信息是指用于计算网络数据包对应的置信度的关联信息。数据包指纹的置信度关联信息包括数据包指纹对应的协议类型的协议置信度、在该协议类型下数据包指纹对应的指纹生成算法的算法置信度中的至少一种。例如,匹配成功的目标数据包指纹为由HTTP协议对应的指纹生成算法1生成的数据包指纹,那么该目标数据包指纹对应的置信度关联信息包括HTTP协议对应的协议置信度和在HTTP协议下指纹生成算法1所对应的算法置信度。协议置信度和算法置信度可以是根据实际情况确定的,例如,根据安全经验人工设置的,根据自定义公式计算得到。不同协议下同一指纹生成算法对应的算法置信度可以相同可以不同。目标置信度是用于确定网络数据包的异常程度,目标置信度越大,网络数据包的异常程度越大。
具体地,检测服务器根据匹配结果可以筛选出匹配成功的目标数据包指纹,获取匹配成功的目标数据包指纹对应的置信度关联信息,根据该置信度关联信息计算得到待检测网络数据包对应的目标置信度。
在一个实施例中,检测服务器可以将各个匹配成功的目标数据包指纹对应的协议置信度和算法置信度进行加权求和得到目标置信度。检测服务器也可以将各个协议置信度和各个算法置信度进行加权相乘得到目标置信度。当匹配成功的目标数据包指纹包括不同协议类型对应的目标数据包指纹时,可以先分别计算各个协议类型对应的中间置信度,将各个 中间置信度进行加权求和得到目标置信度。在计算各个协议类型对应的中间置信度时,可以将同一协议类型对应的各个算法置信度进行加权求和得到算法置信度统计值,将算法置信度统计值和对应的协议置信度进行加权相乘得到对应的中间置信度。也可以将各个协议类型对应的中间置信度分别作为待检测网络数据包对应的目标置信度。
步骤S210,获取参考置信度,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果。
其中,参考置信度可以是根据实际情况确定的,例如,根据安全经验人工设置,根据自定义公式计算得到。
具体地,检测服务器可以获取参考置信度,将参考置信度和目标置信度进行比较,根据比较结果确定待检测网络数据包的异常检测结果。异常检测结果包括网络数据包异常和网络数据包正常。当待检测网络数据包的异常检测结果为网络数据包异常时,检测服务器可以向运维人员发送告警信息,以便运维人员及时进行安全维护。当待检测网络数据包的异常检测结果为网络数据包异常时,检测服务器还可以直接阻断待检测网络数据包,阻止黑客攻击。
在一个实施例中,可以是所有待检测网络数据包都使用同一个参考置信度,也就是,只有一个参考置信度。也可以是一个协议类型对应一个参考置信度,例如,匹配成功的目标数据包指纹对应的协议类型包括http协议和tcp协议,http协议对应参考置信度1,tcp协议对应参考置信度2,那么当基于http协议对应的目标数据包指纹的置信度关联信息计算得到的置信度统计值大于参考置信度1,并且基于tcp协议对应的目标数据包指纹的置信度关联信息计算得到的置信度统计值大于参考置信度2时,确定待检测网络数据包的异常检测结果为网络数据包异常。
上述异常网络数据检测方法中,通过获取待检测网络数据包,根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合,目标数据包指纹集合包括至少一个目标数据包指纹。这样,生成的目标数据包指纹可以表征待检测网络数据包的特征信息,多个目标数据包指纹可以从不同维度表征待检测网络数据包的特征信息,从而丰富了异常网络数据包的检测维度,提高了异常网络数据包的检测准确性。将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配,基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度,获取参考置信度,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果。这样,参考数据包指纹库集成了多个异常的参考数据包指纹,因此匹配成功的目标数据包指纹可以表征待检测网络数据包异常的特征信息,基于匹配成功的目标数据包指纹对应的置信度关联信息计算得到的目标置信度可以表征待检测网络数据包的异常程度,基于参考置信度和目标置信度可以快速确定待检测网络数据包的异常检测结果,提高了异常网络数据包的检测准确性和检测效率。
在一个实施例中,如图4所示,获取待检测网络数据包之前,所述方法还包括:
步骤S402,获取候选网络数据包集合;候选网络数据包集合包括在同一时间窗内的多个候选网络数据包。
步骤S404,根据候选网络数据包对应的候选协议类型对候选网络数据包提取特征,生成对应的候选数据包指纹集合,候选数据包指纹集合包括各个候选网络数据包对应的候选 数据包指纹。
具体地,检测服务器可以在一个时间窗内获取多个候选网络数据包组成候选网络数据包集合。检测服务器可以根据候选网络数据包对应的候选协议类型对候选网络数据包提取特征,生成各个候选网络数据包分别对应的至少一个候选数据包指纹,各个候选网络数据包对应的候选数据包指纹组成候选数据包指纹集合。其中,时间窗的长度可以根据需要进行设置,例如,设置为5分钟,检测服务器获取在当前时刻之前的5分钟内采集到的各个网络数据包作为候选网络数据包,将各个候选网络数据包组成候选网络数据包集合。
在一个实施例中,检测服务器可以从候选网络数据包中获取候选网络数据包对应的当前层协议类型,在本地或从其他终端、服务器获取当前层协议类型对应的指纹生成算法,基于指纹生成算法从候选网络数据包中提取与该指纹生成算法匹配的特征字段,根据提取到的特征字段得到由当前层协议类型对应的指纹生成算法生成的候选数据包指纹。当前层协议类型对应的指纹生成算法可以为至少一个。若当前层协议类型对应的指纹生成算法为多个,可以生成当前层协议类型对应的多个候选数据包指纹。同理,检测服务器可以从候选网络数据包中获取候选网络数据包对应的关联层协议类型,在本地或从其他终端、服务器获取关联层协议类型对应的指纹生成算法,基于指纹生成算法从候选网络数据包中提取与该指纹生成算法匹配的特征字段,根据提取到的特征字段得到由关联层协议类型对应的指纹生成算法生成的候选数据包指纹。关联层协议类型对应的指纹生成算法可以为至少一个。当关联层协议类型对应的指纹生成算法为多个时,可以生成关联层协议类型对应的多个候选数据包指纹。可以是各个候选网络数据包的当前层协议类型对应的各个候选数据包指纹组合得到候选数据包指纹集合,也可以是各个候选网络数据包的关联层协议类型对应的各个候选数据包指纹组合得到候选数据包指纹集合,还可以是各个候选网络数据包的当前层协议类型对应的各个候选数据包指纹和关联层协议类型对应的各个候选数据包指纹组合得到候选数据包指纹集合。
步骤S406,基于指纹关联信息对候选数据包指纹集合中的候选数据包指纹进行聚类,得到聚类结果。
其中,指纹关联信息是指数据包指纹的关联信息。数据包指纹的指纹关联信息包括用于生成数据包指纹的协议类型和指纹生成算法。
具体地,检测服务器可以基于指纹关联信息对候选数据包指纹集合中的候选数据包指纹进行聚类,具体可以是将基于同一指纹关联信息生成的相同候选数据包指纹聚类在一起,得到多个不同的聚类簇。
在一个实施例中,基于指纹关联信息对候选数据包指纹集合中的候选数据包指纹进行聚类,得到聚类结果,包括:将基于同一协议类型对应的同一指纹生成算法生成的相同候选数据包指纹进行聚类得到多个不同的聚类簇,统计同一聚类簇内的候选数据包指纹的数量,得到各个聚类簇对应的统计值。
具体地,检测服务器可以将基于同一协议类型对应的同一指纹生成算法生成的相同候选数据包指纹进行聚类,得到多个不同的聚类簇。一个聚类簇表示一种类别的候选数据包指纹。检测服务器可以统计同一聚类簇内的候选数据包指纹的数量,得到各个聚类簇对应的统计值。也就是,检测服务器可以在一个时间窗内,统计在同协议类型、同指纹生成算法下生成的相同候选数据包指纹的数量,得到多个统计值。例如,统计由HTTP协议对应 的指纹生成算法1生成的候选数据包指纹a的数量,统计由HTTP协议对应的指纹生成算法2生成的候选数据包指纹b的数量,统计由SSH协议对应的指纹生成算法3生成的候选数据包指纹c的数量,统计由SSH协议对应的指纹生成算法1生成的候选数据包指纹d的数量。
步骤S408,根据聚类结果从候选数据包指纹集合中确定参考数据包指纹,各个参考数据包指纹组合得到参考数据包指纹库。
具体地,检测服务器可以根据聚类结果从候选数据包指纹集合中确定参考数据包指纹,具体可以是当聚类簇对应的统计值大于统计阈值时,将该聚类簇对应的候选数据包指纹作为参考数据包指纹,各个参考数据包指纹组合得到参考数据包指纹库。参考数据包指纹库还可以包括各个参考数据包指纹对应的指纹关联信息和统计值。
在一个实施例中,根据聚类结果从候选数据包指纹集合中确定参考数据包指纹,各个参考数据包指纹组合得到参考数据包指纹库,包括:将统计值大于统计阈值的聚类簇内的候选数据包指纹作为参考数据包指纹。
具体地,当聚类簇对应的统计值大于统计阈值时,表明该聚类簇内的候选数据包指纹在同一时间窗内频繁出现,极有可能是黑客攻击所导致的,因此检测服务器可以将该聚类簇内的候选数据包指纹作为参考数据包指纹。其中,统计阈值可以根据时间窗的大小进行设置,时间窗越大,统计阈值越大。
在一个实施例中,时间窗可以是动态变化的,那么参考数据包指纹库也是动态更新的。例如,时间窗始终保持为当前时刻之前的5分钟,那么检测服务器可以每隔5分钟对5分钟内获取到的各个候选网络数据包所对应的候选数据包指纹进行聚类,根据聚类结果对参考数据包指纹库进行更新。
可以理解,参考数据包指纹库中基于同一协议类型对应的同一指纹生成算法生成的参考数据包指纹可以有多个。例如,参考数据包指纹库包括基于HTTP协议对应的指纹生成算法fatt生成的参考数据包指纹1、基于HTTP协议对应的指纹生成算法fatt生成的参考数据包指纹2和基于HTTP协议对应的指纹生成算法fatt生成的参考数据包指纹3。
本实施例中,通过获取同一时间窗内的多个候选网络数据包,计算各个候选网络数据包对应的候选数据包指纹,基于指纹关联信息对候选数据包指纹进行聚类分析,得到各个聚类簇,计算各个聚类簇对应的统计值,将统计值大于统计阈值的聚类簇内的候选数据包指纹作为参考数据包指纹,各个参考数据包指纹组合得到参考数据包指纹库。这样,因为黑客攻击通常是一个持续性的攻击,所以对同一时间窗内的候选网络数据包对应的候选数据包指纹进行聚类分析,通过将各个聚类簇对应的统计值和统计阈值进行比较可以快速建立数据包指纹黑库。
在一个实施例中,将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配,包括:将目标数据包指纹和当前参考数据包指纹进行比较,得到数据包指纹比较结果;将目标数据包指纹对应的指纹关联信息和当前参考数据包指纹对应的指纹关联信息进行比较,得到指纹关联信息比较结果;当数据包指纹比较结果和指纹关联信息比较结果均为一致时,确定目标数据包指纹和当前参考数据包指纹匹配成功。
具体地,在将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配时,检测服务器可以将目标数据包指纹和当前参考数据包指纹进行比较,得到数据包指纹比较 结果,数据包指纹比较结果包括一致和不一致,将目标数据包指纹对应的指纹关联信息和当前参考数据包指纹对应的指纹关联信息进行比较,得到指纹关联信息比较结果,指纹关联信息比较结果包括一致和不一致。当数据包指纹比较结果和指纹关联信息比较结果均为一致时,检测服务器可以确定目标数据包指纹和当前参考数据包指纹匹配成功。
举例说明,当目标数据包指纹a和当前参考数据包指纹b一致,目标数据包指纹a对应的协议类型和当前参考数据包指纹b对应的协议类型均为HTTP协议,目标数据包指纹a对应的指纹生成算法和当前参考数据包指纹b对应的指纹生成算法均为pmercury时,确定目标数据包指纹a和当前参考数据包指纹b匹配成功。可以理解,即使目标数据包指纹a对应的网络数据包A和当前参考数据包指纹b对应的网络数据包B不一样,但是当目标数据包指纹a和当前参考数据包指纹b一致,并且用于生成目标数据包指纹a和当前参考数据包指纹b的协议类型和指纹生成算法也一致时,表明网络数据包A和网络数据包B包括相同的特征信息,极有可能是同一黑客发送的两个网络数据包。
本实施例中,通过数据包指纹比较结果和指纹关联信息可以快速确定目标数据包指纹和参考数据包指纹的匹配结果,从而有助于提高待检测网络数据包的检测效率。
在一个实施例中,如图5所示,基于匹配成功的目标数据包指纹对应的置信度关联信息计算得到待检测网络数据包对应的目标置信度,包括:
步骤S502,获取目标协议类型对应的目标协议置信度。
步骤S504,基于匹配成功的目标数据包指纹对应的指纹生成算法获取对应的目标算法置信度。
步骤S506,基于目标协议置信度和目标算法置信度得到目标置信度。
具体地,检测服务器可以获取目标协议类型对应的目标协议置信度,获取各个匹配成功的目标数据包指纹对应的指纹生成算法所对应的目标算法置信度,将目标协议置信度和目标算法置信度进行融合得到目标置信度。在进行融合时,检测服务器可以是将目标协议置信度和目标算法置信度相乘得到目标置信度,也可以是先对各个目标算法置信度进行统计得到算法置信度统计值,再将算法置信度统计值和目标协议置信度进行融合得到目标置信度。对各个目标算法置信度进行统计具体可以是将各个目标算法置信度相加得到算法置信度统计值,计算各个目标算法置信度的平均值得到算法置信度统计值,也可以是按照预设公式将各个目标算法置信度进行融合得到算法置信度统计值。
本实施例中,通过获取目标协议类型对应的目标协议置信度,基于匹配成功的目标数据包指纹对应的指纹生成算法获取对应的目标算法置信度,基于目标协议置信度和目标算法置信度得到目标置信度。这样,目标置信度的计算综合考虑了目标协议类型和匹配成功的目标数据包指纹对应的指纹生成算法,计算得到的目标置信度更准确、更可靠。
在一个实施例中,如图6所示,目标协议类型包括当前层协议类型和关联层协议类型,基于目标协议置信度和目标算法置信度得到目标置信度,包括:
步骤S602,将同一层协议类型对应的各个目标算法置信度从大到小进行排序,得到各层协议类型对应的排序结果。
步骤S604,根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度。
步骤S606,将同一层协议类型对应的排序结果中剩余的目标算法置信度和对应的第 一置信度从大到小进行排序,得到各层协议类型对应的更新的排序结果,返回所述根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度的步骤,直至同一层协议类型对应的各个目标算法置信度均参与数据处理,得到各层协议类型对应的第二置信度。
步骤S608,基于同一层协议类型对应的目标协议置信度和第二置信度得到各层协议类型对应的中间置信度,根据各个中间置信度得到目标置信度。
具体地,目标协议类型包括当前层协议类型和关联层协议类型,不同层的协议类型需要区分计算。检测服务器可以将同一层协议类型对应的各个目标算法置信度从大到小进行排序,得到各层协议类型对应的排序结果。获取同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度,将同一层协议类型对应的排序第一和排序第二的目标算法置信度进行加权求和得到各层协议类型对应的第一置信度。同一层协议类型对应的排序结果中剩余的目标算法置信度和对应的第一置信度从大到小进行排序,得到各层协议类型对应的更新排序结果,获取同一层协议类型对应的更新排序结果中排序第一和排序第二的目标算法置信度,将同一层协议类型对应的排序第一和排序第二的目标算法置信度进行加权求和得到各层协议类型对应的第一更新置信度,以此类推,直到同一层协议类型对应的各个目标算法置信度都参与计算后,将各层协议类型对应的最终计算结果作为各层协议类型对应的第二置信度。将同一层协议类型对应的目标协议置信度和第二置信度进行相乘得到各层协议类型对应的中间置信度。可以将各个中间置信度进行加权求和得到目标置信度,也可以将各个中间置信度分别作为目标置信度。
在一个实施例中,根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度,包括:获取当前排序第一的目标算法置信度和当前排序第二的目标算法置信度对应的融合权重;当前排序第二的目标算法置信度对应的融合权重是基于当前排序第一的目标算法置信度确定的;基于同一层协议类型对应的融合权重,对同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度进行融合,得到各层协议类型对应的第一置信度。
具体地,在计算第一置信度时,可以基于同一层协议类型对应的排序结果中排序第一的目标算法置信度确定排序第二的目标算法置信度对应的融合权重,例如,将当前排序第一的目标算法置信度和预设算法置信度的差值作为当前排序第二的目标算法置信度对应的融合权重。其中,预设算法置信度可以根据需要进行设置,例如,设置为1。在计算第一置信度时,同一层协议类型对应的排序结果中排序第一的目标算法置信度对应的融合权重可以为预设权重,预设权重可以根据需要进行设置,例如,设置为1。在得到各层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度对应的融合权重后,检测服务器可以基于同一层协议类型对应的融合权重,对同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度进行融合,从而得到各层协议类型对应的第一置信度。
在一个实施例中,协议置信度可以用score pro(pro i)=M i表示,其中score pro表示协议置信度,pro i表示协议类型i,M i表示协议类型i对应的协议置信度。M i的取值范围为[0,1],M i越大,对应的网络数据包的异常程度越大。算法置信度可以用score alg(<pro i,alg j>)=N k表示,其中score alg表示算法置信度,<pro i,alg j>表示协议类型i下的指纹生成算法j,N k表示协议类型i下的指纹生成算法j对应的算法置信度。N k的取值范围为[0,1], N k越大,对应的网络数据包的异常程度越大。
参考图7,<pro i,alg j,hash ij>表示目标数据包指纹数组,<l_pro i,l_alg j,l_hash ij,l_count ij>表示参考数据包指纹数组。pro i表示目标数据包指纹对应的协议类型i,alg j表示目标数据包指纹对应的指纹生成算法j,hash ij表示基于协议类型i对应的指纹生成算法j生成的目标数据包指纹,l_pro i表示参考数据包指纹对应的协议类型i,l_alg j表示参考数据包指纹对应的指纹生成算法j,l_hash ij表示基于协议类型i对应的指纹生成算法j生成的参考数据包指纹,l_count ij表示参考数据包指纹对应的统计值。根据l_hashij和hashij的比对结果可以确定各个匹配成功的目标数据包指纹,然后循环累加各个匹配成功的目标数据包指纹对应的协议类型的中间置信度,得到待检测网络数据包对应的目标置信度score。目标置信度score+=sum(M i*sumN),i++。当目标置信度score超过参考置信度mb_THR时,则认为待检测网络数据包为异常网络数据包,否则认为待检测网络数据包为正常网络数据包。当待检测网络数据包为异常网络数据包时,可以向运维人员发出告警信息,也可以阻断待检测网络数据包。其中,在生成目标置信度score时,对同一层协议类型对应的各个目标算法置信度从大到小进行逆序排序,即sort(score(<P,alg k>),greater),再按照累加公式进行累加。累加公式为sumN+=(1-sumN)*N k,k++。该累加公式可以保证当匹配成功的目标数据包指纹越多时,在同一层协议类型下累加得到的算法置信值越接近1,但是不超过1,可以保证在同一层协议类型下累加得到的算法置信值大于最大的目标算法置信度。
举例说明,score pro(http)=0.9,score alg(<http,fatt>)=0.8,score alg(<http,pmercury>)=0.6,mb_THR=0.7。其中,mb_THR表示参考置信度。
待检测网络数据包的一个目标数据包指纹和参考数据包指纹库中基于HTTP协议对应的指纹生成算法fatt生成的某个参考数据包指纹匹配,以及另一个目标数据包指纹和参考数据包指纹库中基于HTTP协议对应的指纹生成算法pmercury生成的某个参考数据包指纹匹配。那么,待检测网络数据包对应的目标置信度score=0.9*[0.8+(1-0.8)*0.6]=0.828>0.7,因此确定待检测网络数据包为异常网络数据包。可以理解,由于待检测网络数据包的目标数据包指纹和参考数据包指纹库中基于TCP协议生成的参考数据包指纹不匹配,因此可以不需要计算TCP协议对应的中间置信度,直接将HTTP协议对应的中间置信度作为待检测网络数据包对应的目标置信度。
在一个实施例中,基于参考置信度和目标置信度确定待检测网络数据包的异常检测结果,包括:当目标置信度大于参考置信度时,确定异常检测结果为网络数据包异常;当目标置信度小于或等于参考置信度时,确定异常检测结果为网络数据包正常。
具体地,参考置信度可以是一个综合的置信度阈值。也可以是一个协议类型对应一个置信度阈值,即存在多个参考置信度。当目标置信度大于参考置信度时,确定异常检测结果为网络数据包异常,当目标置信度小于或等于参考置信度时,确定异常检测结果为网络数据包正常。不同的协议类型对应的置信度阈值可以相同可以不同。例如,HTTP协议对应的置信度阈值可以为0.7,TCP协议对应的置信度阈值可以为0.65。
本实施例中,通过比较目标置信度和参考置信度,可以快速确定待检测网络数据包的异常检测结果。
在一个实施例中,所述方法还包括:当异常检测结果为网络数据包异常时,获取阻断 数据包;将阻断数据包发送至待检测网络数据包对应的接收方,以使接收方停止建立与待检测网络数据包对应的发送方的通信连接。
具体地,当待检测网络数据包的异常检测结果为网络数据包异常时,检测服务器可以获取阻断数据包,将阻断数据包发送至待检测网络数据包对应的接收方。检测服务器可以从待检测网络数据包中获取发送方的相关信息,将发送方的相关信息携带至阻断数据包中,那么当接受方接收到该阻断数据包后,可以主动停止建立与该发送方的通信连接,以此来阻断待检测网络数据包。
在一个实施例中,阻断数据包可以是RST(Reset)包。RST包用于断开通信双方的通信连接。
本实施例中,当待检测网络数据包的异常检测结果为网络数据包异常时,将阻断数据包发送至待检测网络数据包对应的接收方,可以快速断开待检测网络数据包对应的发送方和接受方的通信连接,有效阻止黑客攻击。
在一个实施例中,所述方法还包括:定时统计各个参考数据包指纹对应的匹配成功率;将匹配成功率小于预设阈值的参考数据包指纹从参考数据包指纹库中滤除。
具体地,为了提高参考数据包指纹库的空间利用率,可以定时对参考数据包指纹库中的参考数据包指纹进行过滤,滤除长期未匹配到的参考数据包指纹。可以理解,若参考数据包指纹长期未匹配成功,则可以认为该参考数据包指纹对应的黑客攻击已经停止。因为参考数据包指纹库是不断扩充的,所以可以将长期未匹配到的参考数据包指纹滤除,以此来提高参考数据包指纹库的空间利用率。检测服务器可以定时统计参考数据包指纹库中各个参考数据包指纹对应的匹配成功率,将匹配成功率小于预设阈值的参考数据包指纹从参考数据包指纹库中滤除。检测服务器具体可以根据参考数据包指纹的匹配成功次数和参考数据包指纹在参考数据包指纹库中存储时间得到参考数据包指纹对应的匹配成功率。不同的存储时间对应不同的参考匹配次数,将匹配成功次数和参考匹配次数的比值作为匹配成功率。其中,预设阈值可以根据实际需要进行设置。
本申请还提供一种应用场景,该应用场景应用上述的异常网络数据检测方法。具体地,该异常网络数据检测方法在该应用场景的应用如下:
该异常网络数据检测方法可以用于秒拨攻击检测。秒拨是指利用国内家用宽带拨号上网的原理,每一次断线重连就会获取一个新的IP,实现按秒拨号的IP跳变。黑客可以利用秒拨技术攻击计算机设备,以窃取核心数据。
如图8所示,正常用户和秒拨用户都可以通过终端向业务服务器发送网络数据包。终端可以利用交换机和业务服务器进行通信。检测服务器可以从交换机上通过旁路引流的方式获取同一时间窗内的候选网络数据包,生成各个候选网络数据包分别对应的候选数据包指纹,根据候选数据包指纹建立秒拨黑库。检测服务器可以从交换机上通过旁路引流的方式获取目标网络数据包(实时流量),生成目标网络数据包对应的目标数据包指纹,将目标数据包指纹和参考数据包指纹进行匹配,计算匹配成功的目标数据包指纹对应的目标置信度,根据目标置信度和置信度阈值的比较结果来识别秒拨IP。当目标置信度大于置信度阈值时,确定目标网络数据包为异常网络数据包,确定匹配到秒拨IP攻击,进行告警,实时阻断异常网络数据包。当目标置信度小于或等于置信度阈值时,确定目标网络数据包为正常网络数据包。
如图9所示,检测服务器识别秒拨攻击的具体过程如下:
1、建立秒拨黑库。
1-1、从交换机中采集同一时间窗内的多个候选网络数据包(即在时间窗内统计流量)。
1-2、生成各个候选网络数据包分别对应的至少一个候选数据包指纹,各个候选数据包指纹组合得到候选数据包指纹集合。
具体地,获取候选网络数据包对应的当前层协议类型和关联层协议类型,基于当前层协议类型对应的至少一个指纹生成算法从候选网络数据包中获取匹配的特征字段,根据特征字段生成对应的候选数据包指纹,基于关联层协议类型对应的至少一个指纹生成算法从候选网络数据包中获取匹配的特征字段,根据特征字段生成对应的候选数据包指纹,各个候选网络数据包对应的多个候选数据包指纹组合得到候选数据包指纹集合。
举例说明,HTTP协议网络数据包对应的当前层协议类型为HTTP协议,对应的关联层协议类型为TCP协议,HTTP协议对应的指纹生成算法包括fatt和pmercury,TCP协议对应的指纹生成算法包括pmercury。基于HTTP协议对应的指纹生成算法fatt从候选网络数据包中获取匹配的特征字段,根据特征字段生成对应的候选数据包指纹hash1。基于HTTP协议对应的指纹生成算法pmercury从候选网络数据包中获取匹配的特征字段,根据特征字段生成对应的候选数据包指纹hash2。基于TCP协议对应的指纹生成算法pmercury从候选网络数据包中获取匹配的特征字段,根据特征字段生成对应的候选数据包指纹hash3。HTTP协议网络数据包对应的候选数据包指纹包括候选数据包指纹hash1、候选数据包指纹hash2和候选数据包指纹hash3。
1-3、对候选数据包指纹进行聚类,根据聚类结果建立秒拨黑库。
具体地,基于同一协议类型对应的同一指纹生成算法生成的相同候选数据包指纹进行聚类得到多个不同的聚类簇,统计同一聚类簇内的候选数据包指纹的数量,得到各个聚类簇对应的统计值。将统计值大于统计阈值的聚类簇内的候选数据包指纹作为参考数据包指纹,各个参考数据包指纹组合得到秒拨黑库(即参考数据包指纹库)。
举例说明,假设统计阈值为50。在候选数据包指纹库中,若基于HTTP协议对应的指纹生成算法fatt生成的候选数据包指纹hash1的总数为55>50,则可以将候选数据包指纹hash1作为参考数据包指纹加入秒拨黑库。秒拨黑库可以按照<HTTP协议,指纹生成算法fatt,hash1,55>这样的数组存储候选数据包指纹hash1。
2、从交换机中采集实时流量,基于秒拨黑库对实时流量进行秒拨攻击检测。
2-1、生成待检测网络数据包(即实时流量)对应的至少一个目标数据包指纹,各个目标数据包指纹组合得到目标数据包指纹集合。
具体地,获取待检测网络数据包(即实时流量)对应的当前层协议类型和关联层协议类型,基于当前层协议类型对应的至少一个指纹生成算法从待检测网络数据包中获取匹配的特征字段,根据特征字段生成对应的目标数据包指纹。基于关联层协议类型对应的至少一个指纹生成算法从待检测网络数据包中获取匹配的特征字段,根据特征字段生成对应的目标数据包指纹,各个目标数据包指纹组合得到待检测网络数据包对应的目标数据包指纹集合。
2-2、目标数据包指纹和秒拨黑库中的参考数据包指纹进行匹配
具体地,将目标数据包指纹和秒拨黑库中的参考数据包指纹进行匹配。当目标数据包 指纹和参考数据包指纹一致,目标数据包指纹对应的协议类型和参考数据包指纹对应的协议类型一致,目标数据包指纹对应的指纹生成算法和参考数据包指纹对应的指纹生成算法一致时,确定该目标数据包指纹和该参考数据包指纹匹配成功。
2-3、基于匹配成功的各个目标数据包指纹计算待检测网络数据包对应的目标置信度。
具体地,基于匹配成功的各个目标数据包指纹对应的指纹生成算法获取对应的目标算法置信度,将同一层协议类型对应的各个目标算法置信度从大到小进行排序,按照排序顺序,根据累加公式将同一层协议类型对应的各个目标算法置信度累加得到各层协议类型对应的算法置信度统计值(即第二置信度)。将同一层协议类型对应的目标协议置信度和算法置信度统计值进行相乘得到各层协议类型对应的中间置信度,将各个中间置信度进行相加得到目标置信度。
2-4、获取参考置信度,根据目标置信度和参考置信度的比较结果判断是否出现秒拨攻击。
具体地,获取相应的参考置信度(即置信度阈值),当目标置信度大于参考置信度时,确定待检测网络数据包的异常检测结果为网络数据包异常,当目标置信度小于或等于参考置信度时,确定待检测网络数据包的异常检测结果为网络数据包正常。当待检测网络数据包的异常检测结果为网络数据包异常时,则判断出现秒拨攻击,检测服务器可以向运维人员对应的终端发送告警信息,或者直接阻断该待检测网络数据包。
举例说明,score pro(http)=0.9,score alg(<http,fatt>)=0.8,score alg(<http,pmercury>)=0.6,mb_THR(http)=0.7。score pro(tcp)=0.7,score alg(<tcp,pmercury>)=0.7,mb_THR(tcp)=0.45。
待检测网络数据包的一个目标数据包指纹和参考数据包指纹库中基于HTTP协议对应的指纹生成算法fatt生成的某个参考数据包指纹匹配,另一个目标数据包指纹和参考数据包指纹库中基于HTTP协议对应的指纹生成算法pmercury生成的某个参考数据包指纹匹配,另一个目标数据包指纹和参考数据包指纹库中基于HTTP协议对应的指纹生成算法pmercury生成的某个参考数据包指纹匹配。那么,score(http)=0.9*[0.8+(1-0.8)*0.6]=0.828>0.7,score(tcp)=0.7*0.7=0.49>0.45,因此确定待检测网络数据包为异常网络数据包。
上述异常网络数据检测方法,规避了传统检测手段对于IP的依赖,将关注重点放在了请求内容、客户端甚至攻击者本身,也就是网络数据包的特征信息,从而可以降低秒拨攻击的误判率,提高秒拨攻击的检测准确性。此外,同一网络数据包对应的数据包指纹集合包括多种协议类型、多种指纹生成算法对应的数据包指纹,丰富了数据包指纹,可以覆盖更多的攻击类型,可以进一步提高秒拨攻击的检测准确性。
应该理解的是,虽然图2、4、5、6的流程图中的各个步骤按照箭头的指示依次显示,但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,图2、4、5、6中的至少一部分步骤可以包括多个步骤或者多个阶段,这些步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者 交替地执行。
在一个实施例中,如图10所示,提供了一种异常网络数据检测装置,该装置可以采用软件模块或硬件模块,或者是二者的结合成为计算机设备的一部分,该装置具体包括:数据包获取模块1002、数据包指纹生成模块1004、数据包指纹匹配模块1006、置信度计算模块1008和检测结果确定模块1010,其中:
数据包获取模块1002,用于获取待检测网络数据包;
数据包指纹生成模块1004,用于根据待检测网络数据包对应的目标协议类型对待检测网络数据包提取特征,生成对应的目标数据包指纹集合;目标数据包指纹集合包括至少一个目标数据包指纹;
数据包指纹匹配模块1006,用于将目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配;
置信度计算模块1008,用于基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到待检测网络数据包对应的目标置信度;
检测结果确定模块1010,用于获取参考置信度,基于参考置信度和所述目标置信度确定待检测网络数据包的异常检测结果。
在一个实施例中,如图11所示,所述装置还包括:
参考数据包指纹库建立模块1001,用于获取候选网络数据包集合;候选网络数据包集合包括在同一时间窗内的多个候选网络数据包;根据候选网络数据包对应的候选协议类型对候选网络数据包提取特征,生成对应的候选数据包指纹集合,候选数据包指纹集合包括各个候选网络数据包对应的候选数据包指纹;基于指纹关联信息对候选数据包指纹集合中的候选数据包指纹进行聚类,得到聚类结果;根据聚类结果从候选数据包指纹集合中确定参考数据包指纹,各个参考数据包指纹组合得到参考数据包指纹库。
在一个实施例中,当前网络数据包为待检测网络数据包,数据包指纹生成模块还用于获取当前网络数据包对应的当前协议类型,当前协议类型包括当前层协议类型和关联层协议类型;获取当前协议类型对应的至少一个指纹生成算法;基于指纹生成算法从当前网络数据包中提取匹配的特征字段,根据特征字段得到当前协议类型对应的各个指纹生成算法生成的当前数据包指纹;各个当前数据包指纹组合得到当前网络数据包对应的数据包指纹集合。
在一个实施例中,当前网络数据包为候选网络数据包,参考数据包指纹库建立模块还用于获取当前网络数据包对应的当前协议类型,当前协议类型包括当前层协议类型和关联层协议类型;获取当前协议类型对应的至少一个指纹生成算法;基于指纹生成算法从当前网络数据包中提取匹配的特征字段,根据特征字段得到当前协议类型对应的各个指纹生成算法生成的当前数据包指纹;各个当前数据包指纹组合得到当前网络数据包对应的数据包指纹集合。
在一个实施例中,指纹关联信息包括候选数据包指纹对应的候选协议类型和指纹生成算法,参考数据包指纹库建立模块还用于将基于同一协议类型对应的同一指纹生成算法生成的相同候选数据包指纹进行聚类得到多个不同的聚类簇,统计同一聚类簇内的候选数据包指纹的数量,得到各个聚类簇对应的统计值。参考数据包指纹库建立模块还用于将统计 值大于统计阈值的聚类簇内的候选数据包指纹作为参考数据包指纹。
在一个实施例中,数据包指纹匹配模块还用于将目标数据包指纹和当前参考数据包指纹进行比较,得到数据包指纹比较结果;将目标数据包指纹对应的指纹关联信息和当前参考数据包指纹对应的指纹关联信息进行比较,得到指纹关联信息比较结果;当数据包指纹比较结果和指纹关联信息比较结果均为一致时,确定目标数据包指纹和当前参考数据包指纹匹配成功。
在一个实施例中,置信度计算模块还用于获取目标协议类型对应的目标协议置信度;基于匹配成功的目标数据包指纹对应的指纹生成算法获取对应的目标算法置信度;基于目标协议置信度和目标算法置信度得到目标置信度。
在一个实施例中,目标协议类型包括当前层协议类型和关联层协议类型,置信度计算模块还用于将同一层协议类型对应的各个目标算法置信度从大到小进行排序,得到各层协议类型对应的排序结果;根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度;将同一层协议类型对应的排序结果中剩余的目标算法置信度和对应的第一置信度从大到小进行排序,得到各层协议类型对应的更新的排序结果,返回根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度的步骤,直至同一层协议类型对应的各个目标算法置信度均参与数据处理,得到各层协议类型对应的第二置信度;基于同一层协议类型对应的目标协议置信度和第二置信度得到各层协议类型对应的中间置信度,根据各个中间置信度得到目标置信度。
在一个实施例中,置信度计算模块还用于获取当前排序第一的目标算法置信度和当前排序第二的目标算法置信度对应的融合权重,当前排序第二的目标算法置信度对应的融合权重是基于当前排序第一的目标算法置信度确定的,基于同一层协议类型对应的融合权重,对同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度进行融合,得到各层协议类型对应的第一置信度。
在一个实施例中,检测结果确定模块还用于当目标置信度大于参考置信度时,确定异常检测结果为网络数据包异常;当目标置信度小于或等于参考置信度时,确定异常检测结果为网络数据包正常。
在一个实施例中,如图11所示,所述装置还包括:
通信连接阻断模块1011,用于当异常检测结果为网络数据包异常时,获取阻断数据包;将阻断数据包发送至待检测网络数据包对应的接收方,以使接收方停止建立与待检测网络数据包对应的发送方的通信连接。
在一个实施例中,如图11所示,所述装置还包括:
参考数据包指纹库更新模块1012,用于定时统计各个参考数据包指纹对应的匹配成功率;将匹配成功率小于预设阈值的参考数据包指纹从参考数据包指纹库中滤除。
上述异常网络数据检测装置,生成的目标数据包指纹可以表征待检测网络数据包的特征信息,多个目标数据包指纹可以从不同维度表征待检测网络数据包的特征信息,从而丰富了异常网络数据包的检测维度,提高了异常网络数据包的检测准确性。进一步的,参考数据包指纹库集成了多个异常的参考数据包指纹,因此匹配成功的目标数据包指纹可以表征待检测网络数据包异常的特征信息,基于匹配成功的目标数据包指纹对应的置信度关联 信息计算得到的目标置信度可以表征待检测网络数据包的异常程度,基于参考置信度和目标置信度可以快速确定待检测网络数据包的异常检测结果,提高了异常网络数据包的检测准确性和检测效率。
关于异常网络数据检测装置的具体限定可以参见上文中对于异常网络数据检测方法的限定,在此不再赘述。上述异常网络数据检测装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中,也可以以软件形式存储于计算机设备中的存储器中,以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中,提供了一种计算机设备,该计算机设备可以是服务器,其内部结构图可以如图12所示。该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中,该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储参考数据包指纹、协议置信度、算法置信度、指纹生成算法等数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令被处理器执行时以实现一种异常网络数据检测方法。
本领域技术人员可以理解,图12中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。
在一个实施例中,还提供了一种计算机设备,包括存储器和一个或多个处理器,存储器中存储有计算机可读指令,该一个或多个处理器执行计算机可读指令时实现上述各方法实施例中的步骤。
在一个实施例中,提供了一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,存储有计算机可读指令,该计算机可读指令被一个或多个处理器执行时实现上述各方法实施例中的步骤。
在一个实施例中,提供了一种计算机程序产品或计算机程序,该计算机程序产品或计算机程序包括计算机可读指令,该计算机可读指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机可读指令,处理器执行该计算机可读指令,使得该计算机设备执行上述各方法实施例中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机可读指令来指令相关的硬件来完成,所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中,该计算机可读指令在执行时,可包括如上述各方法的实施例的流程。其中,本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用,均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory,ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory,RAM)或外部高速缓冲存储器。作为说明而非局限,RAM可以是多种形式,比如静态随机存取存储器(Static Random Access Memory, SRAM)或动态随机存取存储器(Dynamic Random Access Memory,DRAM)等。
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对发明专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。

Claims (20)

  1. 一种异常网络数据检测方法,其特征在于,由检测服务器执行,所述方法包括:
    获取待检测网络数据包;
    根据所述待检测网络数据包对应的目标协议类型对所述待检测网络数据包提取特征,生成对应的目标数据包指纹集合;所述目标数据包指纹集合包括至少一个目标数据包指纹;
    将所述目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配;
    基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到所述待检测网络数据包对应的目标置信度;及
    获取参考置信度,基于所述参考置信度和所述目标置信度确定所述待检测网络数据包的异常检测结果。
  2. 根据权利要求1所述的方法,其特征在于,所述获取待检测网络数据包之前,所述方法还包括:
    获取候选网络数据包集合;所述候选网络数据包集合包括在同一时间窗内的多个候选网络数据包;
    根据所述候选网络数据包对应的候选协议类型对所述候选网络数据包提取特征,生成对应的候选数据包指纹集合,所述候选数据包指纹集合包括各个候选网络数据包对应的候选数据包指纹;
    基于指纹关联信息对所述候选数据包指纹集合中的候选数据包指纹进行聚类,得到聚类结果;及
    根据所述聚类结果从所述候选数据包指纹集合中确定参考数据包指纹,各个参考数据包指纹组合得到所述参考数据包指纹库。
  3. 根据权利要求2所述的方法,其特征在于,当前网络数据包为所述待检测网络数据包或所述候选网络数据包,所述当前网络数据包对应的数据包指纹集合的生成,包括以下步骤:
    获取所述当前网络数据包对应的当前协议类型,所述当前协议类型包括当前层协议类型和关联层协议类型;
    获取所述当前协议类型对应的至少一个指纹生成算法;
    基于所述指纹生成算法从所述当前网络数据包中提取匹配的特征字段,根据所述特征字段得到所述当前协议类型对应的各个指纹生成算法生成的当前数据包指纹;及
    各个当前数据包指纹组合得到所述当前网络数据包对应的数据包指纹集合。
  4. 根据权利要求2所述的方法,其特征在于,所述指纹关联信息包括候选数据包指纹对应的候选协议类型和指纹生成算法,所述基于指纹关联信息对所述候选数据包指纹集合中的候选数据包指纹进行聚类,得到聚类结果,包括:
    将基于同一协议类型对应的同一指纹生成算法生成的相同候选数据包指纹进行聚类得到多个不同的聚类簇,统计同一聚类簇内的候选数据包指纹的数量,得到各个聚类簇对应的统计值;
    所述根据聚类结果从所述候选数据包指纹集合中确定参考数据包指纹,各个参考数据包指纹组合得到所述参考数据包指纹库,包括:
    将所述统计值大于统计阈值的聚类簇内的候选数据包指纹作为所述参考数据包指纹。
  5. 根据权利要求1所述的方法,其特征在于,所述将所述目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配,包括:
    将所述目标数据包指纹和当前参考数据包指纹进行比较,得到数据包指纹比较结果;
    将所述目标数据包指纹对应的指纹关联信息和所述当前参考数据包指纹对应的指纹关联信息进行比较,得到指纹关联信息比较结果;及
    当所述数据包指纹比较结果和所述指纹关联信息比较结果均为一致时,确定所述目标数据包指纹和所述当前参考数据包指纹匹配成功。
  6. 根据权利要求1所述的方法,其特征在于,所述基于匹配成功的目标数据包指纹对应的置信度关联信息计算得到所述待检测网络数据包对应的目标置信度,包括:
    获取所述目标协议类型对应的目标协议置信度;
    基于匹配成功的目标数据包指纹对应的指纹生成算法获取对应的目标算法置信度;及
    基于所述目标协议置信度和所述目标算法置信度得到所述目标置信度。
  7. 根据权利要求6所述的方法,其特征在于,所述目标协议类型包括当前层协议类型和关联层协议类型,所述基于所述目标协议置信度和所述目标算法置信度得到所述目标置信度,包括:
    将同一层协议类型对应的各个目标算法置信度从大到小进行排序,得到各层协议类型对应的排序结果;
    根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度;将同一层协议类型对应的排序结果中剩余的目标算法置信度和对应的第一置信度从大到小进行排序,得到各层协议类型对应的更新的排序结果,返回所述根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度的步骤,直至同一层协议类型对应的各个目标算法置信度均参与数据处理,得到各层协议类型对应的第二置信度;及
    基于同一层协议类型对应的目标协议置信度和第二置信度得到各层协议类型对应的中间置信度,根据各个中间置信度得到所述目标置信度。
  8. 根据权利要求7所述的方法,其特征在于,所述根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度,包括:
    获取当前排序第一的目标算法置信度和当前排序第二的目标算法置信度对应的融合权重;所述当前排序第二的目标算法置信度对应的融合权重是基于所述当前排序第一的目标算法置信度确定的;及
    基于同一层协议类型对应的融合权重,对同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度进行融合,得到各层协议类型对应的第一置信度。
  9. 根据权利要求1所述的方法,其特征在于,所述基于所述参考置信度和所述目标置信度确定所述待检测网络数据包的异常检测结果,包括:
    当所述目标置信度大于所述参考置信度时,确定所述异常检测结果为网络数据包异常;及
    当所述目标置信度小于或等于所述参考置信度时,确定所述异常检测结果为网络数据包正常。
  10. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    当所述异常检测结果为网络数据包异常时,获取阻断数据包;及
    将所述阻断数据包发送至所述待检测网络数据包对应的接收方,以使所述接收方停止建立与所述待检测网络数据包对应的发送方的通信连接。
  11. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    定时统计各个参考数据包指纹对应的匹配成功率;及
    将所述匹配成功率小于预设阈值的参考数据包指纹从所述参考数据包指纹库中滤除。
  12. 一种异常网络数据检测装置,其特征在于,所述装置包括:
    数据包获取模块,用于获取待检测网络数据包;
    数据包指纹生成模块,用于根据所述待检测网络数据包对应的目标协议类型对所述待检测网络数据包提取特征,生成对应的目标数据包指纹集合;所述目标数据包指纹集合包括至少一个目标数据包指纹;
    数据包指纹匹配模块,用于将所述目标数据包指纹和参考数据包指纹库中的参考数据包指纹进行匹配;
    置信度计算模块,用于基于匹配成功的目标数据包指纹对应的置信度关联信息,计算得到所述待检测网络数据包对应的目标置信度;及
    检测结果确定模块,用于获取参考置信度,基于所述参考置信度和所述目标置信度确定所述待检测网络数据包的异常检测结果。
  13. 根据权利要求12所述的装置,其特征在于,所述装置还包括:
    参考数据包指纹库建立模块,用于获取候选网络数据包集合;所述候选网络数据包集合包括在同一时间窗内的多个候选网络数据包;根据候选网络数据包对应的候选协议类型对候选网络数据包提取特征,生成对应的候选数据包指纹集合,所述候选数据包指纹集合包括各个候选网络数据包对应的候选数据包指纹;基于指纹关联信息对所述候选数据包指纹集合中的候选数据包指纹进行聚类,得到聚类结果;根据聚类结果从所述候选数据包指纹集合中确定参考数据包指纹,各个参考数据包指纹组合得到所述参考数据包指纹库。
  14. 根据权利要求13所述的装置,其特征在于,所述指纹关联信息包括候选数据包指纹对应的候选协议类型和指纹生成算法,所述参考数据包指纹库建立模块还用于将基于同一协议类型对应的同一指纹生成算法生成的相同候选数据包指纹进行聚类得到多个不同的聚类簇,统计同一聚类簇内的候选数据包指纹的数量,得到各个聚类簇对应的统计值;所述参考数据包指纹库建立模块还用于将统计值大于统计阈值的聚类簇内的候选数据包指纹作为所述参考数据包指纹。
  15. 根据权利要求12所述的装置,其特征在于,所述数据包指纹匹配模块还用于将所述目标数据包指纹和当前参考数据包指纹进行比较,得到数据包指纹比较结果,将所述目标数据包指纹对应的指纹关联信息和所述当前参考数据包指纹对应的指纹关联信息进行比较,得到指纹关联信息比较结果,当所述数据包指纹比较结果和所述指纹关联信息比较结果均为一致时,确定所述目标数据包指纹和所述当前参考数据包指纹匹配成功。
  16. 根据权利要求12所述的装置,其特征在于,所述置信度计算模块还用于获取所述目标协议类型对应的目标协议置信度,基于匹配成功的目标数据包指纹对应的指纹生成算法获取对应的目标算法置信度,基于所述目标协议置信度和所述目标算法置信度得到所述目标置信度。
  17. 根据权利要求16所述的装置,其特征在于,所述目标协议类型包括当前层协议类型和关联层协议类型,所述置信度计算模块还用于将同一层协议类型对应的各个目标算法置信度从大到小进行排序,得到各层协议类型对应的排序结果,根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度,将同一层协议类型对应的排序结果中剩余的目标算法置信度和对应的第一置信度从大到小进行排序,得到各层协议类型对应的更新的排序结果,返回所述根据同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度得到各层协议类型对应的第一置信度的步骤,直至同一层协议类型对应的各个目标算法置信度均参与数据处理,得到各层协议类型对应的第二置信度,基于同一层协议类型对应的目标协议置信度和第二置信度得到各层协议类型对应的中间置信度,根据各个中间置信度得到所述目标置信度。
  18. 根据权利要求17所述的装置,其特征在于,所述置信度计算模块还用于获取当前排序第一的目标算法置信度和当前排序第二的目标算法置信度对应的融合权重,所述当前排序第二的目标算法置信度对应的融合权重是基于所述当前排序第一的目标算法置信度确定的,基于同一层协议类型对应的融合权重,对同一层协议类型对应的排序结果中排序第一和排序第二的目标算法置信度进行融合,得到各层协议类型对应的第一置信度。
  19. 一种计算机设备,包括存储器和一个或多个处理器,所述存储器存储有计算机可读指令,其特征在于,所述一个或多个处理器执行所述计算机可读指令时实现权利要求1至11中任一项所述的方法的步骤。
  20. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,存储有计算机可读指令,其特征在于,所述计算机可读指令被一个或多个处理器执行时实现权利要求1至11中任一项所述的方法的步骤。
PCT/CN2021/117975 2020-10-22 2021-09-13 异常网络数据检测方法、装置、计算机设备和存储介质 WO2022083353A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/994,944 US20230089187A1 (en) 2020-10-22 2022-11-28 Detecting abnormal packet traffic using fingerprints for plural protocol types

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011136505.2A CN112019574B (zh) 2020-10-22 2020-10-22 异常网络数据检测方法、装置、计算机设备和存储介质
CN202011136505.2 2020-10-22

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/994,944 Continuation US20230089187A1 (en) 2020-10-22 2022-11-28 Detecting abnormal packet traffic using fingerprints for plural protocol types

Publications (1)

Publication Number Publication Date
WO2022083353A1 true WO2022083353A1 (zh) 2022-04-28

Family

ID=73528001

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/117975 WO2022083353A1 (zh) 2020-10-22 2021-09-13 异常网络数据检测方法、装置、计算机设备和存储介质

Country Status (3)

Country Link
US (1) US20230089187A1 (zh)
CN (1) CN112019574B (zh)
WO (1) WO2022083353A1 (zh)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112019574B (zh) * 2020-10-22 2021-01-29 腾讯科技(深圳)有限公司 异常网络数据检测方法、装置、计算机设备和存储介质
CN112019575B (zh) * 2020-10-22 2021-01-29 腾讯科技(深圳)有限公司 数据包处理方法、装置、计算机设备以及存储介质
CN112468520B (zh) * 2021-01-28 2021-04-20 腾讯科技(深圳)有限公司 一种数据检测方法、装置、设备及可读存储介质
CN112468518B (zh) * 2021-01-28 2021-04-20 腾讯科技(深圳)有限公司 访问数据处理方法、装置、存储介质及计算机设备
CN113037586B (zh) * 2021-02-26 2022-08-19 清华大学 通用且鲁棒的智能家居设备事件指纹提取方法和装置
US11880431B2 (en) * 2021-04-08 2024-01-23 Hewlett Packard Enterprise Development Lp System and method of classifying data and providing an accuracy of classification
CN113132406B (zh) * 2021-04-29 2022-06-07 山东云天安全技术有限公司 一种基于ssh流量发现网络威胁的检测方法、设备及介质
CN113315678A (zh) * 2021-05-26 2021-08-27 深圳市纽创信安科技开发有限公司 加密tcp流量采集方法与装置
CN113595240B (zh) * 2021-06-21 2024-01-19 深圳供电局有限公司 电力数据的检测方法、装置、设备及存储介质
CN114006750B (zh) * 2021-10-29 2024-05-28 北京顶象技术有限公司 异常操作的检测方法、装置和电子设备
CN114157502B (zh) * 2021-12-08 2023-10-27 北京恒安嘉新安全技术有限公司 一种终端识别方法、装置、电子设备及存储介质
CN114650167B (zh) * 2022-02-08 2023-06-27 联想(北京)有限公司 一种异常检测方法、装置、设备及计算机可读存储介质
CN115766204B (zh) * 2022-11-14 2024-04-26 电子科技大学 一种针对加密流量的动态ip设备标识系统及方法
CN116894011A (zh) * 2023-07-17 2023-10-17 上海螣龙科技有限公司 多维度智能化指纹库及多维度智能化指纹库设计和查询方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090290764A1 (en) * 2008-05-23 2009-11-26 Fiebrink Rebecca A System and Method for Media Fingerprint Indexing
CN108833437A (zh) * 2018-07-05 2018-11-16 成都康乔电子有限责任公司 一种基于流量指纹和通信特征匹配的apt检测方法
CN111291070A (zh) * 2020-01-20 2020-06-16 南京星环智能科技有限公司 一种异常sql检测方法、设备及介质
CN112019574A (zh) * 2020-10-22 2020-12-01 腾讯科技(深圳)有限公司 异常网络数据检测方法、装置、计算机设备和存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104348811B (zh) * 2013-08-05 2018-01-26 深圳市腾讯计算机系统有限公司 分布式拒绝服务攻击检测方法及装置
US10320825B2 (en) * 2015-05-27 2019-06-11 Cisco Technology, Inc. Fingerprint merging and risk level evaluation for network anomaly detection
CN106559261A (zh) * 2016-11-03 2017-04-05 国网江西省电力公司电力科学研究院 一种基于特征指纹的变电站网络入侵检测与分析方法
CN106790050B (zh) * 2016-12-19 2019-11-19 北京启明星辰信息安全技术有限公司 一种异常流量检测方法及检测系统
CN108200015A (zh) * 2017-12-18 2018-06-22 北京天融信网络安全技术有限公司 一种检测异常流量的方法、分类模型的构建方法及设备
CN108650218B (zh) * 2018-03-22 2019-10-08 平安科技(深圳)有限公司 网络流量监测方法、装置、计算机设备及存储介质
CN110958225B (zh) * 2019-11-08 2022-02-15 杭州安恒信息技术股份有限公司 基于流量识别网站指纹的方法
CN111181923A (zh) * 2019-12-10 2020-05-19 中移(杭州)信息技术有限公司 流量检测方法、装置、电子设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090290764A1 (en) * 2008-05-23 2009-11-26 Fiebrink Rebecca A System and Method for Media Fingerprint Indexing
CN108833437A (zh) * 2018-07-05 2018-11-16 成都康乔电子有限责任公司 一种基于流量指纹和通信特征匹配的apt检测方法
CN111291070A (zh) * 2020-01-20 2020-06-16 南京星环智能科技有限公司 一种异常sql检测方法、设备及介质
CN112019574A (zh) * 2020-10-22 2020-12-01 腾讯科技(深圳)有限公司 异常网络数据检测方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN112019574B (zh) 2021-01-29
US20230089187A1 (en) 2023-03-23
CN112019574A (zh) 2020-12-01

Similar Documents

Publication Publication Date Title
WO2022083353A1 (zh) 异常网络数据检测方法、装置、计算机设备和存储介质
Xu et al. Am I eclipsed? A smart detector of eclipse attacks for Ethereum
Lima Filho et al. Smart detection: an online approach for DoS/DDoS attack detection using machine learning
US20200344246A1 (en) Apparatus, system and method for identifying and mitigating malicious network threats
Viegas et al. Toward a reliable anomaly-based intrusion detection in real-world environments
Wani et al. SDN‐based intrusion detection system for IoT using deep learning classifier (IDSIoT‐SDL)
Yang et al. DDoS attack identification and defense using SDN based on machine learning method
CN111935170B (zh) 一种网络异常流量检测方法、装置及设备
WO2022083417A1 (zh) 一种数据包处理方法、装置、电子设备、计算机可读存储介质以及计算机程序产品
CN109194680B (zh) 一种网络攻击识别方法、装置及设备
US9473380B1 (en) Automatic parsing of binary-based application protocols using network traffic
US20120011590A1 (en) Systems, methods and devices for providing situational awareness, mitigation, risk analysis of assets, applications and infrastructure in the internet and cloud
EP3304823A1 (en) Method and apparatus for computing cell density based rareness for use in anomaly detection
CN111565203B (zh) 业务请求的防护方法、装置、系统和计算机设备
Chen et al. FCM technique for efficient intrusion detection system for wireless networks in cloud environment
Fei et al. The abnormal detection for network traffic of power iot based on device portrait
Sharma et al. Evaluation of centralised vs distributed collaborative intrusion detection systems in multi-access edge computing
US20220263846A1 (en) METHODS FOR DETECTING A CYBERATTACK ON AN ELECTRONIC DEVICE, METHOD FOR OBTAINING A SUPERVISED RANDOM FOREST MODEL FOR DETECTING A DDoS ATTACK OR A BRUTE FORCE ATTACK, AND ELECTRONIC DEVICE CONFIGURED TO DETECT A CYBERATTACK ON ITSELF
CN115499230A (zh) 网络攻击检测方法和装置、设备及存储介质
Dai et al. Eclipse attack detection for blockchain network layer based on deep feature extraction
Luxemburk et al. Detection of https brute-force attacks with packet-level feature set
Dinh et al. Dynamic economic-denial-of-sustainability (EDoS) detection in SDN-based cloud
CN111163114A (zh) 用于检测网络攻击的方法和设备
Fenil et al. Towards a secure software defined network with adaptive mitigation of dDoS attacks by machine learning approaches
Resende et al. HTTP and contact‐based features for Botnet detection

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 05/09/2023)

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21881772

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21881772

Country of ref document: EP

Kind code of ref document: A1