WO2024007615A1 - 模型训练方法、装置及相关设备 - Google Patents

模型训练方法、装置及相关设备 Download PDF

Info

Publication number
WO2024007615A1
WO2024007615A1 PCT/CN2023/081264 CN2023081264W WO2024007615A1 WO 2024007615 A1 WO2024007615 A1 WO 2024007615A1 CN 2023081264 W CN2023081264 W CN 2023081264W WO 2024007615 A1 WO2024007615 A1 WO 2024007615A1
Authority
WO
WIPO (PCT)
Prior art keywords
network traffic
model
log
instance
detected
Prior art date
Application number
PCT/CN2023/081264
Other languages
English (en)
French (fr)
Inventor
李长轩
蒋振超
吴迪
石晓辉
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202211182844.3A external-priority patent/CN117391214A/zh
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Publication of WO2024007615A1 publication Critical patent/WO2024007615A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • This application relates to the field of Internet technology, and in particular to a model training method, device and related equipment.
  • NTA network traffic analysis
  • NDR network detection and response
  • embodiments of the present application provide a model training method for training to obtain a detection model for detecting abnormal network traffic, so as to use the detection model to improve the accuracy of detecting abnormal network traffic.
  • This application also provides corresponding devices, computing device clusters, computer-readable storage media, and computer program products.
  • embodiments of the present application provide a model training method.
  • the model training method can be executed by a corresponding model training device.
  • the model training device obtains a first log, and the first log is used to record instances to be detected.
  • the instance to be detected may be, for example, a network device such as a server, or may be a virtual instance such as a virtual machine, a container, or a bare metal server, etc., so that the model training device uses wireless data based on the first log.
  • the supervised learning algorithm is trained to obtain a target detection model.
  • the target detection model is used to detect whether there is an abnormality in the network traffic of the instance to be detected.
  • the target detection model includes a machine learning model or an AI model, and the target detection model includes at least one sub-model. .
  • the trained target detection model can be used to detect the network traffic of the instance to be detected, there is no need to use hazard indicators to determine whether the network traffic of the instance to be detected is abnormal. This can get rid of the limitations of hazard indicators to detect abnormal networks. The impact of traffic detection accuracy is improved, thereby improving the accuracy of detecting abnormal network traffic. Moreover, since the target detection model is trained through unsupervised learning using the network traffic logs of the instances to be detected, there is no need to use manually labeled training samples for training, which not only reduces the cost of manual labeling, but also improves the target detection model. training efficiency.
  • the unsupervised learning algorithm may be, for example, one or more of k-means clustering algorithm, random forest algorithm, density-based noise application spatial clustering algorithm, or support vector data description algorithm, or may be others type of algorithm.
  • the model training device may obtain a second log.
  • the second log is used to record the network traffic of the instance to be detected in the second time period.
  • the second time period may not overlap with the above-mentioned first time period, so that the model training device can use the target detection model to detect whether there is an abnormality in the network traffic of the instance to be detected in the second time period based on the second log.
  • using the target detection model to detect the network traffic of the instance to be detected does not need to use hazard indicators to determine whether the network traffic of the instance to be detected is abnormal. This can get rid of the limitations of hazard indicators that affect the detection accuracy of abnormal network traffic. , thereby improving the accuracy of detecting abnormal network traffic.
  • the target detection model includes multiple sub-models.
  • the model detection device detects whether there is an abnormal network traffic in the instance to be detected, it may specifically use multiple sub-models to perform inference based on the second log. Obtain detection results outputted by multiple sub-models respectively. Different sub-models have different hyper-parameters (such as learning rates, etc.), or have different model structures, or complete training based on different unsupervised learning algorithms, so that the model training device can By voting on the detection results output by multiple sub-models, it can be determined whether the network traffic of the instance to be detected is abnormal in the second time period.
  • the model training device can detect abnormalities in the network traffic based on other sub-models among the multiple sub-models, thereby effectively improving the ability of the model training device to detect abnormalities. improve the accuracy of network traffic and improve the robustness of anomaly detection.
  • the model training device when the model training device votes on the detection results respectively output by the multiple sub-models, it may specifically vote on the detection results output by the multiple sub-models according to a target voting strategy, and the target voting strategy Including one or more of the majority voting strategy, the one-vote veto strategy, or the one-vote pass strategy, or other applicable voting strategies.
  • the target-based voting strategy is used to improve the accuracy of the model training device in detecting abnormal network traffic and improve the robustness of anomaly detection.
  • the model training device can also generate a configuration interface, which is used to prompt the user to select a target voting strategy from multiple candidate voting strategies, or the configuration interface is used to prompt the user to define the target voting strategy. .
  • the model training device can support users to configure or customize the target voting strategy, thereby improving the flexibility of detecting abnormal network traffic.
  • the model training device can also generate a configuration interface, which is used to prompt the user to configure one or more of the number of multiple sub-models, model hyperparameters, or unsupervised learning algorithms.
  • the model training device can complete network traffic anomaly detection based on the number of user-configured sub-models, or complete model training based on user-configured model hyperparameters or unsupervised learning algorithms, support users' customization of these parameters, and improve solution implementation. flexibility.
  • the model training device before using multiple sub-models to perform inference based on the second log respectively, can randomly extract sub-models from the model pool to obtain the network traffic used to detect the instance to be detected this time. Whether multiple sub-models are abnormal. In this way, by randomly extracting sub-models, the impact of some sub-models with poor ability to identify abnormal network traffic on the detection performance of the model training device can be reduced.
  • the model training device when the model training device obtains the first log for training the target detection model, it may specifically obtain the indication information of the network traffic of the instance to be detected provided by the user to the cloud service platform, such as The name or IP address of the detection instance is detected, so that the model training device collects the first log from the instance to be detected according to the instruction information. In this way, the model training device can implement a cloud service for detecting network traffic anomalies on examples to be detected.
  • the model training device when it obtains the first log used to train the target detection model, it may specifically obtain multiple original logs, and the multiple original logs are used to record multiple data monitored by the cloud service platform.
  • the network traffic of the instance, the multiple instances include the instance to be detected, so that the first log is filtered out from the multiple original logs according to the IP address of the instance to be detected.
  • the model training device can obtain the first log corresponding to the instance specified by the user, so that the instance specified by the user can be processed based on the first log.
  • Network traffic anomaly detection to meet users’ needs for anomaly detection of instance network traffic.
  • the model training device when it obtains the first log used to train the target detection model, it may specifically obtain multiple candidate logs corresponding to the instance to be detected, and use the deduplication condition to compare the multiple candidates
  • the logs are deduplicated, that is, network traffic of the same type is deduplicated, so that each type of network traffic used to train the detection model may be kept at a similar level to obtain the first log.
  • the deduplication conditions include At least one type identifier of network traffic, which may be five-tuple information recorded in a network traffic log, or may be a value calculated based on the five-tuple information (such as an MD5 value, etc.). In this way, using the deduplicated network traffic to train the target detection model can improve the accuracy of the target detection model in identifying a small number of types of network traffic.
  • the model training device when the model training device uses an unsupervised learning algorithm to train based on the first log to obtain the target detection model, it may specifically extract network traffic characteristics based on the first log.
  • the network traffic characteristics may include instances to be detected.
  • the network traffic characteristics are trained using an unsupervised learning algorithm to obtain a target detection model. In this way, the target detection model can be trained based on the network traffic characteristics in the logs.
  • the model training device when the model training device utilizes network traffic characteristics and uses an unsupervised learning algorithm to train to obtain a target detection model, it may specifically normalize the network traffic characteristics, such as limiting the value of the network traffic characteristics. In the range between 0 and 1, the target network traffic characteristics are obtained, and the target network traffic characteristics are used to train the target detection model using an unsupervised learning algorithm. In this way, even if some of the network traffic features extracted from the database are network traffic features corresponding to abnormal network traffic, the impact of the abnormal network traffic features on the detection accuracy of the target detection model can be reduced through normalization processing, thus The interference of abnormal network traffic can be further eliminated.
  • the model training device can also update the target detection model based on the second log. In this way, by dynamically updating the target detection model, the detection effect of the target detection model can be maintained to be continuously effective.
  • inventions of the present application also provide a model training device.
  • the model training device includes: a log acquisition module for acquiring a first log.
  • the first log is used to record the data of the instance to be detected within a first time period.
  • Network traffic the model training module is used to obtain a target detection model based on the first log and trained using an unsupervised learning algorithm.
  • the target detection model is used to detect whether there are abnormalities in the network traffic of the instance to be detected.
  • the target detection model includes a machine learning model or manual
  • the intelligent AI model and the target detection model include at least one sub-model.
  • the log acquisition module is also used to acquire a second log, and the second log is used to record the network traffic of the instance to be detected in the second time period;
  • the model training device also includes an anomaly detection module.
  • the detection module is configured to use the target detection model to detect whether there are abnormalities in the network traffic of the instance to be detected in the second time period based on the second log.
  • the target detection model includes multiple sub-models and an anomaly detection module, configured to: use multiple sub-models to perform inference based on the second log, and obtain detection results output by multiple sub-models respectively.
  • Different sub-models have Different hyperparameters, or have different model structures, or complete training based on different unsupervised learning algorithms; vote on the detection results output by multiple sub-models to determine whether the network traffic of the instance to be detected is abnormal in the second time period .
  • the anomaly detection module is used to vote on the detection results output by multiple sub-models according to a target voting strategy.
  • the target voting strategy includes a majority voting strategy, a one-vote veto strategy, or a one-vote pass. One or more of the strategies.
  • the model training device further includes: a configuration module for generating a configuration interface, the configuration interface is used to prompt the user to select a target voting strategy from multiple candidate voting strategies, or the configuration interface is used to prompt the user to define Target voting strategy.
  • the model training device further includes: a configuration module, used to generate a configuration interface, and the configuration interface is used to prompt the user to specify one of the number of multiple sub-models, model hyperparameters, or unsupervised learning algorithms. Or multiple configurations.
  • the anomaly detection module is also used to randomly extract sub-models in the model pool to obtain multiple sub-models before using multiple sub-models to perform inference based on the second log.
  • the log acquisition module is configured to: obtain the instruction information provided by the user to the cloud service platform for the network traffic of the instance to be detected; and collect the first log from the instance to be detected according to the instruction information.
  • the log acquisition module is used to: acquire multiple original logs.
  • the multiple original logs are used to record the network traffic of multiple instances monitored by the cloud service platform, and the multiple instances include the instances to be detected; according to The Internet Protocol IP address of the instance to be detected is used to filter out the first log from multiple original logs.
  • the log acquisition module is used to: obtain multiple candidate logs corresponding to the instance to be detected; use deduplication conditions to deduplicate the multiple candidate logs to obtain the first log.
  • the deduplication conditions include: Identifier of at least one type of network traffic.
  • the model training module is used to: extract network traffic characteristics according to the first log.
  • the network traffic characteristics include the number of communications of the instance to be detected, the number of destination Internet protocol IP addresses, the number of source IP addresses, and the number of destination IP addresses.
  • the model training module is used to: normalize network traffic characteristics to obtain target network traffic characteristics; use the target network traffic characteristics to train with an unsupervised learning algorithm to obtain a target detection model.
  • model training device provided in the second aspect corresponds to the model training method provided in the first aspect, so the technical effects of the second aspect and any of the embodiments of the second aspect can be found in the first aspect or The technical effects achieved by the corresponding implementation of the first aspect.
  • the present application provides a computing device cluster.
  • the computing device includes at least one computing device.
  • the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions.
  • the at least one processor executes the instruction stored in the at least one memory, so that the computing device cluster executes the model training method in the above first aspect or any possible implementation of the first aspect.
  • the memory can be integrated into the processor or independent of the processor.
  • the at least one computing device may also to include the bus. Among them, the processor is connected to the memory through a bus.
  • the memory may include readable memory and random access memory.
  • the present application provides a computer-readable storage medium that stores instructions that, when run on at least one computing device, cause the at least one computing device to execute the above-mentioned first aspect. Or the method described in any implementation of the first aspect.
  • the present application provides a computer program product containing instructions that, when run on at least one computing device, cause the at least one computing device to execute the above-mentioned first aspect or any implementation of the first aspect. the method described.
  • Figure 1 is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application.
  • Figure 2 is a schematic diagram of another exemplary application scenario provided by the embodiment of the present application.
  • Figure 3 is a schematic flow chart of a model training method provided by an embodiment of the present application.
  • Figure 4 is a schematic flowchart of a method for anomaly detecting network traffic of an instance to be detected provided by an embodiment of the present application
  • Figure 5 is a schematic diagram of voting on multiple detection results provided by the embodiment of the present application.
  • Figure 6 is a schematic diagram of a configuration interface provided by an embodiment of the present application.
  • Figure 7 is a schematic flowchart of another method for anomaly detecting network traffic of an instance to be detected provided by an embodiment of the present application.
  • Figure 8 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
  • Figure 9 is a schematic structural diagram of a computing device cluster provided by an embodiment of the present application.
  • instances in the network may be subject to network attacks during operation, such as malware attacks, man-in-the-middle attacks, denial of service attacks, injection attacks, zero-day vulnerability attacks, domain name system ( domain name system, DNS) tunnel attacks, etc.
  • network traffic refers to the data traffic generated in the network by instances that can connect to the network. Normally, you can determine whether an instance is under network attack by detecting whether the instance's network traffic meets preset hazard indicators.
  • the network traffic of an instance suddenly increases to several times or dozens of times of the normal state within a period of time, or the number of accesses of other instances by an instance within a period of time exceeds a threshold, etc., it can be determined that the instance is under network attack. In this way, by promptly sensing that an instance is under network attack and performing operations and maintenance, the losses caused by network attacks on the instance can be reduced.
  • preset hazard indicators are easily bypassed by experienced network attackers (such as hackers) using countermeasures, or it is difficult to apply new network attack methods used by some network attackers, such as when network attackers modify data.
  • information such as the port number in the packet is included, based on hazard indicators such as the size of the network traffic or the upper limit of the number of accessed instances, it is usually difficult to detect that the instance has suffered a network attack, resulting in a low accuracy in detecting abnormal network traffic.
  • embodiments of the present application provide a model training method, which can be performed by a corresponding model training device Execution is used to train and obtain a target detection model for detecting whether there are abnormalities in the network traffic of the instance, so as to use the target detection model to improve the accuracy of detecting abnormal network traffic.
  • the model training device obtains a first log for recording the network traffic of the instance to be detected within a first time period, and performs unsupervised training on the target detection model based on the first log.
  • the trained target detection model Used to detect whether there is an abnormality in the network traffic of the instance to be detected, the target detection model may be a machine learning model or an artificial intelligence (AI) model, and the target detection model includes at least one sub-model.
  • AI artificial intelligence
  • the target detection model trained by the model training device can be used to detect the network traffic of the instance to be detected, there is no need to use hazard indicators to determine whether the network traffic of the instance to be detected is abnormal, which can get rid of the limitations of hazard indicators.
  • the impact on the detection accuracy of abnormal network traffic thereby improving the accuracy of detecting abnormal network traffic.
  • the target detection model is trained through unsupervised learning using the network traffic logs of the instances to be detected, there is no need to use manually labeled training samples for training, which not only reduces the cost of manual labeling, but also improves the target detection model. training efficiency.
  • the above model training device can be deployed in the cloud and used to provide a cloud service for a user-side device to generate a detection model for detecting whether the device has network traffic anomalies.
  • the model training device 100 is deployed in a cloud, such as a public cloud, a private cloud, an edge cloud, etc., and can be implemented by a cloud computing device or a computing device cluster, and can be used on the user side.
  • Multiple hosts respectively provide services for generating detection models for detecting whether network traffic anomalies exist on each host.
  • Figure 1 illustrates an example of providing cloud services to hosts 201 to 204.
  • the model training device 100 may include a log acquisition module 101 and a model training module 102.
  • the log acquisition module 101 is used to acquire logs recording network traffic of the host 201 (or other hosts), and provide the acquired logs to the model training module 102.
  • the model training module 102 is used to train a target detection model using an unsupervised learning algorithm based on the received logs.
  • the target detection model is used to detect whether there are abnormalities in the network traffic of the host 201.
  • the model training device 100 may also include other functional modules, such as an anomaly detection module 103, an alarm module 104, a configuration module 105, etc.
  • the anomaly detection module 103 is used to use the target detection model that has completed unsupervised training to detect whether there are abnormalities in the network traffic of the host 201 based on the logs provided by the log acquisition module 101.
  • the alarm module 104 is used to perform corresponding alarm operations after determining that the network traffic of the host 201 is abnormal.
  • the configuration module 105 is used to configure corresponding information for the model training device 100, such as configuring the unsupervised learning algorithm used to train the target detection model.
  • the above model training device can be deployed locally, thereby providing users with a local service of training a target detection model (and using the target detection model to detect abnormal network traffic).
  • the model training device 100 is deployed independently of the terminal 200 on the user side and establishes a connection with the terminal 200; alternatively, the model training device 100 can also be integrated into the user's terminal 200, etc., so that The model training device 100 can locally generate a target detection model for detecting whether there is an abnormality in the network traffic of the terminal 200, so as to use the target detection model to determine whether the terminal 200 is subject to a network attack.
  • model training device 100 can be implemented by software, or can be implemented by hardware.
  • the model training device 100 may include code running on a computing instance.
  • the computing instance may include at least one of a bare metal server (BMS), a virtual machine, and a container. A sort of.
  • the above computing instance may be one or more.
  • the model training apparatus 100 may include code running on multiple BMS/virtual machines/containers. It should be noted that multiple BMS/virtual machines/containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, multiple BMS/virtual machines/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs. Each AZ includes one data center or multiple AZs. geographically close data centers. Among them, usually a region can include multiple AZs.
  • VPC virtual private cloud
  • multiple BMS/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or across multiple VPCs.
  • VPC virtual private cloud
  • Cross-region communication between two VPCs in the same region and between VPCs in different regions requires a communication gateway in each VPC, and the interconnection between VPCs is realized through the communication gateway. .
  • the model training device 100 is an example of a hardware functional unit.
  • the model training device 100 may include at least one computing device, such as a server.
  • the model training device can also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD).
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the above-mentioned PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a general array logic (generic array logic, GAL), or any combination thereof.
  • CPLD complex programmable logical device
  • FPGA field-programmable gate array
  • GAL general array logic
  • the multiple computing devices included in the model training apparatus 100 may be distributed in the same region or in different regions. Multiple computing devices included in the model training device 100 may be distributed in the same AZ or in different AZs. Similarly, multiple computing devices included in the model training apparatus 100 may be distributed in the same VPC or in multiple VPCs.
  • the plurality of computing devices mentioned above may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
  • FIG 3 is a schematic flow chart of a network traffic detection method in an embodiment of the present application. This method can be applied to the application scenarios shown in Figure 1 or Figure 2 above, or can also be applied to other applicable application scenarios. For ease of understanding, the following description takes the application scenario shown in Figure 1 as an example.
  • the network traffic detection method shown in Figure 3 may specifically include:
  • the log acquisition module 101 acquires the first log, which is used to record the network traffic of the instance to be detected in the first time period.
  • the instance to be detected may be an instance in the network that can interact with data and generate network traffic, including a virtual instance or a physical instance.
  • the virtual instance can be, for example, a virtual machine, a container, or a BMS.
  • the physical instance may be, for example, the user's host, a gateway in the network, a server, etc., which is not limited in this embodiment.
  • the model training device 100 can perform anomaly detection on the network traffic of the instance to be detected by analyzing the log.
  • the log acquisition module 101 first obtains a log used to record the network traffic of the instance to be detected within a period of time. In order to facilitate distinction and description, it is hereinafter referred to as the first log.
  • the obtained first log is used to record the network traffic of the instance to be detected. Detect the network traffic of the instance within the first time period.
  • the number of the first logs may be one or more.
  • the log acquisition module 101 can obtain the log stored in the instance to be detected by accessing the corresponding storage area of the instance to be detected.
  • the log acquisition module 101 can acquire multiple instances monitored by the cloud service platform. Multiple original logs are sent, and the multiple original logs are used to record the network traffic of the multiple instances (one of which is the above-mentioned instance to be detected), so that the log acquisition module 101 can use the IP address of the instance to be detected or other information for Identify the information of the instance to be detected, and filter out the first log belonging to the instance to be detected from multiple original logs.
  • the log acquisition module 101 can determine the instance specified by the user as the instance to be detected, and determine the first log corresponding to the instance to be detected from multiple original logs; or, the log acquisition module 101 can poll and detect multiple Whether there is network traffic abnormality in each instance in the instance, at this time, the log acquisition module 101 can determine the currently polled instance as the instance to be detected, and further determine the first log corresponding to the instance to be detected from multiple original logs.
  • the log acquisition module 101 can obtain the instruction information provided by the user to the cloud service platform where the model training device 100 is located for the network traffic of the instance to be detected, which may include, for example, the IP address of the instance to be detected, the time period of the network traffic, and other information. Therefore, the log acquisition module 101 can collect the first log from the instance to be detected according to the instruction information, so as to obtain the first log corresponding to the instance to be detected in a targeted manner under the instruction of the user.
  • the model training device 100 can use an artificial intelligence (AI) model or a machine learning (ML) model to implement anomaly detection on the network traffic of the instance to be detected, and the AI model or ML model can Trained based on unsupervised learning algorithm.
  • AI artificial intelligence
  • ML machine learning
  • the log acquisition module 101 may send the first log to the model training module 102.
  • the model training module 102 uses an unsupervised learning algorithm to train to obtain a target detection model.
  • the target detection model is used to detect whether there is an abnormality in the network traffic of the instance to be detected.
  • the target detection model includes an ML model or an AI model.
  • the target detection model includes at least one sub-model.
  • the target detection model can include one or more sub-models, and each sub-model can be an ML model, such as an isolated forest model, a model based on a one-class support vector machine (OCSVM) algorithm, etc.; or,
  • the sub-models in the target detection model can be AI models, such as models based on neural networks.
  • some of the sub-models in the target detection model may be ML models, and another part of the sub-models may be AI models, which is not limited in this embodiment.
  • the model training module 102 may first convert the first log into data suitable for input to the target detection model.
  • the model training module 102 may first extract information such as quintuples and timestamps in the first log, and extract network traffic characteristics from the first log.
  • the model training module 102 can determine the logs located within the sliding time window in the first log based on the length of the preset sliding time window (such as 10 minutes, 30 minutes, etc.), and further select the logs located within the sliding time window. Extract network traffic characteristics from logs.
  • the model training module 102 may combine the first log or the logs located within the sliding time window into a session, and perform feature extraction on the session to obtain the network traffic characteristics.
  • the extracted network traffic characteristics may be one or more of the following characteristics:
  • the model training module 102 can count the number of communications between the instance to be detected and other instances within the first time period or within the sliding time window based on the first log (or the log located within the sliding time window).
  • the model training module 102 can count the number of destination IP addresses based on the five-tuple information extracted from the first log (or the log located within the sliding time window).
  • the extracted network traffic characteristics can also be other types of characteristics, such as the number of communications, the number of destination IP addresses, the number of destination ports, and the destination when the IP address of the instance to be detected is used as the source IP address.
  • the number of IP addresses, the sum of the number of destination ports, etc. are not limited in this embodiment.
  • the first log received by the model training module 102 may contain errors in part of the log information.
  • the five-tuple information in part of the log contains default values (that is, the parameter values in the log are not updated in time. It is the value of the initialization state).
  • the model training module 102 may, after obtaining the first log, filter the first log, such as filtering logs including default values, etc., and extract network traffic characteristics from the filtered log.
  • the log acquisition module 101 may also filter the collected logs corresponding to the instances to be detected, such as filtering logs containing default values in the five-tuple information, etc.
  • the model training module 102 can use the extracted network traffic characteristics to train a pre-built target network model.
  • the model training device 102 can input network traffic characteristics into each sub-model included in the target detection model, and use a preset unsupervised learning algorithm to perform unsupervised training on each sub-model in the target detection model.
  • the unsupervised learning algorithm can be, for example, k-means clustering algorithm, random forest algorithm, density-based spatial clustering of applications with noise (DBSCAN) algorithm, or support vector data description ( One or more of the support vector domain description (SVDD) algorithms, or other types of algorithms, which are not limited in this embodiment.
  • the unsupervised learning algorithm used to train each sub-model may be the same or different, which is not limited in this embodiment.
  • the model training module 102 can extract multiple sets of network traffic characteristics from multiple logs corresponding to the instances to be detected in a similar manner as described above, and store the multiple sets of network traffic characteristics in a database, such as a Redis database, so that When training the target detection model, the model training module 102 can randomly extract some network traffic features from the database as training samples, and use the extracted network traffic features to perform unsupervised training on the target detection model. Under normal circumstances, the vast majority (or all network traffic) of the instance to be detected is normal traffic data. Correspondingly, based on the network traffic characteristics extracted from multiple logs corresponding to the instance to be detected, the vast majority ( Or all network traffic characteristics) are network traffic characteristics corresponding to normal network traffic.
  • the model training module 102 randomly extracts network traffic characteristics with a smaller sample size to target the target data. Training the target detection model can reduce the probability of using the characteristics corresponding to abnormal network traffic to train the target detection model, thereby effectively reducing the impact of abnormal network traffic in the database on the detection accuracy of the target detection model. For example, the model training module 102 can randomly extract 3% of network traffic features from the database as samples for training the target detection model.
  • the network traffic characteristics stored in the database can be released after completing the training of the target detection model; or, the life cycle can be set for the network traffic characteristics when storing them, so that when the network traffic characteristics are stored in the database for According to the duration of the life cycle, the network traffic characteristics are released from the database to reduce the occupation of storage resources by the network traffic characteristics.
  • the model training module 102 can also normalize the network traffic characteristics, such as limiting the value of the network traffic characteristics to a range between 0 and 1, and then, the model The training module 102 then uses the target network traffic characteristics obtained after normalization to train the target detection model. In this way, even if some of the network traffic features extracted from the database are network traffic features corresponding to abnormal network traffic, the impact of the abnormal network traffic features on the detection accuracy of the target detection model can be reduced through normalization processing, thus The interference of abnormal network traffic can be further eliminated.
  • abnormal network traffic data (or abnormal network traffic data + normal network traffic data) can also be used to test each sub-model in the target detection model to verify Whether the accuracy of each sub-model in the target detection model in identifying abnormal network traffic meets the preset requirements, such as verifying whether the recognition accuracy of each sub-model in the target detection model reaches a threshold (such as 90%).
  • a threshold such as 90%
  • the model training device 100 can use the target detection model to detect abnormal network traffic; and when the target When the accuracy of some or all sub-models in the detection model does not meet the preset requirements, the model training module 102 can adjust the hyperparameters (such as learning rate, etc.) of the sub-models in the target detection model that do not meet the preset requirements and continue to train the sub-models. The model is trained until a target detection model that meets the accuracy requirements is obtained. In this way, after the target detection model is trained, the target detection model can be used to detect anomalies in the network traffic of the instance to be detected. Based on this, after training the target detection model, the model training module 102 can provide it to the anomaly detection module 103, and this embodiment can further include the following steps S303 and S304.
  • the log acquisition module 101 acquires the second log, which is used to record the network traffic of the instance to be detected in the second time period.
  • the model training device 100 can use the target detection model trained by logs in a period of time, and then use the trained target detection model to perform anomaly detection on the network traffic of the instance to be detected in another period of time.
  • the log obtained by the log acquisition module 101 and used to record the network traffic of the instance to be detected in the second time period is referred to as the second log.
  • the second time period and the first time period are different time periods, and the second time period may be later than the first time period, or the second time period may be earlier than the first time period.
  • the log acquisition module 101 provides the acquired second log to the anomaly detection module 103 .
  • the anomaly detection module 103 uses the target detection model to detect whether there are abnormalities in the network traffic of the instance to be detected in the second time period based on the second log.
  • the anomaly detection module 103 can extract network traffic characteristics from the second log and input them into the target detection model to obtain the inference results of the target detection model.
  • the inference results are used to indicate that the instance to be detected is in the second time period. Is there any abnormality in the network traffic within the system? Among them, the anomaly detection module 103 extracts network traffic characteristics from the second log.
  • the process please refer to the aforementioned description of extracting network traffic characteristics from the first log, and will not be described in detail here.
  • the anomaly detection module 103 can determine whether there is an abnormality in the network traffic of the instance to be detected in the second time period based on the inference results of the sub-model; and when the target detection model includes multiple sub-models When The result determines whether there is an abnormality in the network traffic of the instance to be detected in the second time period.
  • the anomaly detection module 103 detects an abnormality in the network traffic of the instance to be detected, it can also notify the alarm module 104 so that the alarm module 104 can execute the corresponding alarm policy.
  • the alarm module 104 can generate an alarm interface and display the alarm interface on the display of the model training device 100 to prompt the user that there is an abnormality in the network traffic of the instance to be detected. .
  • the alarm module 104 can generate an alarm message for the instance to be detected.
  • the alarm message carries the identifier of the instance to be detected.
  • the alarm message The article can also include detailed information about network traffic anomalies, such as anomaly types, anomaly data, etc.
  • the alarm module 104 can send the alarm message to the client provided externally by the model training device 100, or send the alarm message to a pre-specified terminal instance, so that the client or the terminal instance prompts the user that the instance to be detected currently exists on the network. Traffic abnormality.
  • the model training device 100 uses the trained target detection model to detect the network traffic of the instance to be detected, instead of using the hazard indicator to determine whether the network traffic of the instance to be detected is abnormal, which can get rid of the limitations of the hazard indicator. influence on the detection accuracy of abnormal network traffic, thereby improving the accuracy of the model training device 100 in detecting abnormal network traffic.
  • the target detection model is trained through unsupervised learning using the network traffic logs of the instances to be detected, there is no need to use manually labeled training samples for training, which not only reduces the cost of manual labeling, but also improves the target detection model. training efficiency.
  • the model training device 100 uses the inference results output by the sub-model to determine whether there is an abnormality in the network traffic of the instance to be detected; when the target detection model includes multiple sub-models, the model training device 100 can use The inference results output by multiple sub-models comprehensively determine whether the network traffic of the instance to be detected is abnormal, so as to further improve the accuracy of network traffic anomaly detection and improve user experience.
  • the inference results output by multiple sub-models comprehensively determine whether the network traffic of the instance to be detected is abnormal, so as to further improve the accuracy of network traffic anomaly detection and improve user experience.
  • the method may specifically include:
  • the log acquisition module 101 acquires the first log, which is used to record the network traffic of the instance to be detected in the first time period.
  • the model training module 102 performs unsupervised training on multiple different sub-models based on the first log.
  • the multiple sub-models are all used to detect whether there are abnormalities in the network traffic of the instance to be detected.
  • Different sub-models have different hyper-parameters, Or have different model structures, or complete training based on different unsupervised learning algorithms.
  • the model training module 102 may be configured with multiple sub-models. Among them, there are differences in the hyperparameters in different sub-models. For example, the learning rate of sub-model A is 0.01, while the learning rate of sub-model B is 0.001, and the learning rate of sub-model C is 0.01. is 0.0001, etc., so the model training module 102 uses the first log to perform unsupervised training on multiple sub-models with different hyper-parameters, and obtains multiple sub-models based on different hyper-parameters. For example, the model training module 102 can generate multiple hyperparameters through a grid search strategy, and set multiple different sub-models based on the multiple hyperparameters, so as to use the first log to separately train the multiple different sub-models. submodel.
  • sub-model P adopts a tree structure
  • sub-model Q adopts a neural network structure. Therefore, the model training module 102 uses the first log to perform unsupervised training on each sub-model respectively, and obtains multiple sub-models with different model structures.
  • the model training module 102 uses the first log to separately train multiple sub-models using different unsupervised learning algorithms, such as using k-means clustering algorithm to perform unsupervised training on sub-model X, and using random forest algorithm to perform unsupervised training on sub-model X.
  • Model Y is trained, and the DBSCAN algorithm is used to train sub-model Z, etc.
  • the model training module 102 can combine multiple model hyperparameters, multiple model structures, and multiple unsupervised learning algorithms, and train a sub-model based on each combination.
  • the model training module 102 using the first log to train each sub-model please refer to the relevant description of step S302 in the previous embodiment, and will not be described again here.
  • the model training module 102 can save the multiple sub-models into a model pool and perform persistent storage.
  • the log acquisition module 101 acquires the second log, which is used to record the network traffic of the instance to be detected in the second time period.
  • S404 The anomaly detection module 103 randomly selects N sub-models from the model pool.
  • N is a positive integer greater than 1, and the value of N is less than the total number of sub-models in the model pool. In practical applications, N can be an odd number. It can be understood that the extracted N sub-models are multiple sub-models in the target detection model in the aforementioned embodiment.
  • the anomaly detection module 103 uses the randomly selected N sub-models to perform inference based on the second log, and obtains the detection results output by the N sub-models respectively.
  • the detection results output by each sub-model are used to indicate whether the network traffic of the instance to be detected determined by the sub-model is abnormal in the second time period. In this way, the anomaly detection module 103 can obtain N detection results.
  • the anomaly detection module 103 can use sub-model 1 to sub-model N randomly selected from the model pool to perform inference based on the second log, respectively, to obtain N detection results, that is, detection result 1 to detection result N, and The N detection results are added to the voting pool so that subsequent voting on the N detection results can be performed to determine whether there are abnormalities in the network traffic of the instance to be detected.
  • the anomaly detection module 103 votes on the detection results respectively output by the N sub-models to determine whether the network traffic of the instance to be detected is abnormal in the second time period.
  • the detection results output by N sub-models may be inconsistent.
  • the detection results output by some sub-models indicate that the network traffic of the instance to be detected is abnormal
  • the detection results output by other sub-models indicate the network traffic of the instance to be detected.
  • the anomaly detection module 103 can vote on the N detection results in the voting pool, and finally determine whether there is an abnormality in the network traffic of the instance to be detected in the second time period based on the voting results.
  • the anomaly detection module 103 can vote on the detection results respectively output by the N sub-models based on the target voting strategy.
  • the target voting strategy includes a majority voting strategy, a one-vote veto strategy, or a one-vote pass strategy.
  • the multi-vote voting strategy refers to voting in compliance with the principle of the minority obeying the majority, and the result with a larger number of votes will be regarded as the final result.
  • the anomaly detection module 103 votes on the detection results output by 9 sub-models.
  • the detection results output by 3 sub-models indicate that the network traffic of the instance to be detected is normal, and the detection results output by the remaining 6 sub-models indicate that the instance to be detected is normal. If there is an abnormality in the network traffic of the instance to be detected, then based on the principle of majority submission, the anomaly detection module 103 may finally determine that there is an abnormality in the network traffic of the instance to be detected.
  • the one-vote veto strategy means that when any detection result indicates that the network traffic of the instance to be detected is abnormal, the anomaly detection module 103 can finally determine that the network traffic of the instance to be detected is abnormal. When all detection results indicate that the network traffic of the instance to be detected is normal, the anomaly detection module 103 will finally determine that there is no abnormality in the network traffic of the instance to be detected.
  • the one-vote pass strategy means that when any detection result indicates that the network traffic of the instance to be detected is not abnormal, the anomaly detection module 103 can finally determine that the network traffic of the instance to be detected is normal. When all detection results indicate that the network traffic of the instance to be detected is abnormal, the anomaly detection module 103 will finally determine that the network traffic of the instance to be detected is abnormal.
  • the anomaly detection module 103 can detect anomalies in the network traffic based on other sub-models among the multiple sub-models, thereby effectively improving the model training device 100 The accuracy of detecting abnormal network traffic and improving the robustness of anomaly detection.
  • the impact of some sub-models with poor ability to identify abnormal network traffic on the detection performance of the model training device 100 can be reduced.
  • the anomaly detection module 103 detects an abnormality in the network traffic of the instance to be detected, it can also notify the alarm module 104 so that the alarm module 104 can execute the corresponding alarm policy.
  • the model training device 100 can also support the user to configure the target voting strategy.
  • the model training device 100 may also include a configuration module 105, and the configuration module 105 is used to generate a configuration interface.
  • the configuration interface is used to prompt the user to select a target voting strategy from multiple candidate voting strategies, and the configuration interface
  • the configuration interface is presented to the user through a client provided externally by the model training device 100, or the configuration interface is presented to the user through a display included in the model training device 100 (which is deployed on the user side).
  • the configuration module 105 can generate a configuration interface as shown in Figure 6, which includes multiple candidate voting strategies for the user to select, such as Candidate Voting Strategy 1 to Candidate Voting Strategy 4 in Figure 6, so that the user One candidate voting strategy can be selected from multiple candidate voting strategies for abnormal network traffic detection; accordingly, the configuration module 105 can determine the candidate voting strategy selected by the user as the target voting strategy and provide it to the anomaly detection module 103 Perform testing.
  • the configuration interface generated by the configuration module 105 can be used to prompt the user to customize a voting strategy, that is, to customize a voting strategy that determines the final detection result based on the detection results of multiple sub-models.
  • a voting strategy that is, to customize a voting strategy that determines the final detection result based on the detection results of multiple sub-models.
  • the strategy name and strategy as shown in Figure 6 Content etc.
  • the user can define the policy content as follows: when more than 85% of the detection results indicate that the network traffic of the instance to be detected is abnormal, the final detection result is determined to be abnormal network traffic, etc. This embodiment does not limit this. .
  • the model training device 100 can also support the user to configure other information.
  • users can not only configure voting strategies but also anomaly detection on the configuration interface shown in Figure 6
  • Module 103 configures one or more of the number of sub-models used each time (that is, the value of N), the hyperparameters of each sub-model, or the unsupervised learning algorithm used to train each sub-model.
  • the interface for users to configure voting strategies can be two independent interfaces from the interface for users to configure the number of sub-models, model hyperparameters, and unsupervised learning algorithms. It is worth noting that the above implementation methods of information configuration are only used as some exemplary explanations.
  • users can also configure information in other ways, or users can configure other types of information, such as configuring model pool neutrons. Maximum/minimum number of models, maximum size of each submodel, etc.
  • the distribution of network traffic of the instance to be detected may change dynamically during normal operation.
  • the instance to be detected interacts with a large number of servers 1 to 10 between 0:00-12:00. Data packets, and a larger number of data packets are exchanged with servers 11 to 20 from 12:00 to 24:00.
  • the model training device 100 detects abnormal network traffic for an instance to be detected, it can train one or more sub-models that match the time period based on the traffic distribution of the instance to be detected in different time periods, so that at different times, Segment uses different sub-models to detect abnormal network traffic.
  • the model training device 100 can dynamically update one or more trained sub-models.
  • the model training device 100 can use this The network traffic within a period of time automatically updates one or more sub-models in the model pool, so that the sub-models in the model pool can adapt to changes in the network traffic distribution of the instance to be detected through dynamic updates, thereby ensuring that the sub-models The detection effect is continuously effective.
  • the model training device 100 can use the second log to dynamically update multiple sub-models in the target detection model.
  • the model training device 100 can train each sub-model in the target detection model based on the logs of the entire network traffic. In other possible embodiments, for the instance to be detected and other For repeated network traffic between instances, the model training device 100 can deduplicate the network traffic and train the sub-model based on the deduplicated network traffic, thereby improving the inference accuracy of the sub-model.
  • FIG. 7 an exemplary description will be given with reference to FIG. 7 .
  • the method may specifically include:
  • the log acquisition module 101 acquires multiple candidate logs, which are used to record the network traffic of the instance to be detected in the first time period.
  • step S701 For the specific implementation process of step S701, please refer to the relevant description of the aforementioned step S301, and will not be described again here.
  • the log acquisition module 101 uses the deduplication condition to deduplicate the plurality of candidate logs to obtain the first log.
  • the deduplication condition includes at least one type identifier of network traffic.
  • the instance to be detected may have a large amount of data interaction with the same external instance, resulting in a large amount of network traffic of the same type.
  • training a sub-model based on a large number of network traffic of the same type may affect the identification accuracy of the sub-model for network traffic with a small number of samples due to large sample differences between different types of network traffic.
  • the log acquisition module 101 can deduplicate multiple candidate logs, that is, deduplicate network traffic of the same type, so that each type of network traffic used to train the target detection model likely to remain at a similar level. In this way, using the deduplicated network traffic to train the target detection model can improve the target detection model. Accuracy in identifying parts of a smaller number of types of network traffic.
  • the log acquisition module 101 uses deduplication conditions to deduplicate multiple candidate logs, and the deduplication conditions include at least one type identifier of network traffic.
  • the type identifier of the network traffic can be, for example, the five-tuple information recorded in the network traffic log. Different network traffic with the same five-tuple information can be regarded as network traffic of the same type; or, the network traffic
  • the traffic type identifier can be, for example, a value calculated based on five-tuple information, such as an MD5 value calculated based on the MD5 message digest algorithm (MD5 mssage-digest algorithm).
  • the log acquisition module 101 can determine the type identifier corresponding to each piece of network traffic based on the candidate logs, and deduplicate multiple candidate logs based on the type identifier. Specifically, the network traffic recorded in multiple candidate logs belongs to the same type.
  • the log is deduplicated and the deduplicated log is obtained.
  • the log obtained after deduplication is called the first log in the following. In this way, the number of different types of network traffic recorded in the first log obtained after deduplication can be kept at a similar level.
  • the deduplication condition in addition to including the type identifier corresponding to the network traffic, may also include an upper limit on the number of network traffic for each type identifier.
  • the network traffic of this type will be deduplicated.
  • the model training device 100 can issue an abnormality alarm for this type of network traffic.
  • the log acquisition module 101 may also calculate the new type identifier. This type identifier is added to the deduplication condition to automatically update the deduplication condition.
  • the model training module 102 performs unsupervised training on multiple different sub-models based on the first log.
  • the multiple sub-models are all used to detect whether there are abnormalities in the network traffic of the instance to be detected.
  • Different sub-models have different hyper-parameters, Or have different model structures, or complete training based on different unsupervised learning algorithms.
  • the log acquisition module 101 acquires the second log, which is used to record the network traffic of the instance to be detected in the second time period.
  • S705 The anomaly detection module 103 randomly selects N sub-models from the model pool.
  • the extracted N sub-models are the above-mentioned target detection models.
  • the anomaly detection module 103 uses the randomly selected N sub-models to perform inference based on the second log, and obtains the detection results output by the N sub-models respectively.
  • the anomaly detection module 103 votes on the detection results respectively output by the N sub-models to determine whether the network traffic of the instance to be detected is abnormal in the second time period.
  • steps S703 to S707 please refer to the relevant descriptions of steps S401 to S406 in the foregoing embodiments, and will not be described again here.
  • the quantitative difference between different types of training samples as sub-models in the target detection model can be reduced, so that the training The sub-model can achieve more accurate anomaly detection for various types of network traffic.
  • each module in the model training device 100 is only used as an example.
  • the log acquisition module 101 can be used to execute the above model training method. Any step in the method.
  • the model training module 102, anomaly detection module 103, alarm module 104, and configuration module 105 can be used to perform any step in the embodiments shown in Figure 3, Figure 4, and Figure 7, and , the steps responsible for implementation by the log acquisition module 101, model training module 102, anomaly detection module 103, alarm module 104, and configuration module 105 can be specified as needed, through the log acquisition module 101, model training module 102, anomaly detection module 103, and alarm module 104.
  • the configuration module 105 implements different steps in the model training method to realize all functions of the model training device 100.
  • the model training device (including the above-mentioned log acquisition module 101, model training module 102, anomaly detection module 103, alarm module 104, and configuration module 105) involved in the model training process may be configured on the computing device.
  • the software on the computing device cluster and by running the software on the computing device or the computing device cluster, the computing device or the computing device cluster can realize the functions of the above model training device.
  • the model training devices involved in the model training process are introduced in detail.
  • Figure 8 shows a schematic structural diagram of a computing device.
  • the above model training device can be deployed on the computing device.
  • the computing device can be a computing device (such as a server) in a cloud environment, or a computing device in an edge environment, or Terminal equipment, etc. can be specifically used to implement the functions of the log acquisition module 101, the model training module 102, the anomaly detection module 103, the alarm module 104, and the configuration module 105 in the above embodiments.
  • computing device 800 includes processor 820 , memory 810 , communication interface 830 , and bus 840 .
  • the processor 820, the memory 810 and the communication interface 830 communicate through the bus 840.
  • the bus 840 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc.
  • the bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 8, but it does not mean that there is only one bus or one type of bus.
  • the communication interface 830 is used to communicate with the outside, such as receiving a first log used to record the network traffic of the instance to be detected in a first time period, etc.
  • the processor 820 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits.
  • the processor 820 may also be an integrated circuit chip with signal processing capabilities.
  • the functions of each module in the model training device can be completed by instructions in the form of hardware integrated logic circuits or software in the processor 820 .
  • the processor 820 can also be a general-purpose processor, a digital signal process (DSP), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, Discrete hardware components can implement or execute the methods, steps and logical block diagrams disclosed in the embodiments of this application.
  • DSP digital signal process
  • FPGA field programmable gate array
  • the general processor can be a microprocessor or the processor can be any conventional processor, etc.
  • the method disclosed in combination with the embodiments of the present application can be directly implemented as a hardware decoding processor to complete the execution, or can be performed using decoding processing.
  • the combination of hardware and software modules in the device is executed.
  • the software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field.
  • the storage medium is located in the memory 810.
  • the processor 820 reads the information in the memory 810 and completes some or all functions of the model training device in combination with its hardware.
  • the memory 810 may include volatile memory, such as random access memory (RAM).
  • RAM random access memory
  • the memory 810 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, HDD or SSD.
  • ROM read-only memory
  • HDD hard disk drive
  • the memory 810 stores executable code
  • the processor 820 executes the executable code to execute the aforementioned model training device. Set 100 to the method executed.
  • module 103, alarm module 104, and configuration module 105 are implemented by software, execute the log acquisition module 101, model training module 102, anomaly detection module 103, alarm module 104, and configuration module in Figures 3, 4, and 7
  • the software or program code required for the functions of 105 is stored in the memory 810.
  • the processor is used to execute the instructions in the memory 810 to implement the above-mentioned model training device 100. method of execution.
  • FIG. 9 shows a schematic structural diagram of a computing device cluster.
  • the computing device cluster 90 shown in FIG. 9 includes multiple computing devices, and the above-mentioned model training device 100 can be deployed on multiple computing devices in the computing device cluster 90 in a distributed manner.
  • the computing device cluster 90 includes multiple computing devices 900.
  • Each computing device 900 includes a memory 910, a processor 920, a communication interface 930, and a bus 940.
  • the memory 910, the processor 920, and the communication interface 930 pass through Bus 940 implements communication connections between each other.
  • Processor 920 may employ a CPU, GPU, ASIC, or one or more integrated circuits.
  • the processor 920 may also be an integrated circuit chip with signal processing capabilities. During the implementation process, part of the functions of the model training device 100 can be completed by instructions in the form of integrated logic circuits or software in the hardware of the processor 920 .
  • the processor 920 can also be a DSP, FPGA, general-purpose processor, other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute some of the methods, steps, and logical block diagrams disclosed in the embodiments of this application.
  • the general processor may be a microprocessor or the processor may be any conventional processor, etc.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly implemented as a hardware decoding processor, or may be executed using a decoding processor.
  • the combination of hardware and software modules in the code processor is executed.
  • the software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field.
  • the storage medium is located in the memory 910.
  • the processor 920 reads the information in the memory 910, and combined with its hardware, can complete part of the functions of the model training device 100.
  • Memory 910 may include ROM, RAM, static storage devices, dynamic storage devices, hard disks (eg, SSD, HDD), etc.
  • the memory 910 may store program codes, for example, part or all of the program code used to implement the log acquisition module 101, part or all of the program code used to implement the model training module 102, part or all of the program code used to implement the anomaly detection module 103 , part or all of the program code used to implement the alarm module 104, part or all of the program code used to implement the configuration module 105, etc.
  • the processor 920 executes part of the method executed by the model training device 100 based on the communication interface 930 , for example, a part of the computing device 900 may be used to execute The other part of the computing device 900 is used to execute the method executed by the above-mentioned alarm module 104 and configuration module 105.
  • the memory 910 can also store data, such as intermediate data or result data generated by the processor 920 during execution, such as the above-mentioned target detection model, results indicating whether there are abnormalities in the network traffic of the instance to be detected, etc.
  • the communication interface 903 in each computing device 900 is used to communicate with the outside, such as interacting with other computing devices 900 and so on.
  • the bus 940 may be a peripheral component interconnection standard bus or an extended industry standard architecture bus, or the like.
  • the bus 940 in each computing device 900 in Figure 9 is represented by only one thick line, but it does not mean that there is only one bus or one type. bus.
  • Any computing device may be a computing device (eg, a server) in a cloud environment, a computing device in an edge environment, or a terminal device.
  • embodiments of the present application also provide a computer-readable storage medium that stores instructions that, when run on one or more computing devices, cause the one or more computing devices to execute Methods executed by each module of the model training device 100 in the above embodiment.
  • embodiments of the present application also provide a computer program product.
  • the computer program product When the computer program product is executed by one or more computing devices, the one or more computing devices execute any one of the foregoing model training methods.
  • the computer program product can be a software installation package. If it is necessary to use any of the foregoing model training methods, the computer program product can be downloaded and executed on the computer.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physically separate.
  • the physical unit can be located in one place, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between modules indicates that there are communication connections between them, which can be specifically implemented as one or more communication buses or signal lines.
  • the present application can be implemented by software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memories, Special components, etc. to achieve. In general, all functions performed by computer programs can be easily implemented with corresponding hardware. Moreover, the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special-purpose circuits. circuit etc. However, for this application, software program implementation is a better implementation in most cases. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or that contributes to the existing technology.
  • the computer software product is stored in a readable storage medium, such as a computer floppy disk. , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to cause a computer device (which can be a personal computer, training device, or network device, etc.) to execute the steps described in various embodiments of this application. method.
  • a computer device which can be a personal computer, training device, or network device, etc.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, the computer instructions may be transferred from a website, computer, training device, or data
  • the center transmits data to the network through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website, computer, training device or data center.
  • wired such as coaxial cable, optical fiber, digital subscriber line (DSL)
  • wireless such as infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that a computer can store, or a data storage device such as a training device or a data center integrated with one or more available media.
  • the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (Solid State Disk, SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请提供了一种模型训练方法，包括：获取第一日志，该第一日志用于记录待检测实例在第一时间段内的网络流量，从而基于该第一日志，采用无监督学习算法训练得到目标检测模型，该目标检测模型用于检测该待检测实例的网络流量是否存在异常，该目标检测模型包括机器学习模型或者AI模型，并且，目标检测模型包括至少一个子模型。如此，利用训练得到的目标检测模型，能够用于对待检测实例的网络流量进行检测，这可以提高检测异常的网络流量的准确率。并且，无需利用人工标注的训练样本训练目标检测模型，从而不仅可以减少人工标注成本，而且，也能提高目标检测模型的训练效率。此外，本申请还提供了对应的装置及相关设备。

Description

模型训练方法、装置及相关设备
本申请要求于2022年07月05日提交中国国家知识产权局、申请号为202210784579.X、申请名称为“一种网络流量检测方法、装置及相关设备”的中国专利申请的优先权，并要求于2022年9月27日提交中国国家知识产权局、申请号为202211182844.3、申请名称为“模型训练方法、装置及相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。其全部内容通过引用结合在本申请中。
技术领域
本申请涉及互联网技术领域，尤其涉及一种模型训练方法、装置及相关设备。
背景技术
在互联网时代，用户的关键资产(如主机、网关等)容易受到网络攻击，这会对用户的资产安全产生重大影响。为此，各大网络安全产商通常会推出网络流量分析(network traffic analysis，NTA)、网络检测与响应(network detection response，NDR)等产品，通过检测关键资产的网络流量是否存在异常，来及时感知关键资产是否受到网络攻击，以便及时对用户的关键资产进行运维，尽可能避免或者挽回用户资产损失。
目前，通常是检测关键资产的网络流量是否满足预先设定的危害指标，并且，当检测到网络流量满足该危害指标时，确定网络流量存在异常，也即确定用户的关键资产受到网络攻击。但是，这种检测方式，容易受到危害指标的局限，在实际应用场景中检测出异常的网络流量的准确率较低，难以满足实际应用的需求。
发明内容
有鉴于此，本申请实施例提供了一种模型训练方法，用于训练得到用于检测异常网络流量的检测模型，以便利用该检测模型提高检测异常的网络流量的准确率。本申请还提供了对应的装置、计算设备集群、计算机可读存储介质以及计算机程序产品。
第一方面，本申请实施例提供了一种模型训练方法，该模型训练方法可以由相应的模型训练装置执行，具体的，模型训练装置获取第一日志，该第一日志用于记录待检测实例在第一时间段内的网络流量，该待检测实例例如可以是服务器等网络设备，或者可以是虚拟机、容器或者裸金属服务器等虚拟实例等，从而模型训练装置基于该第一日志，采用无监督学习算法训练得到目标检测模型，该目标检测模型用于检测该待检测实例的网络流量是否存在异常，该目标检测模型包括机器学习模型或者AI模型，并且，该目标检测模型包括至少一个子模型。
由于所训练得到的目标检测模型，能够用于对待检测实例的网络流量进行检测，从而无需利用危害指标来判断待检测实例的网络流量是否异常，这可以摆脱危害指标所具有的局限性对异常网络流量的检测准确性的影响，从而实现提高检测异常的网络流量的准确率。并且，由于目标检测模型是利用待检测实例的网络流量日志通过无监督学习的方式训练得到，无需利用人工标注的训练样本进行训练，从而不仅可以减少人工标注成本，而且，也能提高目标检测模型的训练效率。
可选地，无监督学习算法，例如可以是k-means聚类算法、随机森林算法、基于密度的噪声应用空间聚类算法、或支持向量数据描述算法中的一种或者多种，或者可以是其它类 型的算法。
在一种可能的实施方式中，在无监督训练得到目标检测模型后，可以模型训练装置可以获取第二日志，该第二日志用于记录待检测实例在第二时间段内的网络流量，该第二时间段例如可以与上述第一时间段不重叠，从而模型训练装置利用该目标检测模型，可以根据第二日志检测待检测实例在第二时间段内的网络流量是否存在异常。如此，利用目标检测模型对待检测实例的网络流量进行检测，无需利用危害指标来判断待检测实例的网络流量是否异常，这可以摆脱危害指标所具有的局限性对异常网络流量的检测准确性的影响，从而实现提高检测异常的网络流量的准确率。
在一种可能的实施方式中，目标检测模型包括多个子模型，此时，模型检测装置在检测待检测实例是否存在网络流量异常时，具体可以是利用多个子模型分别根据第二日志进行推理，得到多个子模型分别输出的检测结果，其中，不同子模型具有不同的超参数(如学习率等)、或具有不同的模型结构、或基于不同的无监督学习算法完成训练，从而模型训练装置对多个子模型分别输出的检测结果进行投票，可以确定待检测实例在第二时间段内的网络流量是否异常。如此，即使部分子模型未能检测出待检测实例的网络流量存在异常，模型训练装置也能基于多个子模型中的其它子模型检测出其网络流量存在异常，从而可以有效提高模型训练装置检测异常的网络流量的准确率、提高异常检测的鲁棒性。
在一种可能的实施方式中，模型训练装置在对多个子模型分别输出的检测结果进行投票时，具体可以是根据目标投票策略，对多个子模型分别输出的检测结果进行投票，该目标投票策略包括多数投票策略、一票否决策略、或一票通过策略中的一种或者多种，或者可以是其它可适用的投票策略等。如此，利用基于目标投票策略，提高模型训练装置检测异常的网络流量的准确率、提高异常检测的鲁棒性。
在一种可能的实施方式中，模型训练装置还可以生成配置界面，该配置界面用于提示用户从多个候选投票策略中选择目标投票策略，或者该配置界面用于提示用户定义该目标投票策略。如此，模型训练装置可以支持用户对目标投票策略进行配置或者自定义，以此可以提高检测异常网络流量的灵活性。
在一种可能的实施方式中，模型训练装置还可以生成配置界面，该配置界面用于提示用户对多个子模型的数量、模型超参数或者无监督学习算法中的一种或者多种进行配置。如此，模型训练装置可以基于用户配置的子模型的数量完成网络流量的异常检测，或者基于用户配置的模型超参数或者无监督学习算法完成模型训练，支持用户对这些参数的自定义，提高方案实现的灵活性。
在一种可能的实施方式中，模型训练装置在利用多个子模型分别根据第二日志进行推理之前，可以先对模型池的子模型进行随机抽取，得到此次用于检测待检测实例的网络流量是否异常的多个子模型。如此，通过随机抽取子模型，可以降低部分识别异常网络流量的能力较差的子模型持续对模型训练装置的检测性能的影响。
在一种可能的实施方式中，模型训练装置在获取用于训练目标检测模型的第一日志时，具体可以是获取用户向云服务平台提供的针对待检测实例的网络流量的指示信息，如待检测实例的名称或者IP地址等，从而模型训练装置根据该指示信息，从待检测实例中采集第一日志。如此，模型训练装置可以实现对待检测示例进行网络流量异常检测的云服务。
在一种可能的实施方式中，模型训练装置在获取用于训练目标检测模型的第一日志时，具体可以是获取多个原始日志，该多个原始日志用于记录云服务平台监控的多个实例的网络流量，该多个实例包括待检测实例，从而根据该待检测实例的IP地址，从多个原始日志中筛选出第一日志。如此，云服务平台在提供针对多个实例的网络流量异常检测服务时，模型训练装置可以根据用户所指定的实例获取该实例对应的第一日志，以便根据该第一日志对用户指定的实例进行网络流量异常检测，以满足用户对于实例的网络流量的异常检测需求。
在一种可能的实施方式中，模型训练装置在获取用于训练目标检测模型的第一日志时，具体可以是获取待检测实例对应的多个候选日志，并利用去重条件，对多个候选日志进行去重，也即对具有相同类型的网络流量进行去重，使得用于训练检测模型的每种类型的网络流量可能保持在相近的水平，以此得到第一日志，该去重条件包括至少一种网络流量的类型标识，如可以是网络流量日志中记录的五元组信息，或者可以是基于五元组信息所计算出的值(如MD5值等)。如此，利用去重后的网络流量训练目标检测模型，可以提高目标检测模型识别部分数量较少的类型的网络流量的准确性。
在一种可能的实施方式中，模型训练装置在基于第一日志采用无监督学习算法训练得到目标检测模型时，具体可以是根据第一日志提取网络流量特征，该网络流量特征可以包括待检测实例的通信次数、目的IP地址数量、源IP地址数量、目的IP地址数量与端口数量总和、或源IP地址与目的IP地址与源端口与目的端口的信息熵中的一种或者多种，从而利用该网络流量特征采用无监督学习算法训练得到目标检测模型。如此，可以基于日志中的网络流量特征实现对目标检测模型的训练。
在一种可能的实施方式中，模型训练装置在利用网络流量特征采用无监督学习算法训练得到目标检测模型时，具体可以是对网络流量特征进行归一化，如将网络流量特征的取值限制在0至1之间的范围，得到目标网络流量特征，并利用该目标网络流量特征采用无监督学习算法训练得到目标检测模型。如此，即使从数据库中抽取的部分网络流量特征为存在异常的网络流量对应的网络流量特征，也能通过归一化处理减少该异常的网络流量特征对于目标检测模型的检测准确性的影响，从而可以进一步排除异常网络流量的干扰。
在一种可能的实施方式中，模型训练装置还可以基于第二日志，更新目标检测模型。如此，通过对目标检测模型进行动态更新，可以保持目标检测检测模型的检测效果持续有效。
第二方面，本申请实施例还提供了一种模型训练装置，该模型训练装置包括：日志获取模块，用于获取第一日志，第一日志用于记录待检测实例在第一时间段内的网络流量；模型训练模块，用于基于第一日志，采用无监督学习算法训练得到目标检测模型，目标检测模型用于检测待检测实例的网络流量是否存在异常，目标检测模型包括机器学习模型或人工智能AI模型，目标检测模型包括至少一个子模型。
在一种可能的实施方式中，日志获取模块，还用于获取第二日志，第二日志用于记录待检测实例在第二时间段内的网络流量；模型训练装置还包括异常检测模块，异常检测模块，用于利用目标检测模型，根据第二日志检测待检测实例在第二时间段内的网络流量是否存在异常。
在一种可能的实施方式中，目标检测模型包括多个子模型，异常检测模块，用于：利用多个子模型分别根据第二日志进行推理，得到多个子模型分别输出的检测结果，不同子模型具有不同的超参数、或具有不同的模型结构、或基于不同的无监督学习算法完成训练；对多个子模型分别输出的检测结果进行投票，确定待检测实例在第二时间段内的网络流量是否异常。
在一种可能的实施方式中，异常检测模块，用于：根据目标投票策略，对多个子模型分别输出的检测结果进行投票，目标投票策略包括多数投票策略、一票否决策略、或一票通过策略中的一种或者多种。
在一种可能的实施方式中，模型训练装置还包括：配置模块，用于生成配置界面，配置界面用于提示用户从多个候选投票策略中选择目标投票策略，或者配置界面用于提示用户定义目标投票策略。
在一种可能的实施方式中，模型训练装置还包括：配置模块，用于生成配置界面，配置界面用于提示用户对多个子模型的数量、模型超参数、或无监督学习算法中的一种或者多种进行配置。
在一种可能的实施方式中，异常检测模块，还用于在利用多个子模型分别根据第二日志进行推理之前，对模型池中的子模型进行随机抽取，得到多个子模型。
在一种可能的实施方式中，日志获取模块，用于：获取用户向云服务平台提供的针对待检测实例的网络流量的指示信息；根据指示信息，从待检测实例中采集第一日志。
在一种可能的实施方式中，日志获取模块，用于：获取多个原始日志，多个原始日志用于记录云服务平台监控的多个实例的网络流量，多个实例包括待检测实例；根据待检测实例的互联网协议IP地址，从多个原始日志中筛选出第一日志。
在一种可能的实施方式中，日志获取模块，用于：获取待检测实例对应的多个候选日志；利用去重条件，对多个候选日志进行去重，得到第一日志，去重条件包括至少一种网络流量的类型标识。
在一种可能的实施方式中，模型训练模块，用于：根据第一日志，提取网络流量特征，网络流量特征包括待检测实例的通信次数、目的互联网协议IP地址数量、源IP地址数量、目的IP地址数量与端口数量总和、或源IP地址与目的IP地址与源端口与目的端口的信息熵中的一种或者多种；利用网络流量特征采用无监督学习算法训练得到目标检测模型。
在一种可能的实施方式中，模型训练模块，用于：对于网络流量特征进行归一化，得到目标网络流量特征；利用目标网络流量特征采用无监督学习算法训练得到目标检测模型。
值得注意的是，第二方面提供的模型训练装置，对应于第一方面提供的模型训练方法，故第二方面以及第二方面中任一实施方式所具有的技术效果，可参见第一方面或者第一方面的相应实施方式所具有的技术效果。
第三方面，本申请提供一种计算设备集群，所述计算设备包括至少一个计算设备，所述至少一个计算设备包括至少一个处理器和至少一个存储器；所述至少一个存储器用于存储指令，所述至少一个处理器执行所述至少一个存储器存储的该指令，以使所述计算设备集群执行上述第一方面或第一方面任一种可能实现方式中的模型训练方法。需要说明的是，该存储器可以集成于处理器中，也可以是独立于处理器之外。所述至少一个计算设备还可 以包括总线。其中，处理器通过总线连接存储器。其中，存储器可以包括可读存储器以及随机存取存储器。
第四方面，本申请提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在至少一个计算设备上运行时，使得所述至少一个计算设备执行上述第一方面或第一方面的任一种实现方式所述的方法。
第五方面，本申请提供了一种包含指令的计算机程序产品，当其在至少一个计算设备上运行时，使得所述至少一个计算设备执行上述第一方面或第一方面的任一种实现方式所述的方法。
本申请在上述各方面提供的实现方式的基础上，还可以进行进一步组合以提供更多实现方式。
附图说明
图1为本申请实施例提供的一示例性应用场景示意图；
图2为本申请实施例提供的另一示例性应用场景示意图；
图3为本申请实施例提供的一种模型训练方法的流程示意图；
图4为本申请实施例提供的一种对待检测实例的网络流量进行异常检测的方法流程示意图；
图5为本申请实施例提供的对多个检测结果进行投票的示意图；
图6为本申请实施例提供的一种配置界面的示意图；
图7为本申请实施例提供的另一种对待检测实例的网络流量进行异常检测的方法流程示意图；
图8为本申请实施例提供的一种计算设备的结构示意图；
图9为本申请实施例提供的一种计算设备集群的结构示意图。
具体实施方式
下面将结合本申请中的附图，对本申请提供的实施例中的方案进行描述。
目前，网络中的实例(如主机、网关、虚拟机等)在运行过程中，可能会受到网络攻击，如恶意软件攻击、中间人攻击、拒绝服务攻击、注入攻击、零日漏洞攻击、域名系统(domain name system，DNS)隧道攻击等，相应地，该实例的网络流量会因为网络攻击而产生异常。其中，网络流量，是指能够连接网络的实例在网络中产生的数据流量。通常情况下，可以通过检测实例的网络流量是否满足预先设定的危害指标的方式，来判定该实例是否受到网络攻击。比如，当实例在一段时间内的网络流量突增至正常状态下的几倍或者几十倍，或者实例在一段时间段内访问其它实例的数量超出阈值等，则可以确定该实例受到网络攻击。这样，通过及时感知实例受到网络攻击并进行运维，可以减少该实例因为遭受网络攻击而产生的损失。
但是，预先设定的危害指标容易被有经验的网络攻击者(如黑客等)使用对抗手段进行绕过，或者难以适用部分网络攻击者所采用的新型网络攻击手段，如当网络攻击者修改数据包中的端口号等信息时，基于网络流量大小或者访问实例的数量上限等危害指标，通常难以检测出该实例已遭受网络攻击，从而导致检测出异常的网络流量的准确率较低。
基于此，本申请实施例提供了一种模型训练方法，该方法可以由相应的模型训练装置 执行，用于训练得到用于检测实例的网络流量是否存在异常的目标检测模型，以便利用目标检测模型来提高检测异常网络流量的准确率。具体地，模型训练装置获取用于记录待检测实例正在第一时间段内的网络流量的第一日志，并基于该第一日志对目标检测模型进行无监督训练，所训练得到的目标检测模型，用于检测该待检测实例的网络流量是否存在异常，该目标检测模型可以是机器学习模型或者人工智能(artificial intelligence，AI)模型，并且，目标检测模型包括至少一个子模型。
由于模型训练装置训练得到的目标检测模型，能够用于对待检测实例的网络流量进行检测，从而无需利用危害指标来判断待检测实例的网络流量是否异常，这可以摆脱危害指标所具有的局限性对异常网络流量的检测准确性的影响，从而实现提高检测异常的网络流量的准确率。
并且，由于目标检测模型是利用待检测实例的网络流量日志通过无监督学习的方式训练得到，无需利用人工标注的训练样本进行训练，从而不仅可以减少人工标注成本，而且，也能提高目标检测模型的训练效率。
作为一种示例，上述模型训练装置可以被部署于云端，用于为用户侧的设备提供生成用于检测该设备是否存在网络流量异常的检测模型的云服务。例如，在图1所示的应用场景中，模型训练装置100部署于云端，如公有云、私有云、边缘云等，并且，可以由云端的计算设备或者计算设备集群实现，能够为用户侧的多个主机分别提供生成用于检测各个主机是否存在网络流量异常的检测模型的服务，图1中以为主机201至主机204提供云服务为例进行示例性说明。
如图1所示，模型训练装置100可以包括日志获取模块101、模型训练模块102。其中，日志获取模块101，用于获取记录主机201(或者其它主机)的网络流量的日志，并将获取的日志提供给模型训练模块102。模型训练模块102，用于根据接收到的日志采用无监督学习算法训练目标检测模型，该目标检测模型用于检测主机201的网络流量是否存在异常。进一步地，模型训练装置100还可以包括其它功能模块，如还可以包括异常检测模块103、告警模块104、配置模块105等。其中，异常检测模块103，用于利用完成无监督训练的目标检测模型，根据日志获取模块101提供的日志检测主机201的网络流量是否存在异常。告警模块104，用于在确定主机201的网络流量存在异常后执行相应的告警操作等。配置模块105，用于对模型训练装置100进行相应的信息配置，如配置训练目标检测模型所采用的无监督学习算法。
作为另一种示例，上述模型训练装置可以被部署于本地，从而可以为用户提供本地的训练目标检测模型(以及利用目标检测模型检测异常网络流量)的服务。例如，在图2所示的应用场景中，模型训练装置100在用户侧独立于终端200进行部署，并与终端200建立连接；或者，模型训练装置100也可以集成于用户的终端200等，从而模型训练装置100可以在本地生成用于检测终端200的网络流量是否存在异常的目标检测模型，以便利用该目标检测模型判断终端200是否遭受网络攻击。
实际应用时，上述模型训练装置100可以通过软件实现，或者可以通过硬件实现。
模型训练装置100作为软件功能单元的一种举例，可以包括运行在计算实例上的代码。其中，计算实例可以包括裸金属服务器(bare metal server，BMS)、虚拟机、容器中的至少 一种。进一步地，上述计算实例可以是一台或者多台。例如，模型训练装置100可以包括运行在多个BMS/虚拟机/容器上的代码。需要说明的是，用于运行该代码的多个BMS/虚拟机/容器可以分布在相同的区域(region)中，也可以分布在不同的region中。进一步地，用于运行该代码的多个BMS/虚拟机/容器可以分布在相同的可用区(availability zone，AZ)中，也可以分布在不同的AZ中，每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中，通常一个region可以包括多个AZ。
同样，用于运行该代码的多个BMS/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud，VPC)中，也可以分布在多个VPC中。其中，通常一个VPC设置在一个region内，同一region内两个VPC之间，以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关，经通信网关实现VPC之间的互连。
模型训练装置100作为硬件功能单元的一种举例，模型训练装置100可以包括至少一个计算设备，如服务器等。或者，模型训练装置也可以是利用专用集成电路(application-specific integrated circuit，ASIC)实现、或可编程逻辑器件(programmable logic device，PLD)实现的设备等。其中，上述PLD可以是复杂程序逻辑器件(complex programmable logical device，CPLD)、现场可编程门阵列(field-programmable gate array，FPGA)、通用阵列逻辑(generic array logic，GAL)或其任意组合实现。
模型训练装置100包括的多个计算设备可以分布在相同的region中，也可以分布在不同的region中。模型训练装置100包括的多个计算设备可以分布在相同的AZ中，也可以分布在不同的AZ中。同样，模型训练装置100包括的多个计算设备可以分布在同一个VPC中，也可以分布在多个VPC中。其中，上述多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。
接下来，对网络流量检测过程的各种非限定性的具体实施方式进行详细描述。
参阅图3，为本申请实施例中一种网络流量检测方法的流程示意图。该方法可以应用于上述图1或者图2所示的应用场景中，或者也可以是应用于其它可适用的应用场景中。为便于理解，下面以应用于图1所示的应用场景为例进行说明。图3所示的网络流量检测方法具体可以包括：
S301：日志获取模块101获取第一日志，该第一日志用于记录待检测实例在第一时间段内的网络流量。
其中，待检测实例，可以是网络中能够进行数据交互并产生网络流量的实例，包括虚拟实例或者物理实例。其中虚拟实例，例如可以是虚拟机、容器或者BMS等。物理实例，例如可以是用户的主机、网络中的网关、服务器等，本实施例对此并不进行限定。
通常情况下，待检测实例在运行过程中，通常会基于其与其它实例进行数据通信所产生的网络流量生成相应的日志，该日志用于记录待检测实例的网络流量。通常情况下，所生成的日志，可以包括待检测实例与其它实例进行数据通信的时间戳、五元组信息、流量大小、服务质量(quality of service，QoS)、通信次数等信息。其中，五元组信息，可以包括源互联网协议(internet protocol，IP)地址、目的IP地址、源端口、目的端口、通信协议。如此，模型训练装置100可以通过对该日志进行分析，以实现对该待检测实例的网络流量进行异常检测。
具体实现时，日志获取模块101先获取用于记录待检测实例在一段时间内的网络流量的日志，为便于区分和描述，以下称之为第一日志，所获取的第一日志用于记录待检测实例在第一时间段内的网络流量，该第一日志的数量可以是一个或者多个。
其中，当模型训练装置100为待检测实例提供本地化的服务时，日志获取模块101可以通过访问该待检测实例中相应的存储区域，获得待检测实例存储的日志。
而当模型训练装置100为待检测实例提供云服务时，由于模型训练装置100通常可以同时为多个实例进行网络流量的异常检测，因此，日志获取模块101可以获取云服务平台监控的多个实例发送的多个原始日志，该多个原始日志用于记录该多个实例(其中一个实例为上述待检测实例)的网络流量，从而日志获取模块101可以根据待检测实例的IP地址或者其它用于标识待检测实例的信息，从多个原始日志中筛选出属于待检测实例的第一日志。实际应用时，日志获取模块101可以将用户指定的实例确定为待检测实例，并从多个原始日志中确定该待检测实例对应的第一日志；或者，日志获取模块101可以轮询检测多个实例中的各个实例是否存在网络流量异常，此时，日志获取模块101可以将当前轮询的实例确定为待检测实例，并进一步从多个原始日志中确定待检测实例对应的第一日志。
或者，日志获取模块101可以获取用户向模型训练装置100所在的云服务平台提供的针对待检测实例的网络流量的指示信息，如可以包括待检测实例的IP地址、网络流量的时间段等信息，从而日志获取模块101可以根据该指示信息，从待检测实例中采集第一日志，以便在用户的指示下，针对性的获取该待检测实例对应的第一日志。
本实施例中，模型训练装置100可以利用人工智能(artificial intelligence，AI)模型或者机器学习(machine learning，ML)模型实现对待检测实例的网络流量进行异常检测，并且，该AI模型或者ML模型可以基于无监督学习算法训练得到。为此，日志获取模块101在获得待检测实例对应的第一日志后，可以将该第一日志发送给模型训练模块102。
S302：模型训练模块102基于第一日志，采用无监督学习算法训练得到目标检测模型，该目标检测模型用于检测该待检测实例的网络流量是否存在异常，该目标检测模型包括ML模型或AI模型，该目标检测模型包括至少一个子模型。
其中，目标检测模型可以包括一个或者多个子模型，每个子模型可以是ML模型，如孤立森林模型、基于一类支持向量机(one-class support vector machine，OCSVM)算法构建的模型等；或者，目标检测模型中的子模型可以是AI模型，如基于神经网络(neural networks)构建的模型。或者，当目标检测模型包括多个子模型时，目标检测模型中的部分子模型可以是ML模型，另一部分子模型可以是AI模型，本实施例对此并不进行限定。
在利用第一日志对目标检测模型进行无监督训练时，模型训练模块102可以先将第一日志转换为能够适用于目标检测模型输入的数据。
在一种可能的实施方式中，模型训练模块102可以先提取第一日志中的五元组以及时间戳等信息，并从该第一日志中提取网络流量特征。或者，模型训练模块102可以根据预先设定的滑动时间窗的长度(如10分钟、30分钟等)，确定第一日志中位于该滑动时间窗内的日志，并进一步从位于该滑动时间窗内的日志中提取网络流量特征。实际应用时，模型训练模块102可以将第一日志或者位于滑动时间窗内的日志组成一个会话，并对该会话进行特征提取，得到该网络流量特征。
示例性地，所提取的网络流量特征，例如可以是以下特征中的一种或者多种：
1、待检测实例与其它实例之间的通信次数。
具体地，模型训练模块102可以根据第一日志(或者位于该滑动时间窗内的日志)，统计待检测实例在第一时间段内或者在该滑动时间窗内与其它实例之间的通信次数。
2、第一日志(或者位于该滑动时间窗内的日志)包括的目的IP地址的数量。
具体地，模型训练模块102可以根据从第一日志(或者位于该滑动时间窗内的日志)中提取出的五元组信息，统计目的IP地址的数量。
3、第一日志(或者位于该滑动时间窗内的日志)包括的目的端口的数量。
4、第一日志(或者位于该滑动时间窗内的日志)包括的目的IP地址的数量与目的端口的数量总和。
5、第一日志(或者位于该滑动时间窗内的日志)中的源IP地址、目的IP地址、源端口、目的端口之间的信息熵。
实际应用时，所提取的网络流量特征，也可以是其它类型的特征，如可以是待检测实例的IP地址作为源IP地址时对应的通信次数、目的IP地址的数量、目的端口的数量、目的IP地址的数量与目的端口的数量总和等，本实施例对此并不进行限定。
实际应用场景中，模型训练模块102所接收到的第一日志中可能存在部分日志的信息存在错误，如部分日志中的五元组信息含有缺省值(即日志中的参数值未及时更新，其为初始化状态的值)。为此，模型训练模块102可以在获取到第一日志后，可以对该第一日志进行过滤，如过滤包括缺省值的日志等，并从过滤后的日志中提取出网络流量特征。在其它实施例中，也可以是由日志获取模块101对采集到的待检测实例对应的日志进行过滤，如过滤五元组信息中含缺省值的日志等。
然后，模型训练模块102可以利用提取出的网络流量特征训练预先构建的目标网络模型。例如，模型训练装置102可以将网络流量特征输入至目标检测模型包括的各个子模型中，并利用预设的无监督学习算法对目标检测模型中的各个子模型进行无监督训练。其中，无监督学习算法，例如可以是k-means聚类算法、随机森林算法、基于密度的噪声应用空间聚类(density-based spatial clustering of applications with noise，DBSCAN)算法、或支持向量数据描述(support vector domain description，SVDD)算法中的一种或者多种，或者可以是其它类型的算法，本实施例对此并不进行限定。其中，当目标检测模型包括多个子模型时，训练每个子模型所采用的无监督学习算法可以相同，也可以不同，本实施例对此并不进行限定。
实际应用时，模型训练模块102可以针对待检测实例对应的多个日志，按照上述类似方式提取得到多组网络流量特征，并将该多组网络流量特征存储至数据库中，如Redis数据库等，从而在训练目标检测模型时，模型训练模块102可以从该数据库中随机抽取部分网络流量特征作为训练样本，并利用抽取的网络流量特征对目标检测模型进行无监督训练。通常情况下，待检测实例的网络流量绝大多数(或者全部网络流量)都是正常的流量数据，相应地，基于待检测实例对应的多个日志所提取出的网络流量特征，绝大多数(或者全部网络流量特征)都是正常网络流量对应的网络流量特征。如此，即使数据库中混入了少量的异常网络流量对应的特征，模型训练模块102通过随机抽取较少样本量的网络流量特征对目 标检测模型进行训练，可以降低利用异常网络流量对应的特征训练目标检测模型的概率，从而可以有效降低数据库中的异常网络流量对于目标检测模型的检测准确性的影响。示例性地，模型训练模块102可以从数据库中随机抽取3％的网络流量特征，作为训练目标检测模型的样本。其中，数据库中存储的网络流量特征，可以在完成对目标检测模型的训练后进行释放；或者，可以在存储网络流量特征时为其设置生命周期，从而当网络流量特征在数据库中的存储时长达到该生命周期的时长，从数据库中释放该网络流量特征，以便减少网络流量特征对于存储资源的占用。
进一步地，模型训练模块102在利用网络流量特征训练目标检测模型之前，还可以对网络流量特征进行归一化，如将网络流量特征的取值限制在0至1之间的范围，然后，模型训练模块102再利用经过归一化处理后所得到的目标网络流量特征训练目标检测模型。如此，即使从数据库中抽取的部分网络流量特征为存在异常的网络流量对应的网络流量特征，也能通过归一化处理减少该异常的网络流量特征对于目标检测模型的检测准确性的影响，从而可以进一步排除异常网络流量的干扰。
实际应用时，在训练得到的目标检测模型后，还可以利用异常的网络流量数据(或者异常的网络流量数据+正常的网络流量数据)对目标检测模型中的各个子模型进行测试，以校验目标检测模型中的各个子模型识别异常的网络流量的准确率是否达到预设要求，如校验目标检测模型中的各个子模型的识别准确率是否达到阈值(如90％)等。
进一步地，当目标检测模型的准确率达到预设要求时(即所有子模型的准确率均达到预设要求)，模型训练装置100可以利用该目标检测模型进行异常网络流量的检测；而当目标检测模型中存在部分或者全部子模型的准确率未达到预设要求时，模型训练模块102可以调整目标检测模型中未达到预设要求的子模型的超参数(如学习率等)继续对该子模型进行训练，直至训练得到满足准确率要求的目标检测模型。如此，在训练得到目标检测模型后，即可利用该目标检测模型对待检测实例的网络流量进行异常检测。基于此，模型训练模块102在训练得到的目标检测模型后，可以将其提供给异常检测模块103，并且，本实施例还可以进一步包括如下步骤S303以及步骤S304。
S303：日志获取模块101获取第二日志，该第二日志用于记录待检测实例在第二时间段内的网络流量。
本实施例中，模型训练装置100可以利用一段时间内的日志训练的目标检测模型，再利用完成训练的目标检测模型对待检测实例在另外一段时间内的网络流量进行异常检测。为便于描述，以下将日志获取模块101获取的、用于记录待检测实例在第二时间段内的网络流量的日志，称之为第二日志。其中，第二时间段与第一时间段为不同的时间段，并且，第二时间段可以晚于第一时间段，或者第二时间段也可以早于第一时间段。
然后，日志获取模块101将获取的第二日志提供给异常检测模块103。
S304：异常检测模块103利用目标检测模型，根据第二日志检测待检测实例在第二时间段内的网络流量是否存在异常。
具体实现时，异常检测模块103可以从第二日志中提取网络流量特征，并将其输入至目标检测模型，得到目标检测模型的推理结果，该推理结果用于指示待检测实例在第二时间段内的网络流量是否存在异常。其中，异常检测模块103从第二日志中提取网络流量特征的 过程，可参见前述从第一日志中提取网络流量特征的相关之处描述，在此不做赘述。
其中，当目标检测模型包括一个子模型时，异常检测模块103可以根据该子模型的推理结果判定待检测实例在第二时间段内的网络流量是否存在异常；而当目标检测模型包括多个子模型时，异常检测模块103可以将第二日志对应的网络流量日志分别输入至各个子模型中，以此得到各个子模型分别输出的推理结果，从而异常检测模块103可以根据多个子模型分别输出的推理结果判定待检测实例在第二时间段内的网络流量是否存在异常，其具体判定过程可参见图4所示实施例的相关之处描述，在此不做赘述。
进一步地，当异常检测模块103检测到待检测实例的网络流量存在异常时，还可以将其通知给告警模块104，从而告警模块104可以执行相应的告警策略。
作为一种实现示例，当模型训练装置100部署于用户侧时，告警模块104可以生成告警界面，并在模型训练装置100的显示器中显示该告警界面，以便提示用户待检测实例的网络流量存在异常。
作为另一种实现示例，当模型训练装置100部署于云端时，告警模块104可以生成针对该待检测实例的告警报文，该告警报文中携带待检测实例的标识，进一步地，该告警报文中还可以包括网络流量异常的详细信息，如异常类型、异常数据等。然后，告警模块104可以向模型训练装置100对外提供的客户端发送该告警报文，或者向预先指定的终端实例发送该告警报文，以便客户端或者该终端实例提示用户待检测实例当前存在网络流量异常。
如此，模型训练装置100通过利用训练得到的目标检测模型对待检测实例的网络流量进行检测，而并非是利用危害指标来判断待检测实例的网络流量是否异常，这可以摆脱危害指标所具有的局限性对异常网络流量的检测准确性的影响，从而实现提高模型训练装置100检测异常的网络流量的准确率。并且，由于目标检测模型是利用待检测实例的网络流量日志通过无监督学习的方式训练得到，无需利用人工标注的训练样本进行训练，从而不仅可以减少人工标注成本，而且，也能提高目标检测模型的训练效率。
当目标检测模型包括一个子模型时，模型训练装置100利用该子模型输出的推理结果，判定待检测实例的网络流量是否存在异常；当目标检测模型包括多个子模型时，模型训练装置100可以利用多个子模型分别输出的推理结果综合判定待检测实例的网络流量是否存在异常，以便进一步提高网络流量异常检测的准确性，提高用户体验。下面，结合图4对利用多个子模型对待检测实例进行网络流量异常检测的具体实现过程进行示例性说明。
参见图4，示出了本申请提供的一种对待检测实例的网络流量进行异常检测的方法流程示意图，该方法具体可以包括：
S401：日志获取模块101获取第一日志，该第一日志用于记录待检测实例在第一时间段内的网络流量。
S402：模型训练模块102基于第一日志对多个不同的子模型进行无监督训练，该多个子模型均用于检测该待检测实例的网络流量是否存在异常，不同子模型具有不同的超参数、或具有不同的模型结构、或基于不同的无监督学习算法完成训练。
本实施例中，模型训练模块102中可以配置有多个子模型。其中，不同子模型中的超参数存在差异，如子模型A的学习率为0.01，而子模型B的学习率为0.001，子模型C的学习率 为0.0001等，从而模型训练模块102利用第一日志分别对多个超参数不同的子模型进行无监督训练，得到多个基于不同超参数的子模型。比如，模型训练模块102可以通过网格搜索(grid search)的策略生成多个超参数，并基于该多个超参数设置多个不同的子模型，以便利用第一日志分别训练该多个不同的子模型。
或者，不同子模型中的模型结构存在差异，如子模型P采用树形结构，子模型Q采用神经网络结构，从而模型训练模块102利用第一日志分别对各个子模型进行无监督训练，得到多个具有不同模型结构的子模型。
又或者，模型训练模块102利用第一日志，采用不同的无监督学习算法对多个子模型分别进行训练，如采用k-means聚类算法对子模型X进行无监督训练，采用随机森林算法对子模型Y进行训练，采用DBSCAN算法对子模型Z进行训练等。
实际应用时，模型训练模块102可以对多个模型超参数、多种模型结构、多种无监督学习算法进行组合，并基于每种组合训练得到一个子模型。其中，模型训练模块102利用第一日志训练每个子模型的具体实现方式，可参见前述实施例中步骤S302的相关之处描述，在此不做赘述。
模型训练模块102在训练得到多个子模型后，可以将该多个子模型保存至模型池中并进行持久化存储。
S403：日志获取模块101获取第二日志，该第二日志用于记录待检测实例在第二时间段内的网络流量。
S404：异常检测模块103从模型池中随机抽取N个子模型。
其中，N为大于1的正整数，并且，N的取值小于模型池中的子模型的总数。实际应用时，N可以为奇数。可以理解，所抽取的N个子模型，即为前述实施例中的目标检测模型中的多个子模型。
S405：异常检测模块103利用随机抽取的N个子模型分别根据第二日志进行推理，得到N个子模型分别输出的检测结果。
其中，每个子模型所输出的检测结果，用于指示该子模型所判定的待检测实例在第二时间段内的网络流量是否异常，如此，异常检测模块103可以得到N个检测结果。
如图5所示，异常检测模块103可以利用从模型池随机抽取的子模型1至子模型N分别根据第二日志进行推理，得到N个检测结果，即检测结果1至检测结果N，并将该N个检测结果添加至投票池中，以便后续通过对N个检测结果进行投票，以确定待检测实例的网络流量是否存在异常。
S406：异常检测模块103对N个子模型分别输出的检测结果进行投票，确定该待检测实例在第二时间段内的网络流量是否异常。
实际应用场景中，N个子模型所输出的检测结果可能并不一致，如部分子模型输出的检测结果指示待检测实例的网络流量存在异常，另一部分子模型输出的检测结果指示待检测实例的网络流量正常，此时，异常检测模块103可以通过对投票池中的N个检测结果进行投票，并根据投票结果最终判定待检测实例在第二时间段内的网络流量是否存在异常。
示例性地，异常检测模块103可以基于目标投票策略，对N个子模型分别输出的检测结果进行投票。其中，目标投票策略包括多数投票策略、一票否决策略、或者一票通过策略 中的一种或者多种，或者可以是其它可适用的投票策略，又或者可以是用户自定义的投票策略等。
其中，多票投票策略，是指遵从少数服从多数的原则进行投票，将投票数量较多的结果作为最终的结果。比如，假设异常检测模块103对9个子模型输出的检测结果进行投票，其中，存在3个子模型输出的检测结果指示待检测实例的网络流量正常，而剩余6个子模型输出的检测结果指示待检测实例的网络流量存在异常，则基于少数服从多数的原则，异常检测模块103可以最终确定待检测实例的网络流量存在异常。
一票否决策略，是指当任意一个检测结果指示待检测实例的网络流量存在异常时，异常检测模块103即可最终确定待检测实例的网络流量存在异常。而当所有的检测结果均指示待检测实例的网络流量正常时，异常检测模块103才会最终确定待检测实例的网络流量不存在异常。
一票通过策略，是指当任意一个检测结果指示待检测实例的网络流量不存在异常时，异常检测模块103即可最终确定待检测实例的网络流量正常。而当所有的检测结果均指示待检测实例的网络流量存在异常时，异常检测模块103才会最终确定待检测实例的网络流量存在异常。
如此，即使部分子模型未能检测出待检测实例的网络流量存在异常，异常检测模块103也能基于多个子模型中的其它子模型检测出其网络流量存在异常，从而可以有效提高模型训练装置100检测异常的网络流量的准确率、提高异常检测的鲁棒性。
并且，通过从模型池中随时随机抽取N个子模型，可以降低部分识别异常网络流量的能力较差的子模型持续对模型训练装置100的检测性能的影响。
进一步地，当异常检测模块103检测到待检测实例的网络流量存在异常时，还可以将其通知给告警模块104，从而告警模块104可以执行相应的告警策略。
本实施例中，模型训练装置100还可以支持用户对目标投票策略进行配置。具体实现时，模型训练装置100还可以包括配置模块105，并且，配置模块105用于生成配置界面，该配置界面用于提示用户从多个候选投票策略中选择目标投票策略，并将该配置界面通过模型训练装置100对外提供的客户端呈现给用户，或者将该配置界面通过模型训练装置100(其部署于用户侧)包括的显示器呈现给用户。例如，配置模块105可以生成如图6所示的配置界面，该配置界面中包括多个可供用户选择的候选投票策略，如图6中的候选投票策略1至候选投票策略4等，从而用户可以多个候选投票策略中选择一个候选投票策略用于进行异常网络流量检测；相应地，配置模块105可以将用户所选择的候选投票策略确定为目标投票策略，并将其提供给异常检测模块103进行检测。
或者，配置模块105所生成的配置界面，可以用于提示用户自定义投票策略，即自定义根据多个子模型的检测结果确定最终的检测结果的投票策略，如图6所示定义策略名称和策略内容等。举例来说，用户可以定义策略内容为：当超出85％的检测结果均表示待检测实例的网络流量存在异常时，确定最终的检测结果为网络流量异常等，本实施例对此并不进行限定。
在进一步可能的实施方式中，模型训练装置100还可以支持用户对其它信息进行配置。比如，用户可以在图6所示的配置界面上不仅可以对投票策略进行配置，还可以对异常检测 模块103每次使用的子模型的数量(也即N的值)、各个子模型的超参数、或训练各个子模型所采用的无监督学习算法中的一种或者多种进行配置。或者，用户配置投票策略的界面，可以与用户配置子模型的数量、模型超参数、以及无监督学习算法的界面为两个独立的界面等。值得注意的是，上述信息配置的实现方式仅作为一些示例性说明，在其它实施例中，用户也可以是采用其它方式进行信息配置，或者用户可以配置其它类型的信息，如配置模型池中子模型的数量上限/下限、每个子模型的尺寸上限等。
实际应用时，由于待检测实例在正常运行过程中的网络流量的分布可能会发生动态变化，如待检测实例在0:00-12：00时与服务器1至服务器10之间交互较多数量的数据包，而在12:00-24:00时与服务器11至服务器20之间交互较多数量的数据包等。基于此，模型训练装置100针对待检测实例进行异常网络流量的检测时，可以针对待检测实例在不同时间段内的流量分布训练与该时间段相匹配的一个或者多个子模型，从而在不同时间段利用不同的子模型进行异常网络流量的检测。或者，模型训练装置100可以对已训练的一个或者多个子模型进行动态更新，例如当确定待检测实例在过去的1小时或者24小时内的网络流量不存在异常时，模型训练装置100可以利用这段时间段内的网络流量自动更新模型池中的一个或者多个子模型等，以便于模型池中的子模型能够通过动态更新实现自适应待检测实例的网络流量分布的变化，从而可以保障子模型的检测效果持续有效。或者，模型训练装置100在基于第二日志判定待检测实例未发生网络流量异常后，可以利用该第二日志对目标检测模型中的多个子模型进行动态更新。
上述图3、图4所示的实施例中，模型训练装置100可以基于全量的网络流量的日志训练目标检测模型中的各个子模型，而在其它可能的实施例中，对于待检测实例与其它实例之间重复的网络流量，模型训练装置100可以通过对这些网络流量进行去重，并基于去重后的网络流量训练子模型，以此可以提高子模型的推理准确性。下面，结合图7进行示例性说明。
参见图7，示出了本申请提供的另一种对待检测实例的网络流量进行异常检测的方法流程示意图，该方法具体可以包括：
S701：日志获取模块101获取多个候选日志，该多个候选日志用于记录待检测实例在第一时间段内的网络流量。
其中，步骤S701的具体实现过程，可参见前述步骤S301处的相关之处描述，在此不做赘述。
S702：日志获取模块101利用去重条件，对该多个候选日志进行去重，得到第一日志，该去重条件包括至少一种网络流量的类型标识。
实际应用场景中，待检测实例可能与外部同一实例之间存在较多数量的数据交互，从而产生较多相同类型的网络流量。此时，基于大量相同类型的网络流量训练子模型，可能会因为不同类型的网络流量之间的样本差异较大，影响子模型对于少量样本的网络流量的识别准确率。
为此，本实施例中，日志获取模块101可以通过对多个候选日志进行去重，也即对具有相同类型的网络流量进行去重，使得用于训练目标检测模型的每种类型的网络流量可能保持在相近的水平。这样，利用去重后的网络流量训练目标检测模型，可以提高目标检测模 型识别部分数量较少的类型的网络流量的准确性。
具体实现时，日志获取模块101利用去重条件，对多个候选日志进行去重，该去重条件包括至少一种网络流量的类型标识。其中，网络流量的类型标识，例如可以是网络流量日志中记录的五元组信息，对于具有同一五元组信息的不同网络流量，可以将其视为属于同一类型的网络流量；或者，网络流量的类型标识，例如可以是基于五元组信息所计算出的值，如基于MD5信息摘要算法(MD5 mssage-digest algorithm)所计算出的MD5值等。
这样，日志获取模块101可以根据候选日志确定每条网络流量对应的类型标识，并根据该类型标识对多个候选日志进行去重，具体是对多个候选日志中记录的网络流量属于同一类型的日志进行去重，得到去重后的日志，为便于描述，以下将去重后所得到的日志称之为第一日志。如此，去重后所得到的第一日志中所记录的不同类型的网络流量的数量可以保持在相近水平。
在其它可能的实施例中，去重条件，在包括网络流量对应的类型标识的基础上，还可以包括每种类型标识的网络流量数量上限。这样，在对网络流量进行过滤时，当候选日志中记录的一种类型的网络流量的总数未超出该类型的网络流量数量上限，则对该类型的网络流量进行去重。而当候选日志中记录的一种类型的网络流量的总数超出该类型的网络流量数量上限，表明该类型的网络流量可能存在异常，如收到网络攻击而导致该类型的网络流量突增等，从而模型训练装置100可以针对该类型的网络流量进行异常告警。
进一步地，当日志获取模块101根据候选日志中所记录的网络流量计算出的新的类型标识时，如果后续确定该类型标识所指示的网络流量属于正常网络流量，则日志获取模块101还可以将该类型标识添加至去重条件中，以对去重条件进行自动化更新。
S703：模型训练模块102基于第一日志对多个不同的子模型进行无监督训练，该多个子模型均用于检测该待检测实例的网络流量是否存在异常，不同子模型具有不同的超参数、或具有不同的模型结构、或基于不同的无监督学习算法完成训练。
S704：日志获取模块101获取第二日志，该第二日志用于记录待检测实例在第二时间段内的网络流量。
S705：异常检测模块103从模型池中随机抽取N个子模型。
所抽取的N个子模型，即为上述目标检测模型。
S706：异常检测模块103利用随机抽取的N个子模型分别根据第二日志进行推理，得到N个子模型分别输出的检测结果。
S707：异常检测模块103对N个子模型分别输出的检测结果进行投票，确定该待检测实例在第二时间段内的网络流量是否异常。
其中，步骤S703至步骤S707的具体实现过程，可参见前述实施例中的步骤S401至步骤S406的相关之处描述，在此不做赘述。
本实施例中，通过对候选日志进行过滤，也即对候选日志中的网络流量进行过滤，可以减小作为目标检测模型中的子模型的不同类型的训练样本之间的数量差异，从而训练得到的子模型能够对多种类型的网络流量均能实现较为准确的异常检测。
需要说明的是，本实施例中，对于模型训练装置100内的各个模块的划分以及功能描述仅作为一种示例。比如，在其他实施例中，日志获取模块101可以用于执行上述模型训练方 法中的任意步骤，类似的，模型训练模块102、异常检测模块103、告警模块104、配置模块105均可以用于执行上述图3、图4以及图7所示实施例中的任意步骤，并且，日志获取模块101、模型训练模块102、异常检测模块103、告警模块104、配置模块105负责实现的步骤可根据需要指定，通过日志获取模块101、模型训练模块102、异常检测模块103、告警模块104、配置模块105分别实现模型训练方法中不同的步骤来实现模型训练装置100的全部功能。
上述各实施例中，针对模型训练过程中所涉及到的模型训练装置(包括上述日志获取模块101、模型训练模块102、异常检测模块103、告警模块104、配置模块105)可以是配置于计算设备或者计算设备集群上的软件，并且，通过在计算设备或者计算设备集群上运行该软件，可以使得计算设备或者计算设备集群实现上述模型训练装置所具有的功能。下面，基于硬件设备实现的角度，对模型训练的过程中所涉及的模型训练装置进行详细介绍。
图8示出了一种计算设备的结构示意图，上述模型训练装置可以部署在该计算设备上，该计算设备可以是云环境中的计算设备(如服务器)，或边缘环境中的计算设备，或终端设备等具体可以用于实现上述各实施例中的日志获取模块101、模型训练模块102、异常检测模块103、告警模块104、配置模块105的功能。
如图8所示，计算设备800包括处理器820、存储器810、通信接口830和总线840。处理器820、存储器810和通信接口830之间通过总线840通信。总线840可以是外设部件互连标准(peripheral component interconnect，PCI)总线或扩展工业标准结构(extended industry standard architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图8中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。通信接口830用于与外部通信，例如接收用于记录待检测实例在第一时间段内的网络流量的第一日志等。
其中，处理器820可以为中央处理器(central processing unit，CPU)、专用集成电路(application specific integrated circuit,ASIC)、图形处理器(graphics processing unit,GPU)或者一个或多个集成电路。处理器820还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，模型训练装置中各个模块的功能可以通过处理器820中的硬件的集成逻辑电路或者软件形式的指令完成。处理器820还可以是通用处理器、数据信号处理器(digital signal process,DSP)、现场可编程逻辑门阵列(field programmable gate array,FPGA)或者其他可编程逻辑器件，分立门或者晶体管逻辑器件，分立硬件组件，可以实现或者执行本申请实施例中公开的方法、步骤及逻辑框图。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，结合本申请实施例所公开的方法可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器、闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器810，处理器820读取存储器810中的信息，结合其硬件完成模型训练装置中的部分或全部功能。
存储器810可以包括易失性存储器(volatile memory)，例如随机存取存储器(random access memory，RAM)。存储器810还可以包括非易失性存储器(non-volatile memory)，例如只读存储器(read-only memory，ROM)，快闪存储器，HDD或SSD。
存储器810中存储有可执行代码，处理器820执行该可执行代码以执行前述模型训练装 置100所执行的方法。
具体地，在实现图3、图4以及图7所示实施例的情况下，且图3、图4以及图7所示实施例中所描述的日志获取模块101、模型训练模块102、异常检测模块103、告警模块104、配置模块105为通过软件实现的情况下，执行图3、图4以及图7中的日志获取模块101、模型训练模块102、异常检测模块103、告警模块104、配置模块105的功能所需的软件或程序代码存储在存储器810中，另外，日志获取模块101与其它设备的交互通过通信接口830实现，处理器用于执行存储器810中的指令，实现上述模型训练装置100所执行的方法。
图9示出的一种计算设备集群的结构示意图。其中，图9所示的计算设备集群90包括多个计算设备，上述模型训练装置100可以分布式地部署在该计算设备集群90中的多个计算设备上。如图9所示，计算设备集群90包括多个计算设备900，每个计算设备900包括存储器910、处理器920、通信接口930以及总线940，其中，存储器910、处理器920、通信接口930通过总线940实现彼此之间的通信连接。
处理器920可以采用CPU、GPU、ASIC或者一个或多个集成电路。处理器920还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，模型训练装置100的部分功能可用通过处理器920中的硬件的集成逻辑电路或者软件形式的指令完成。处理器920还可以是DSP、FPGA、通用处理器、其他可编程逻辑器件，分立门或者晶体管逻辑器件、分立硬件组件，可以实现或者执行本申请实施例中公开的部分方法、步骤及逻辑框图。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器、闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器910，在每个计算设备900中，处理器920读取存储器910中的信息，结合其硬件可以完成模型训练装置100的部分功能。
存储器910可以包括ROM、RAM、静态存储设备、动态存储设备、硬盘(例如SSD、HDD)等。存储器910可以存储程序代码，例如，用于实现日志获取模块101的部分或者全部程序代码、用于实现模型训练模块102的部分或者全部程序代码、用于实现异常检测模块103的部分或者全部程序代码、用于实现告警模块104的部分或者全部程序代码、用于实现配置模块105的部分或者全部程序代码等。针对每个计算设备900，当存储器910中存储的程序代码被处理器920执行时，处理器920基于通信接口930执行模型训练装置100所执行的部分方法，如其中一部分计算设备900可以用于执行上述日志获取模块101、模型训练模块102、异常检测模块103所执行的方法，另一部分计算设备900用于执行上述告警模块104、配置模块105所执行的方法。存储器910还可以存储数据，例如：处理器920在执行过程中产生的中间数据或结果数据，例如，上述目标检测模型、用于指示待检测实例的网络流量是否存在异常的结果等。
每个计算设备900中的通信接口903用于与外部通信，例如与其它计算设备900进行交互等。
总线940可以是外设部件互连标准总线或扩展工业标准结构总线等。为便于表示，图9中每个计算设备900内的总线940仅用一条粗线表示，但并不表示仅有一根总线或一种类型 的总线。
上述多个计算设备900之间通过通信网络建立通信通路，以实现模型训练装置100的功能。任一计算设备可以是云环境中的计算设备(例如，服务器)，或边缘环境中的计算设备，或终端设备。
此外，本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在一个或者多个计算设备上运行时，使得该一个或者多个计算设备执行上述实施例中模型训练装置100的各个模块所执行的方法。
此外，本申请实施例还提供了一种计算机程序产品，所述计算机程序产品被一个或者多个计算设备执行时，所述一个或者多个计算设备执行前述模型训练方法中的任一方法。该计算机程序产品可以为一个软件安装包，在需要使用前述模型训练方法的任一方法的情况下，可以下载该计算机程序产品并在计算机上执行该计算机程序产品。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，训练设备，或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、训练设备或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向 另一个网站站点、计算机、训练设备或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的训练设备、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (27)

  1. 一种模型训练方法，其特征在于，所述方法包括：
    获取第一日志，所述第一日志用于记录待检测实例在第一时间段内的网络流量；
    基于所述第一日志，采用无监督学习算法训练得到目标检测模型，所述目标检测模型用于检测所述待检测实例的网络流量是否存在异常，所述目标检测模型包括机器学习模型或人工智能AI模型，所述目标检测模型包括至少一个子模型。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    获取第二日志，所述第二日志用于记录所述待检测实例在第二时间段内的网络流量；
    利用所述目标检测模型，根据所述第二日志检测所述待检测实例在所述第二时间段内的网络流量是否存在异常。
  3. 根据权利要求2所述的方法，其特征在于，所述目标检测模型包括多个子模型，所述利用所述目标检测模型，根据所述第二日志检测所述待检测实例在所述第二时间段内的网络流量是否存在异常，包括：
    利用所述多个子模型分别根据所述第二日志进行推理，得到所述多个子模型分别输出的检测结果，不同子模型具有不同的超参数、或具有不同的模型结构、或基于不同的无监督学习算法完成训练；
    对所述多个子模型分别输出的检测结果进行投票，确定所述待检测实例在所述第二时间段内的网络流量是否异常。
  4. 根据权利要求3所述的方法，其特征在于，所述对所述多个子模型分别输出的检测结果进行投票，包括：
    根据目标投票策略，对所述多个子模型分别输出的检测结果进行投票，所述目标投票策略包括多数投票策略、一票否决策略、或一票通过策略中的一种或者多种。
  5. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    生成配置界面，所述配置界面用于提示用户从多个候选投票策略中选择所述目标投票策略，或者所述配置界面用于提示用户定义所述目标投票策略。
  6. 根据权利要求3至5任一项所述的方法，其特征在于，所述方法还包括：
    生成配置界面，所述配置界面用于提示用户对所述多个子模型的数量、模型超参数、或无监督学习算法中的一种或者多种进行配置。
  7. 根据权利要求3至6任一项所述的方法，其特征在于，在利用多个子模型分别根据所述第二日志进行推理之前，所述方法还包括：
    对模型池中的子模型进行随机抽取，得到所述多个子模型。
  8. 根据权利要求1至7任一项所述的方法，其特征在于，所述获取第一日志，包括：
    获取用户向云服务平台提供的针对所述待检测实例的网络流量的指示信息；
    根据所述指示信息，从所述待检测实例中采集所述第一日志。
  9. 根据权利要求1至7任一项所述的方法，其特征在于，所述获取第一日志，包括：
    获取多个原始日志，所述多个原始日志用于记录云服务平台监控的多个实例的网络流量，所述多个实例包括所述待检测实例；
    根据所述待检测实例的互联网协议IP地址，从所述多个原始日志中筛选出所述第一日 志。
  10. 根据权利要求1至7任一项所述的方法，其特征在于，所述获取第一日志，包括：
    获取所述待检测实例对应的多个候选日志；
    利用去重条件，对所述多个候选日志进行去重，得到所述第一日志，所述去重条件包括至少一种网络流量的类型标识。
  11. 根据权利要求1至10任一项所述的方法，其特征在于，所述基于所述第一日志，采用无监督学习算法训练得到目标检测模型，包括：
    根据所述第一日志，提取网络流量特征，所述网络流量特征包括所述待检测实例的通信次数、目的互联网协议IP地址数量、源IP地址数量、目的IP地址数量与端口数量总和、或源IP地址与目的IP地址与源端口与目的端口的信息熵中的一种或者多种；
    利用所述网络流量特征采用所述无监督学习算法训练得到所述目标检测模型。
  12. 根据权利要求11所述的方法，其特征在于，所述利用所述网络流量特征采用所述无监督学习算法训练得到所述目标检测模型，包括：
    对于所述网络流量特征进行归一化，得到目标网络流量特征；
    利用所述目标网络流量特征采用所述无监督学习算法训练得到所述目标检测模型。
  13. 一种模型训练装置，其特征在于，所述模型训练装置包括：
    日志获取模块，用于获取第一日志，所述第一日志用于记录待检测实例在第一时间段内的网络流量；
    模型训练模块，用于基于所述第一日志，采用无监督学习算法训练得到目标检测模型，所述目标检测模型用于检测所述待检测实例的网络流量是否存在异常，所述目标检测模型包括机器学习模型或人工智能AI模型，所述目标检测模型包括至少一个子模型。
  14. 根据权利要求13所述的装置，其特征在于，所述日志获取模块，还用于获取第二日志，所述第二日志用于记录所述待检测实例在第二时间段内的网络流量；
    所述装置还包括异常检测模块，所述异常检测模块，用于利用所述目标检测模型，根据所述第二日志检测所述待检测实例在所述第二时间段内的网络流量是否存在异常。
  15. 根据权利要求14所述的装置，其特征在于，所述目标检测模型包括多个子模型，所述异常检测模块，用于：
    利用所述多个子模型分别根据所述第二日志进行推理，得到所述多个子模型分别输出的检测结果，不同子模型具有不同的超参数、或具有不同的模型结构、或基于不同的无监督学习算法完成训练；
    对所述多个子模型分别输出的检测结果进行投票，确定所述待检测实例在所述第二时间段内的网络流量是否异常。
  16. 根据权利要求15所述的装置，其特征在于，所述异常检测模块，用于：
    根据目标投票策略，对所述多个子模型分别输出的检测结果进行投票，所述目标投票策略包括多数投票策略、一票否决策略、或一票通过策略中的一种或者多种。
  17. 根据权利要求16所述的装置，其特征在于，所述装置还包括：
    配置模块，用于生成配置界面，所述配置界面用于提示用户从多个候选投票策略中选择所述目标投票策略，或者所述配置界面用于提示用户定义所述目标投票策略。
  18. 根据权利要求15至17任一项所述的装置，其特征在于，所述装置还包括：
    配置模块，用于生成配置界面，所述配置界面用于提示用户对所述多个子模型的数量、模型超参数、或无监督学习算法中的一种或者多种进行配置。
  19. 根据权利要求15至18任一项所述的装置，其特征在于，所述异常检测模块，还用于在利用多个子模型分别根据所述第二日志进行推理之前，对模型池中的子模型进行随机抽取，得到所述多个子模型。
  20. 根据权利要求13至19任一项所述的装置，其特征在于，所述日志获取模块，用于：
    获取用户向云服务平台提供的针对所述待检测实例的网络流量的指示信息；
    根据所述指示信息，从所述待检测实例中采集所述第一日志。
  21. 根据权利要求13至19任一项所述的装置，其特征在于，所述日志获取模块，用于：
    获取多个原始日志，所述多个原始日志用于记录云服务平台监控的多个实例的网络流量，所述多个实例包括所述待检测实例；
    根据所述待检测实例的互联网协议IP地址，从所述多个原始日志中筛选出所述第一日志。
  22. 根据权利要求13至19任一项所述的装置，其特征在于，所述日志获取模块，用于：
    获取所述待检测实例对应的多个候选日志；
    利用去重条件，对所述多个候选日志进行去重，得到所述第一日志，所述去重条件包括至少一种网络流量的类型标识。
  23. 根据权利要求13至22任一项所述的装置，其特征在于，所述模型训练模块，用于：
    根据所述第一日志，提取网络流量特征，所述网络流量特征包括所述待检测实例的通信次数、目的互联网协议IP地址数量、源IP地址数量、目的IP地址数量与端口数量总和、或源IP地址与目的IP地址与源端口与目的端口的信息熵中的一种或者多种；
    利用所述网络流量特征采用所述无监督学习算法训练得到所述目标检测模型。
  24. 根据权利要求23所述的装置，其特征在于，所述模型训练模块，用于：
    对于所述网络流量特征进行归一化，得到目标网络流量特征；
    利用所述目标网络流量特征采用所述无监督学习算法训练得到所述目标检测模型。
  25. 一种计算设备集群，其特征在于，包括至少一个计算设备，每个计算设备包括处理器和存储器；
    所述处理器用于执行所述存储器中存储的指令，以使得所述计算设备集群执行权利要求1至12中任一项所述的方法。
  26. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有指令，当其在至少一个计算设备上运行时，使得所述至少一个计算设备执行如权利要求1至12任一项所述的方法。
  27. 一种包含指令的计算机程序产品，其特征在于，当其在至少一个计算设备上运行时，使得所述至少一个计算设备执行如权利要求1至12中任一项所述的方法。
PCT/CN2023/081264 2022-07-05 2023-03-14 模型训练方法、装置及相关设备 WO2024007615A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210784579.X 2022-07-05
CN202210784579 2022-07-05
CN202211182844.3A CN117391214A (zh) 2022-07-05 2022-09-27 模型训练方法、装置及相关设备
CN202211182844.3 2022-09-27

Publications (1)

Publication Number Publication Date
WO2024007615A1 true WO2024007615A1 (zh) 2024-01-11

Family

ID=89454085

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/081264 WO2024007615A1 (zh) 2022-07-05 2023-03-14 模型训练方法、装置及相关设备

Country Status (1)

Country Link
WO (1) WO2024007615A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108200005A (zh) * 2017-09-14 2018-06-22 国网浙江省电力公司宁波供电公司 基于非监督学习的电力二次系统网络流量异常检测方法
US20190260778A1 (en) * 2018-02-19 2019-08-22 Nec Laboratories America, Inc. Unsupervised spoofing detection from traffic data in mobile networks
CN113656254A (zh) * 2021-08-25 2021-11-16 上海明略人工智能(集团)有限公司 基于日志信息的异常检测方法、系统和计算机设备
CN113709125A (zh) * 2021-08-18 2021-11-26 北京明略昭辉科技有限公司 一种异常流量的确定方法、装置、存储介质及电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108200005A (zh) * 2017-09-14 2018-06-22 国网浙江省电力公司宁波供电公司 基于非监督学习的电力二次系统网络流量异常检测方法
US20190260778A1 (en) * 2018-02-19 2019-08-22 Nec Laboratories America, Inc. Unsupervised spoofing detection from traffic data in mobile networks
CN113709125A (zh) * 2021-08-18 2021-11-26 北京明略昭辉科技有限公司 一种异常流量的确定方法、装置、存储介质及电子设备
CN113656254A (zh) * 2021-08-25 2021-11-16 上海明略人工智能(集团)有限公司 基于日志信息的异常检测方法、系统和计算机设备

Similar Documents

Publication Publication Date Title
US20220014556A1 (en) Cybersecurity profiling and rating using active and passive external reconnaissance
CN112003870B (zh) 一种基于深度学习的网络加密流量识别方法及装置
US9386028B2 (en) System and method for malware detection using multidimensional feature clustering
US9843594B1 (en) Systems and methods for detecting anomalous messages in automobile networks
US11316851B2 (en) Security for network environment using trust scoring based on power consumption of devices within network
US20220014560A1 (en) Correlating network event anomalies using active and passive external reconnaissance to identify attack information
US11159542B2 (en) Cloud view detection of virtual machine brute force attacks
EP3619903B1 (en) Non-protocol specific system and method for classifying suspect ip addresses as sources of non-targeted attacks on cloud based machines
CN110166462B (zh) 访问控制方法、系统、电子设备及计算机存储介质
US20220224723A1 (en) Ai-driven defensive cybersecurity strategy analysis and recommendation system
US20180131705A1 (en) Visibility of Non-Benign Network Traffic
CN111274583A (zh) 一种大数据计算机网络安全防护装置及其控制方法
US11496495B2 (en) System and a method for detecting anomalous patterns in a network
US10320833B2 (en) System and method for detecting creation of malicious new user accounts by an attacker
EP3465515B1 (en) Classifying transactions at network accessible storage
US10489720B2 (en) System and method for vendor agnostic automatic supplementary intelligence propagation
CN113660273B (zh) 超融合架构下基于深度学习的入侵检测方法及装置
US11516240B2 (en) Detection of anomalies associated with fraudulent access to a service platform
CN110955890B (zh) 恶意批量访问行为的检测方法、装置和计算机存储介质
CN117391214A (zh) 模型训练方法、装置及相关设备
WO2024007615A1 (zh) 模型训练方法、装置及相关设备
CN115033889B (zh) 非法提权检测方法和装置、存储介质、计算机设备
CN112699369A (zh) 一种通过栈回溯检测异常登录的方法及装置
CN111258845A (zh) 事件风暴的检测
WO2024032032A1 (zh) 云平台测试方法、装置、服务节点及云平台

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23834399

Country of ref document: EP

Kind code of ref document: A1