CN114915446A - Intelligent network security detection method fusing priori knowledge - Google Patents

Abstract

The invention provides an intelligent network security detection method fusing prior knowledge, which comprises the following steps: defining a characterization form of a vulnerability body, storing and extracting the vulnerability knowledge, and forming a vulnerability knowledge base; acquiring information from an environment to be measured; constructing a state information matrix; taking a state information matrix as an input of an agent and the vulnerability knowledge base; constructing an intelligent agent; determining a behavior strategy of an agent based on a state information matrix acquired by the environment to be tested and the potential vulnerability information acquired by the vulnerability knowledge base; and based on the behavior strategy of the intelligent agent, executing specific behaviors, calculating reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feeding the reward information back to the intelligent agent, and guiding the intelligent agent to update the strategy. According to the scheme of the invention, the network security detection of the network environment is automatically realized, the efficiency of the network security detection is improved, and the problem that the automatic network security detection is difficult to apply in a complex environment is solved.

Description

An Intelligent Network Security Detection Method Integrating Prior Knowledge

technical field

The invention relates to the field of cyberspace security, in particular to an intelligent network security detection method integrating prior knowledge.

Background technique

Regular security testing is an important process for assessing asset resilience and compliance, especially confidentiality, availability and integrity. Penetration Testing is widely regarded as the best way to assess the security of digital assets by identifying and exploiting vulnerabilities. In the process of PT, security experts are faced with complex environment, repetitive operations and similar problems. PT task automation is obviously an efficient method that saves manpower and resources. Early research focused on improving PT systems by optimizing the planning phase, which was modeled as an attack graph or decision tree problem, reflecting the practical nature of continuous decision-making. Nonetheless, due to the static nature of the method and its limitations on the planning phase, most of the work is related to vulnerability assessment rather than PT.

In recent years, Machine Learning (ML) has opened up new avenues for effectively solving complex problems. ML has been shown to handle difficult problems faster and more accurately than humans in some cases. There are three types of ML: supervised learning, unsupervised learning, and reinforcement learning. Supervised and unsupervised learning have been used in intrusion detection, malware detection, privacy-preserving systems, etc. Preparing a larger dataset for training is a prerequisite for developing a secure solution. However, in the real-time, continuous environment of network security detection, it is difficult to preprocess behavior datasets, so neither supervised nor unsupervised learning is suitable for solving automated PT problems. Reinforcement Learning (RL) is a type of machine learning that learns by exploring the environment and accumulating experience. RL agents can adapt themselves to real-time, continuous environments without prior datasets.

In 2013, Sarraute et al. established a 4AL decomposition algorithm, which divided a large network into smaller networks according to the network structure, and solved them one by one through Partially Observable Markov Decision Processes (POMDP). In 2014, Durkota et al. proposed an algorithm for calculating the optimal attack strategy with action cost and failure probability attack graph, which transformed the optimal path planning problem of the attack graph into Markov Decision Processes (Markov Decision Processes, MDP) to generate the best attack strategy to guide network security detection. In 2017, Shmaryahu et al. modeled PT as a partially observable episodic problem and designed an episodic planning tree algorithm to plan attack paths. In the same year, Alexander preschner introduced POMDP into industrial control systems in an attempt to automatically verify the security of industrial control systems. In 2018, Ghanem and Chen modeled the system as a POMDP and tested it with an external POMDP solver. In 2019, Zhou et al. described PT as an MDP process and proposed an attack planning based on network information gain (NIG-AP) algorithm. Use network information to obtain rewards, guide the agent to choose the best response action, and discover the hidden attack path from the perspective of the intruder. In 2020, Hu et al. built an automated network security detection framework based on deep reinforcement learning to automatically find the best attack path for a given topology. In 2021, Zennaro et al. formalized the simple CTF problem as a network security detection problem, and solved such problems based on model-free reinforcement learning.

The POMDP-based findings confirm the hypothesis that reinforcement learning can improve the accuracy and reliability of cybersecurity detection. However, due to the large number of hosts in the network security detection environment and the complex host configuration, it is very difficult to accurately solve the POMDP. MDP-based reinforcement learning can in principle allow model-free learning, but in practice may need to rely on some form of prior knowledge to solve the problem.

SUMMARY OF THE INVENTION

In order to solve the above technical problems, the present invention proposes an intelligent network security detection method integrating prior knowledge, which is used to solve the problem that the automatic network security detection method in the prior art is not efficient and practical, and is difficult to be applied to real large-scale applications. Technical issues for network scenarios.

According to a first aspect of the present invention, an intelligent network security detection method incorporating prior knowledge is provided, the method comprising the following steps:

Step S1: define the representation form of the vulnerability ontology, the vulnerability ontology is characterized based on the concepts, attributes and relationships of the vulnerability ontology; based on the acquired knowledge source and the representation form of the vulnerability ontology, extract the vulnerability from the acquired knowledge source knowledge; store the extracted vulnerability knowledge to form a vulnerability knowledge base;

Step S2: Build an environmental information acquisition module, the environmental information acquisition module is used to acquire the following basic information from the environment to be tested: operating host IP, operating system, survival port, service information, and classify the acquired basic information according to categories number storage;

Step S3: judging whether the preset target is reached, if yes, the method ends; if not, go to step S4; the preset target is to achieve network security detection for a specific target;

Step S4: based on the environmental information acquisition module, acquire the basic information of the environment to be tested; based on the number information, acquire the network topology, host authority and host configuration information of the environment to be tested, and build a state information matrix; The state information matrix is used as the input of the agent and the vulnerability knowledge base;

Step S5: constructing an agent; based on the environment state information matrix and the potential vulnerability information obtained from the vulnerability knowledge base, determine the behavior strategy of the agent;

Step S6: Execute specific behavior based on the behavior strategy of the agent, and apply the specific behavior to the environment to be tested. According to the result of the behavior execution and the impact on the environment, the reward module calculates the reward information and feeds it back to the agent. , guide the agent to update the strategy; go to step S3.

Further, the vulnerability ontology is characterized based on the concept, attributes and relationships of the vulnerability ontology, the vulnerability ontology refers to the defects existing in the specific implementation of hardware, software, protocols or system security policies, and the attributes of the vulnerability ontology are: Refers to the potential conditions for the existence of vulnerabilities, and the relationship of the vulnerability ontology refers to the interaction relationship between vulnerabilities, wherein the attributes of the vulnerability ontology include the exploiting method of the vulnerability, the effect and impact of the vulnerability, whether there is a vulnerability exploit, whether the vulnerability is exploited or not. The service containing the vulnerability and the operating system corresponding to the service containing the vulnerability, and the relationship includes an intersection relationship, an inheritance relationship, and an attribute relationship.

Further, in the step S2, wherein:

The preset target is to realize network security detection of a specific target, including network security detection of a specific host in a network environment and/or network security detection of a single host starting from a starting host.

Further, the state information matrix of the environment to be tested is defined as follows:

Among them, h _i h _j represents the connection relationship between the i-th host to be tested and the j-th host to be tested, 0 means that the hosts to be tested are not connected, 1 means that the hosts to be tested are connected, and h _{i hi} means For the permission level obtained on the host i to be tested, since the number of network nodes in different network security detection environments is different, the number of nodes is set to a fixed value, p _k (h _i ) indicates whether the host i to be tested contains a serial number is the attribute of k, and privilege(h _i ) represents the authority of the agent on the host i to be tested.

According to a second aspect of the present invention, an intelligent network security detection device incorporating prior knowledge is provided, the device comprising:

Vulnerability knowledge base building module: configured to define the representation form of the vulnerability ontology, the vulnerability ontology is represented based on the concepts, attributes and relationships of the vulnerability ontology; based on the acquired knowledge source and the representation form of the vulnerability ontology, from the acquired Extracting vulnerability knowledge from the knowledge source; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;

Detection module: configured to build an environmental information acquisition module, the environmental information acquisition module is used to acquire the following basic information from the environment to be tested: operating host IP, operating system, survival port, service information, and the acquired basic information Stored by category number;

Judging module: configured to judge whether a preset target is reached, and the preset target is to realize network security detection for a specific target;

State information matrix building module: configured to obtain the basic information of the environment to be tested based on the environment information acquisition module; based on the number information, obtain the network topology, host authority and host configuration information of the environment to be tested, and construct the state information matrix; using the state information matrix as the input of the agent and the vulnerability knowledge base;

Behavior determination module: configured to construct an agent; based on the environment state information matrix and the potential vulnerability information obtained from the vulnerability knowledge base, determine the behavior strategy of the agent;

Update module: configured to be based on the behavior strategy of the agent, execute specific behavior, and apply the specific behavior to the environment to be tested. The agent guides the agent to update the strategy; triggers the judgment module.

According to a third aspect of the present invention, an intelligent network security detection system integrating prior knowledge is provided, including:

a processor for executing multiple instructions;

memory for storing multiple instructions;

Wherein, the plurality of instructions are used to be stored by the memory and loaded by the processor to execute the aforementioned method.

According to a fourth aspect of the present invention, there is provided a computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used by a processor to load and execute the aforementioned method.

According to the above solution of the present invention, the method aims at realizing automatic and intelligent network security detection, integrates prior knowledge, and proposes a network security detection method based on knowledge graph and reinforcement learning. The method combines the relevant technologies of reinforcement learning in the field of artificial intelligence and knowledge graphs to build an automatic model to realize intelligent network security detection. The invention aims to solve the problems that the network environment is complex, the types and numbers of loopholes are numerous, and the automatic network security detection is difficult to realize. The method of the present invention solves the above problems by combining knowledge graph and reinforcement learning, and achieves the following effects: (1) the present invention builds a vulnerability knowledge base based on the knowledge graph, which is convenient for managing and querying the current host and potential potential in the network. Vulnerability information, and at the same time provides convenience for the agent to analyze the optional effective behavior; (2) The method proposed in the present invention changes the behavior selection method of the agent, and transforms the agent from randomly exploring available behaviors to selecting the most available behaviors. (3) The method can realize automatic and intelligent network security detection of the network environment.

The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly, and implement it according to the content of the description, the preferred embodiments of the present invention are described in detail below with the accompanying drawings.

Description of drawings

The accompanying drawings, which form a part of this disclosure, are provided to provide a further understanding of the present disclosure, and the present disclosure is provided with the following figures to illustrate. In the attached image:

1 is a flowchart of an intelligent network security detection method integrating prior knowledge according to an embodiment of the present invention;

2 is a schematic structural diagram of an intelligent network security detection model integrating prior knowledge according to an embodiment of the present invention;

3 is a schematic diagram of a construction method of a vulnerability knowledge base according to an embodiment of the present invention;

4 is a schematic diagram of a vulnerability knowledge ontology according to an embodiment of the present invention;

5 is a schematic structural diagram of an agent neural network according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an intelligent network security detection device integrating prior knowledge according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

First, a flowchart of an intelligent network security detection method integrating prior knowledge according to an embodiment of the present invention is described with reference to FIGS. 1-2 . As shown in Figure 1-2, the method includes the following steps:

Step S1: define the representation form of the vulnerability ontology, the vulnerability ontology is characterized based on the concepts, attributes and relationships of the vulnerability ontology; based on the acquired knowledge source and the representation form of the vulnerability ontology, extract the vulnerability from the acquired knowledge source knowledge; store the extracted vulnerability knowledge to form a vulnerability knowledge base;

Step S2: Build an environmental information acquisition module, the environmental information acquisition module is used to acquire the following basic information from the environment to be tested: operating host IP, operating system, survival port, service information, and classify the acquired basic information according to categories number storage;

Step S3: judging whether the preset target is reached, if yes, the method ends; if not, go to step S4; the preset target is to achieve network security detection for a specific target;

Step S4: based on the environmental information acquisition module, acquire the basic information of the environment to be tested; based on the number information, acquire the network topology, host authority and host configuration information of the environment to be tested, and build a state information matrix; The state information matrix is used as the input of the agent and the vulnerability knowledge base;

Step S5: constructing an agent; based on the environment state information matrix and the potential vulnerability information obtained from the vulnerability knowledge base, determine the behavior strategy of the agent;

Step S6: Execute specific behavior based on the behavior strategy of the agent, and apply the specific behavior to the environment to be tested. According to the result of the behavior execution and the impact on the environment, the reward module calculates the reward information and feeds it back to the agent. , guide the agent to update the strategy; go to step S3.

Since network security detection is a sequential decision-making problem, its task is to determine the network security detection behavior that should be taken according to the current state. Therefore, it can be modeled as a Markov decision process, and the related technologies based on reinforcement learning can be implemented. Automated cybersecurity detection. Reinforcement learning agents can accumulate experience in the training process and make decisions like security experts. However, this comes at the cost of a lot of training. Due to the complex network security detection environment and the large number and types of vulnerabilities, it is difficult for agents to make decisions. To achieve fast and effective learning, the present invention adds a vulnerability knowledge base to the automation model, and introduces the vulnerability knowledge base as the library of the intelligent body. According to the learned experience, choose the final action from the feasible actions to improve the efficiency of learning.

The invention constructs a knowledge graph to store vulnerability-related information, and realizes matching of possible vulnerabilities according to the current state information; realizes an automatic network security detection method based on reinforcement learning, and the state information obtained by the agent from the environment is firstly vulnerability matching, and then from the matching to select the next action.

The step S1, wherein:

Vulnerability information has the characteristics of mass, decentralization and fragmentation. An important condition for building a vulnerability knowledge base is to collect vulnerability information. Vulnerability information is mainly collected by means of Internet query and acquisition. Currently, the relatively well-known international vulnerability-related standards and specifications include CVE (Common Vulnerabilities & Exposures), CPE (Common platformenumeration), CVSS (Common Vulnerability Scoring System) and CNNVD (China National Vulnerability Database of Information Security) four. Among them, CVE is the most credible security vulnerability disclosure and release unit in the world; CPE is a standardized method for describing and identifying applications, operating systems and hardware devices existing in enterprise computing assets; CVSS is a method for evaluating Vulnerability severity, and help determine the industry's public standards for the urgency and importance of responses; CNNVD is an authoritative vulnerability recording platform in China. Collect vulnerability information from CVE, CPE, CVSS and CNNVD, integrate the vulnerability information, and use the obtained information as a knowledge source.

The representation form of the vulnerability ontology is defined based on the knowledge graph. The vulnerability ontology is characterized based on the concepts, attributes and relationships of the vulnerability ontology. The vulnerability ontology refers to the defects existing in the specific implementation of hardware, software, protocols or system security policies. The attribute of the vulnerability ontology refers to the potential conditions for the existence of the vulnerability, and the relationship of the vulnerability ontology refers to the interaction relationship between the vulnerabilities, wherein the attribute of the vulnerability ontology includes the exploiting method of the vulnerability, the effect and influence of the vulnerability. , whether there is a vulnerability exploit, whether the service contains the vulnerability, and the operating system corresponding to the service containing the vulnerability, and the relationship includes an intersection relationship, an inheritance relationship, and an attribute relationship.

In this embodiment, the defined vulnerability ontology is shown in FIG. 4 . First of all, it is necessary to reasonably represent the vulnerability information. Therefore, according to expert experience, a vulnerability ontology is constructed for the association between common operating systems, common services, and common vulnerabilities, which can better describe the association between vulnerability-related knowledge. The structure is shown in Figure 4, where the number is the internal number, which is consistent with the number in the environmental state information, which is convenient for the agent to match the loopholes according to the input state information.

Based on the acquired knowledge source and the representation form of the vulnerability ontology, vulnerability knowledge is extracted from the acquired knowledge source. In this embodiment, the information in the knowledge source comes from different specifications and standards, and may contain information with duplicate or different structures. Therefore, the named entity recognition technology is used to extract and clean the information in the knowledge source, and then the relationship is extracted, and finally Realize knowledge extraction of vulnerability information.

Store and extract the vulnerability knowledge to form a vulnerability knowledge base, that is, select the importance degree greater than the preset threshold, and store the vulnerability knowledge related to the state of the network to be tested and the host to be tested in the graph database using Neo4j to realize the vulnerability knowledge Build the library.

When security experts conduct network security detection, they will judge the possible vulnerabilities in the current environment based on the scanned network and host status information, and use the vulnerabilities to realize network security detection. In this process, the judgment of experts is based on the accumulation of their own knowledge, so the lack of expert experience is one of the great challenges facing the current automated network security detection. Therefore, the present invention builds a prior knowledge base similar to expert experience, collects vulnerability information, the vulnerability information includes vulnerability number, vulnerability level, vulnerability source, functions that can be realized by exploiting the vulnerability, etc., and extracts useful information through named entity recognition technology , process the information to standardize it, and build a knowledge inference model to manage the vulnerability information.

The step S3, wherein:

The preset target is to realize the network security detection of a specific target in the network to be tested, including starting from a certain host in the network to be tested, the network security detection of a specific host in the network environment and/or the network security detection of a single host. Security check.

The step S4, wherein:

Based on the environmental information acquisition module, the basic information of the environment to be tested is acquired, classified and numbered according to the information content, and an environmental state matrix is constructed based on the numbered environmental information as the input information of the agent. The environmental state matrix should include the current agent exploration. to the network topology, host configuration and other information. According to the information scanned by experts from the environment in the actual network security detection process, the state information matrix of the environment to be tested of the automated network security detection model is defined as follows:

Among them, h _i h _j represents the connection relationship between the i-th host to be tested and the j-th host to be tested, 0 means that the hosts to be tested are not connected, 1 means that the hosts to be tested are connected, and h _{i hi} means For the permission level obtained on the host i to be tested, since the number of network nodes in different network security detection environments is different, the number of nodes is set to a fixed value, p _k (h _i ) indicates whether the host i to be tested contains a serial number is the attribute of k, and privilege(h _i ) represents the authority of the agent on the host i to be tested.

The state information matrix is used as the input of the agent and the vulnerability library. The state matrix is used as the input of the agent to provide scene information for the agent. behavior space.

The step S5, wherein:

The agent determines the behavior based on the output of the vulnerability prior knowledge base. The behavior is the output of the agent, which represents the decision made by the agent for the current environment. The output of the agent includes the connection behavior between the hosts in the network to be tested. and exploit behavior, the connection behavior between hosts refers to the behavior of the agent moving laterally between hosts; the exploit is to analyze the input state and compare it with the vulnerability information in the vulnerability knowledge base Match, determine the possible vulnerabilities in the network environment to be tested, and combine the exploits corresponding to the possible vulnerabilities with the connection behaviors between the hosts found in the environment to be tested to form a behavior library that the agent can choose. The stability of the model during the learning process is to set the size of the behavior library to a fixed value, and select and execute actions from the given behavior library, which is different from letting the agent freely explore all the actions in the past.

In an automatic network security detection model based on reinforcement learning, the input received by the agent is the state information obtained from the environment, and the output is the probability of the agent taking each action. Due to the complexity of the network security detection environment, the size of the state space increases exponentially with the network scale and host configuration. The traditional tabular methods such as the Q-learning method are not suitable for realizing automatic network security detection. Therefore, the present invention The deep reinforcement learning technology is introduced to fit the Q function based on the neural network, which can effectively solve the problem of too large state space. However, due to the introduction of prior knowledge, the agent can take different behaviors in different states, resulting in inconsistent number of output nodes. In view of the above situation, it is necessary to redesign the reinforcement learning algorithm to realize intelligent decision-making.

The neural network structure of the agent to update the Q value proposed by the present invention is shown in Figure 5. The neural network model includes three convolutional layers, and the third convolutional layer is connected to the fully connected layer; since the values in the state matrix are mostly is 0 or 1, and the matrix is sparse, so no pooling layer is used. The environment state matrix obtained from the environment to be tested is used as the input of the first convolutional layer, and the fully connected layer outputs the features of the environment state matrix. In this embodiment, the maximum number of network nodes is set to 100, the host configuration includes information such as services and ports, and the value is set to 100, so the scale of the input matrix is 100*200. By matching the received state information with the prior knowledge base, the exploits corresponding to the top 10 vulnerabilities are selected to construct the behavior space of the agent. In addition, in addition to the exploit behavior, the connection behavior is added, indicating that the move from the current host to The behavior of other hosts, so set the behavior space size of the agent output to 11. In addition, hyperparameters such as the number of convolutional layers, the number of convolutional kernels in each layer, and the size of convolutional kernels in each layer need to be determined through experiments. Choose the appropriate output layer activation function and loss function according to the task requirements.

The step S6, wherein:

Reward is the feedback on the behavior of the agent, which is very important for reinforcement learning, determines the learning direction and convergence speed of the agent, and affects the correctness and effectiveness of the agent's decision-making. The present invention divides the reward into positive feedback and There are two parts of negative feedback: the positive reward obtained by the agent successfully completing the behavior and the negative reward obtained by the failure of the behavior execution. The agent receives the reward, adjusts the neural network parameters according to the reward value, and guides the agent to update the strategy and make more accurate predictions. The steps are repeated until the objective in step S3 is met.

FIG. 6 is a schematic structural diagram of an intelligent network security detection device integrating prior knowledge according to an embodiment of the present invention. As shown in FIG. 6 , the device includes:

Vulnerability knowledge base building module: configured to define the representation form of the vulnerability ontology, the vulnerability ontology is represented based on the concepts, attributes and relationships of the vulnerability ontology; based on the acquired knowledge source and the representation form of the vulnerability ontology, from the acquired Extracting vulnerability knowledge from the knowledge source; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;

Detection module: configured to build an environmental information acquisition module, the environmental information acquisition module is used to acquire the following basic information from the environment to be tested: operating host IP, operating system, survival port, service information, and the acquired basic information Stored by category number;

Judging module: configured to judge whether a preset target is reached, and the preset target is to realize network security detection for a specific target;

State information matrix building module: configured to obtain the basic information of the environment to be tested based on the environment information acquisition module; based on the number information, obtain the network topology, host authority and host configuration information of the environment to be tested, and construct the state information matrix; using the state information matrix as the input of the agent and the vulnerability knowledge base;

Behavior determination module: configured to construct an agent; based on the environment state information matrix and the potential vulnerability information obtained from the vulnerability knowledge base, determine the behavior strategy of the agent;

Update module: configured to be based on the behavior strategy of the agent, execute specific behavior, and apply the specific behavior to the environment to be tested. The agent guides the agent to update the strategy; triggers the judgment module.

The embodiment of the present invention further provides an intelligent network security detection system integrating prior knowledge, including:

a processor for executing multiple instructions;

memory for storing multiple instructions;

Wherein, the plurality of instructions are used to be stored by the memory and loaded by the processor to execute the aforementioned method.

The embodiment of the present invention further provides a computer-readable storage medium, where a plurality of instructions are stored in the storage medium; the plurality of instructions are used for loading and executing the foregoing method by a processor.

It should be noted that the embodiments of the present invention and the features of the embodiments may be combined with each other under the condition of no conflict.

In the several embodiments provided by the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined. Either it can be integrated into another system, or some features can be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

The above-mentioned integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, and include several instructions to make a computer device (which can be a personal computer, a physical machine server, or a network cloud server, etc., need to install Windows or Windows Server operating system) to execute each of the present invention. Some steps of the method described in the Examples. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.

The above are only preferred embodiments of the present invention, and do not limit the present invention in any form. Any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention still belong to the present invention. within the scope of the technical solution of the invention.

Claims (7)

1. an intelligent network security detection method integrating prior knowledge, is characterized in that, described method comprises the following steps: Step S1: define the representation form of the vulnerability ontology, the vulnerability ontology is characterized based on the concepts, attributes and relationships of the vulnerability ontology; based on the acquired knowledge source and the representation form of the vulnerability ontology, extract the vulnerability from the acquired knowledge source knowledge; store the extracted vulnerability knowledge to form a vulnerability knowledge base; Step S2: Build an environmental information acquisition module, the environmental information acquisition module is used to acquire the following basic information from the environment to be tested: operating host IP, operating system, survival port, service information, and classify the acquired basic information according to categories number storage; Step S3: judging whether the preset target is reached, if yes, the method ends; if not, go to step S4; the preset target is to achieve network security detection for a specific target; Step S4: based on the environmental information acquisition module, acquire the basic information of the environment to be tested; based on the number information, acquire the network topology, host authority and host configuration information of the environment to be tested, and build a state information matrix; The state information matrix is used as the input of the agent and the vulnerability knowledge base; Step S5: constructing an agent; based on the environment state information matrix and the potential vulnerability information obtained from the vulnerability knowledge base, determine the behavior strategy of the agent; Step S6: Execute specific behavior based on the behavior strategy of the agent, and apply the specific behavior to the environment to be tested. According to the result of the behavior execution and the impact on the environment, the reward module calculates the reward information and feeds it back to the agent. , guide the agent to update the strategy; go to step S3. 2. The method according to claim 1, wherein the vulnerability ontology is characterized based on the concept, attribute and relationship of the vulnerability ontology, and the vulnerability ontology refers to specific implementations of hardware, software, protocols or system security policies. The attributes of the vulnerability ontology refer to the potential conditions for the existence of the vulnerability, and the relationship of the vulnerability ontology refers to the interaction relationship between the vulnerabilities, wherein the attributes of the vulnerability ontology include the exploiting method of the vulnerability, the vulnerability The resulting effect and impact, whether there is a vulnerability exploit, whether the service contains the vulnerability, and the operating system corresponding to the service containing the vulnerability, the relationship includes an intersection relationship, an inheritance relationship, and an attribute relationship. 3. The method of claim 2, wherein the step S2, wherein: The preset target is to realize network security detection of a specific target, including network security detection of a specific host in a network environment and/or network security detection of a single host starting from a starting host. 4. The method of claim 3, wherein the environment state information matrix to be tested is defined as follows:

Among them, h _i h _j represents the connection relationship between the i-th host to be tested and the j-th host to be tested, 0 means that the hosts to be tested are not connected, 1 means that the hosts to be tested are connected, and h _{i hi} means For the permission level obtained on the host i to be tested, since the number of network nodes in different network security detection environments is different, the number of nodes is set to a fixed value, p _k (h _i ) indicates whether the host i to be tested contains a serial number is the attribute of k, and privilege(h _i ) represents the authority of the agent on the host i to be tested. 5. An intelligent network security detection device incorporating prior knowledge, wherein the device comprises: Vulnerability knowledge base building module: configured to define the representation form of the vulnerability ontology, the vulnerability ontology is represented based on the concepts, attributes and relationships of the vulnerability ontology; based on the acquired knowledge source and the representation form of the vulnerability ontology, from the acquired Extracting vulnerability knowledge from the knowledge source; storing the extracted vulnerability knowledge to form a vulnerability knowledge base; Detection module: configured to build an environmental information acquisition module, the environmental information acquisition module is used to acquire the following basic information from the environment to be tested: operating host IP, operating system, survival port, service information, and the acquired basic information Stored by category number; Judging module: configured to judge whether a preset target is reached, and the preset target is to realize network security detection for a specific target; State information matrix building module: configured to obtain the basic information of the environment to be tested based on the environment information acquisition module; based on the number information, obtain the network topology, host authority and host configuration information of the environment to be tested, and construct the state information matrix; using the state information matrix as the input of the agent and the vulnerability knowledge base; Behavior determination module: configured to construct an agent; based on the environment state information matrix and the potential vulnerability information obtained from the vulnerability knowledge base, determine the behavior strategy of the agent; Update module: configured to be based on the behavior strategy of the agent, execute specific behavior, and apply the specific behavior to the environment to be tested. The agent guides the agent to update the strategy; triggers the judgment module. 6. An intelligent network security detection system integrating prior knowledge is characterized in that, comprising: a processor for executing multiple instructions; memory for storing multiple instructions; Wherein, the plurality of instructions are used to be stored by the memory and loaded by the processor to execute the method according to any one of claims 1-4. 7. A computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used to be loaded and executed by a processor as described in any one of claims 1-4. described method.

Images