CN113434869A - Data processing method and AI system based on threat perception big data and artificial intelligence - Google Patents

Data processing method and AI system based on threat perception big data and artificial intelligence Download PDF

Info

Publication number
CN113434869A
CN113434869A CN202110770377.5A CN202110770377A CN113434869A CN 113434869 A CN113434869 A CN 113434869A CN 202110770377 A CN202110770377 A CN 202110770377A CN 113434869 A CN113434869 A CN 113434869A
Authority
CN
China
Prior art keywords
threat
data
perception
past
threat perception
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202110770377.5A
Other languages
Chinese (zh)
Inventor
张倩
田俭
莫晓东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Tianyue Technology Information Co ltd
Original Assignee
Guangzhou Tianyue Technology Information Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Tianyue Technology Information Co ltd filed Critical Guangzhou Tianyue Technology Information Co ltd
Priority to CN202110770377.5A priority Critical patent/CN113434869A/en
Publication of CN113434869A publication Critical patent/CN113434869A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application provides a data processing method and an AI system based on threat perception big data and artificial intelligence, which are characterized in that basic threat perception data and first basic derived threat perception data and second basic derived threat perception data corresponding to basic threat perception situations of the basic threat perception data are adopted to match data derived services, so that first past threat perception data and first derived threat perception data which are matched are obtained, and data integration is performed. Data islands caused in an AI training process can be avoided, threat perception data related to basic threat perception data, namely matched first past threat perception data and first derivative threat perception data, can be derived abundantly, and therefore data integration is carried out on the basic threat perception data, and the feature quantity of follow-up data mining is increased.

Description

Data processing method and AI system based on threat perception big data and artificial intelligence
Technical Field
The application relates to the technical field of big data and information security, in particular to a data processing method and an AI system based on threat perception big data and artificial intelligence.
Background
With the explosive development of big data, the importance of information security is self-evident. Situation awareness is an ability of understanding security risks dynamically and integrally based on environment, and is a way of improving discovery, identification, understanding, analysis and response handling capabilities of security threats from a global perspective based on threat awareness big data, and finally is a way of falling on the ground of security capabilities for decision making and action. Namely, the perception and understanding of the constituent components in the environment in a certain time and space, and then predicting the subsequent change condition of the components.
With the gradual application and development of the artificial intelligence technology, the subsequent AI training process is carried out by collecting relevant basic threat perception data, so that effective information safety guarantee can be realized conveniently by adopting an artificial intelligence recognition mode. However, in the related art, only the basic threat perception data is subjected to subsequent processing, and the feature quantity of data mining is sparse, so that data islanding is easily caused in the subsequent application process, for example, in an AI training process.
Disclosure of Invention
In order to overcome at least the foregoing disadvantages in the prior art, the present application aims to provide a data processing method and AI system based on threat awareness big data and artificial intelligence.
In a first aspect, the present application provides a data processing method based on threat awareness big data and artificial intelligence, which is applied to an AI system, where the AI system performs data interaction with a plurality of threat awareness cloud protection systems, and the method includes:
acquiring basic threat perception data of the threat perception cloud protection system, and extracting threat perception situation of the basic threat perception data to obtain a basic threat perception situation corresponding to the basic threat perception data;
acquiring first basic derived threat perception data corresponding to the basic threat perception data and second basic derived threat perception data corresponding to a basic threat perception situation, and acquiring data derived services of the threat perception cloud protection system, wherein a plurality of first past threat perception data and derived databases corresponding to the first past threat perception data are configured in the data derived services, and the derived databases comprise first derived threat perception data corresponding to the first past threat perception data;
matching the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data with first past threat perception data and first derived threat perception data in the data derived service respectively to obtain the first past threat perception data and the first derived threat perception data which are matched with the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data;
and performing data integration on the basic threat perception data according to the first past threat perception data and the first derived threat perception data obtained by matching.
On this basis, the embodiment of the application further provides a data processing method based on threat awareness big data and artificial intelligence, which is applied to an AI system, wherein the AI system performs data interaction with a plurality of threat awareness cloud protection systems, and the method comprises the following steps:
acquiring the basic threat perception data and subordinate perception data which is subjected to data integration with the basic threat perception data, and summarizing the basic threat perception data and the subordinate perception data into target perception data;
obtaining a key perception data group corresponding to each security protection simulation strategy of the target perception data in a first security protection upgrading stage, wherein the first security protection upgrading stage comprises at least two security protection simulation strategies, and the key perception data group corresponding to each security protection simulation strategy comprises threat attack perception data of a target threat attack channel sensed by threat perception microservices in the target perception data in the corresponding security protection simulation strategies;
determining a sensing data node network between key sensing data clusters corresponding to each safety protection simulation strategy in the first safety protection upgrading stage;
determining a threat perception thermodynamic diagram of the target perception data in the first security protection upgrading stage according to perception data node networks among key perception data clusters corresponding to each security protection simulation strategy in the first security protection upgrading stage;
determining perception capability evaluation information of the target perception data in the first security protection upgrading stage according to the threat perception thermodynamic diagram;
in an embodiment that can be implemented independently, the step of obtaining a key sensing data group corresponding to each security protection simulation policy of target sensing data in a first security protection upgrade stage includes:
acquiring threat attack sensing data of a target threat attack channel captured in a set data area after a first security protection simulation strategy is started by a threat sensing micro-service in the target sensing data, and determining a key sensing data group corresponding to the first security protection simulation strategy according to the threat attack sensing data of the target threat attack channel captured in the set data area after the first security protection simulation strategy is started by the threat sensing micro-service in the target sensing data, wherein the first security protection simulation strategy is any one security protection simulation strategy in a first security protection upgrading stage;
under the condition that a threat perception microservice in the target perception data sets a data area without capturing a target threat attack channel after a second security protection simulation strategy is started, determining a key perception data group corresponding to the second security protection simulation strategy according to threat attack perception data of the target threat attack channel received by the threat perception microservice in the target perception data, wherein the second security protection simulation strategy is any one security protection simulation strategy except the first security protection simulation strategy in the first security protection upgrading stage;
in a separately implementable embodiment, the method further comprises:
the threat-aware microservices in the target-aware data do not capture a target threat attack channel in a set data area after the third security protection simulation strategy is enabled, and the key perception data cliques corresponding to the security protection simulation strategies of the first target quantity which are continuous before the third security protection simulation strategy are all determined according to the threat attack perception data of the target threat attack channel received by the threat perception micro-service, a target threat attack channel capturing request is sent to the threat perception micro-service, such that the threat-aware microservice captures a target threat attack channel in response to the target threat attack channel capture request, the third security protection simulation strategy is any one of the first security protection simulation strategy and the second security protection simulation strategy in the first security protection upgrading stage;
and obtaining threat attack perception data of the target threat attack channel captured by the threat perception micro-service responding to the target threat attack channel capturing request, and determining a key perception data group corresponding to the third security protection simulation strategy according to the threat attack perception data of the target threat attack channel captured by the threat perception micro-service responding to the target threat attack channel capturing request.
For example, in an embodiment that can be implemented independently, the step of determining a network of aware data nodes between key aware data blobs corresponding to the security protection simulation policies in the first security protection upgrade stage includes:
determining a dynamic threat attack perception data cluster from key perception data clusters corresponding to each safety protection simulation strategy in a first safety protection upgrading stage;
respectively determining each key sensing data cluster except the dynamic threat attack sensing data cluster in the key sensing data clusters corresponding to each safety protection simulation strategy in the first safety protection upgrading stage and a sensing data node network between the dynamic threat attack sensing data cluster;
or respectively determining a sensing data node network between key sensing data clusters corresponding to each two related safety protection simulation strategies in the first safety protection upgrading stage;
the sensing data node network comprises a plurality of sensing data nodes and sensing relation data among the sensing data nodes, the sensing data nodes comprise sensing data areas and threat sensing data tracks corresponding to the sensing data areas, and the sensing relation data comprise sensing relation attributes among the sensing data nodes.
For example, in an independently implementable embodiment, the key-aware data cliques corresponding to each security simulation policy in the first security upgrade stage include migratable key-aware data cliques and non-migratable key-aware data cliques, and the threat-aware thermodynamic diagram includes a first threat-aware thermodynamic diagram determined according to the aware data node networks corresponding to the migratable key-aware data cliques of each security simulation policy specified in the first security upgrade stage, and a second threat-aware thermodynamic diagram determined according to the aware data node networks corresponding to the non-migratable key-aware data cliques of each security simulation policy specified in the first security upgrade stage;
the step of determining perception capability evaluation information of the target perception data in the first security protection upgrading stage according to the threat perception thermodynamic diagram comprises the following steps:
and determining perception capability evaluation information of the target perception data in the first security protection upgrading stage according to the first threat perception thermodynamic diagram and the second threat perception thermodynamic diagram.
For example, in an independently implementable embodiment, the step of determining the threat awareness thermodynamic diagram of the target awareness data in the first security protection upgrade stage according to the awareness data node network between the key awareness data cliques corresponding to each security protection simulation policy in the first security protection upgrade stage includes:
determining at least one target migratable key perception data cluster with perception capability value corresponding to perception capability evaluation information of derivative data features of a target threat attack channel higher than a first value threshold and at least one target non-migratable key perception data cluster with perception capability value corresponding to perception capability evaluation information of derivative data features of the target threat attack channel higher than a second value threshold from key perception data clusters corresponding to all safety protection simulation strategies in the first safety protection upgrading stage;
determining the first threat perception thermodynamic diagram according to a perception data node network corresponding to the at least one target migratable key perception data group, and determining the second threat perception thermodynamic diagram according to a perception data node network corresponding to the at least one target non-migratable key perception data group;
wherein the step of determining perception capability assessment information of the target perception data in the first security protection upgrade stage according to the first threat perception thermodynamic diagram and the second threat perception thermodynamic diagram comprises:
determining perception capability evaluation information of the target perception data in the first safety protection upgrading stage as first perception capability evaluation information under the condition that the sparsity of a thermal area of the first threat perception thermodynamic diagram is not smaller than a preset first target sparsity and the sparsity of a thermal area of the second threat perception thermodynamic diagram is not smaller than a preset second target sparsity;
determining perception capability evaluation information of the target perception data in the first safety protection upgrading stage as second perception capability evaluation information under the condition that the sparsity of a thermal area of the first threat perception thermodynamic diagram is not smaller than the first target sparsity and the sparsity of a thermal area of the second threat perception thermodynamic diagram is smaller than the second target sparsity;
and determining the perception capability evaluation information of the target perception data in the first safety protection upgrading stage as third perception capability evaluation information under the condition that the sparsity of the thermodynamic region of the first threat perception thermodynamic diagram is smaller than the first target sparsity and the sparsity of the thermodynamic region of the second threat perception thermodynamic diagram is smaller than the second target sparsity.
For example, in an embodiment, if the perceptual capability evaluation information is a third perceptual capability evaluation information, the method further includes:
acquiring N threat perception intelligence sets corresponding to the third perception capability evaluation information and an intelligence label cluster corresponding to each threat perception intelligence set, wherein each threat perception intelligence set comprises M different key threat perception intelligence, and N and M are positive integers greater than or equal to 1;
determining current frequent intelligence labels corresponding to the threat perception intelligence sets in intelligence label clusters corresponding to the threat perception intelligence sets;
extracting the information label characteristics by adopting the current frequent information labels corresponding to the threat perception information sets to obtain the information label characteristics of each key threat perception information in the threat perception information sets;
based on the intelligence label characteristics of each key threat perception intelligence in N threat perception intelligence sets, conducting extension on the current frequent intelligence labels corresponding to the threat perception intelligence sets to obtain real-time extension intelligence labels corresponding to the threat perception intelligence sets;
adding the real-time extension intelligence label corresponding to the threat perception intelligence set into an intelligence label cluster corresponding to the threat perception intelligence set;
returning and executing the steps to determine the current frequent intelligence labels corresponding to the threat perception intelligence sets in the intelligence label clusters corresponding to the threat perception intelligence sets until the global perception heat power corresponding to the N threat perception intelligence sets is greater than the set perception heat power, and obtaining the update information of the threat perception intelligence intervals corresponding to the N threat perception intelligence sets according to the global perception heat power;
such as in an independently implementable embodiment, the determining current frequent intelligence tags for the threat-aware intelligence sets in intelligence tag clusters corresponding to the threat-aware intelligence sets comprises:
determining an associated frequent intelligence label corresponding to the threat perception intelligence set, current threat perception intelligence interval information and current threat perception intelligence interval information corresponding to the target threat perception intelligence set;
comparing current threat perception information interval information corresponding to the threat perception information sets with current threat perception information interval information corresponding to a target threat perception information set to obtain first coverage range information of the current threat perception information interval information corresponding to the threat perception information sets, wherein the target threat perception information set is all threat perception information sets including the threat perception information sets in the N threat perception information sets;
obtaining second coverage range information of the current threat perception information interval information of the threat perception information set by comparing the current threat perception information interval information corresponding to the threat perception information set with the associated frequent information labels corresponding to the threat perception information set;
based on the second coverage range information and the first coverage range information, determining an associated frequent intelligence label corresponding to the threat perception intelligence set or current threat perception intelligence interval information corresponding to the threat perception intelligence set as an intelligence label corresponding to a current time sequence node of the threat perception intelligence set;
in a second aspect, an embodiment of the present application further provides a data processing system based on threat awareness big data and artificial intelligence, where the data processing system based on threat awareness big data and artificial intelligence includes an AI system and multiple threat awareness cloud protection systems communicatively connected to the AI system;
the AI system is configured to:
acquiring basic threat perception data of the threat perception cloud protection system, and extracting threat perception situation of the basic threat perception data to obtain a basic threat perception situation corresponding to the basic threat perception data;
acquiring first basic derived threat perception data corresponding to the basic threat perception data and second basic derived threat perception data corresponding to a basic threat perception situation, and acquiring data derived services of the threat perception cloud protection system, wherein a plurality of first past threat perception data and derived databases corresponding to the first past threat perception data are configured in the data derived services, and the derived databases comprise first derived threat perception data corresponding to the first past threat perception data;
matching the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data with first past threat perception data and first derived threat perception data in the data derived service respectively to obtain the first past threat perception data and the first derived threat perception data which are matched with the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data;
and performing data integration on the basic threat perception data according to the first past threat perception data and the first derived threat perception data obtained by matching.
According to any one of the aspects, in the embodiment provided by the application, the data derivative service is matched by using the first basic derivative threat perception data and the second basic derivative threat perception data corresponding to the basic threat perception data and the basic threat perception situation thereof, so as to obtain the matched first past threat perception data and first derivative threat perception data, thereby performing data integration. In the process, AI analysis on threat perception data is not needed, and derivative threat perception data related to the threat perception data in the aspects of derivative data characteristics and the like can be considered, so that a data island caused in an AI training process can be avoided, threat perception data related to basic threat perception data, namely first past threat perception data and first derivative threat perception data which are matched with each other, can be derived more abundantly, and the number of characteristics of subsequent data mining is increased by performing data integration on the basic threat perception data.
Drawings
FIG. 1 is a schematic diagram of an application scenario of a data processing system based on threat-aware big data and artificial intelligence provided in an embodiment of the present application;
FIG. 2 is a schematic flowchart of a data processing method based on threat awareness big data and artificial intelligence provided in an embodiment of the present application;
fig. 3 is a schematic block diagram of an AI system for implementing the foregoing data processing method based on threat awareness big data and artificial intelligence according to an embodiment of the present application.
Detailed Description
Fig. 1 is a schematic diagram of an application scenario of a data processing system 10 based on threat-aware big data and artificial intelligence according to an embodiment of the present application. The threat-aware big data and artificial intelligence based data processing system 10 may include an AI system 100 and a threat-aware cloud protection system 200 communicatively coupled to the AI system 100. The threat-aware big data and artificial intelligence based data processing system 10 shown in fig. 1 is but one possible example, and in other possible embodiments, the threat-aware big data and artificial intelligence based data processing system 10 may include only at least some of the components shown in fig. 1 or may include additional components.
The AI system 100 and the threat awareness cloud protection system 200 in the threat awareness big data and artificial intelligence based data processing system 10 may cooperatively perform the data processing method based on threat awareness big data and artificial intelligence described in the following method embodiments, and the detailed description of the following method embodiments may be referred to in the steps of the AI system 100 and the threat awareness cloud protection system 200.
Fig. 2 is a schematic flow chart of a data processing method based on threat awareness big data and artificial intelligence according to an embodiment of the present application, where the data processing method based on threat awareness big data and artificial intelligence according to the present embodiment may be executed by the AI system 100 shown in fig. 1, and the data processing method based on threat awareness big data and artificial intelligence is described in detail below.
Step S101, obtaining basic threat perception data of the threat perception cloud protection system 200, and performing threat perception situation extraction on the basic threat perception data to obtain a basic threat perception situation corresponding to the basic threat perception data.
Step S102, first basic derived threat perception data corresponding to basic threat perception data and second basic derived threat perception data corresponding to basic threat perception situations are obtained, data derived services of the threat perception cloud protection system are obtained, a plurality of first past threat perception data and a plurality of derived databases corresponding to the first past threat perception data are configured in the data derived services, and the derived databases comprise the first derived threat perception data corresponding to the first past threat perception data.
It can be understood that the training service on the threat awareness cloud protection system 200 may upload a segment of threat awareness data through an API provided by the data integration service, and after acquiring a segment of threat awareness data produced by the training service, the data integration service uses the segment of threat awareness data as basic threat awareness data to perform the data integration operation of this embodiment.
For example, the data integration service may extract the threat awareness situation from the basic threat awareness data, and may specifically extract the threat awareness situation from the basic threat awareness data by using a threat awareness situation extraction application, so as to obtain the basic threat awareness situation. For another example, because some basic threat perception statuses are not too valuable in the security protection environment, in order to simplify the processing of the basic threat perception statuses, the data integration service may rule the basic threat perception statuses, for example, may filter the non-valuable basic threat perception statuses such as the perception statuses of the infinite loop state in the obtained basic threat perception statuses, to obtain the remaining basic threat perception statuses.
In this embodiment, the threat perception situation may refer to specific perception information for predicting the subsequent change conditions of each component in the service environment by perceiving and understanding each component in a certain time and space.
The data integration service may then obtain first base derived threat awareness data of the base threat awareness data, where the first base derived threat awareness data is threat awareness data associated with the base threat awareness data in dimensions of a security protection environment, a security protection policy, and the like, for example, the derived threat awareness data of the "threat awareness behavior for abnormal access" may include "threat awareness behavior for information abnormal verification before access" and the like. The data integration service also obtains second basic derived threat perception data of the basic threat perception situation, wherein the second basic derived threat perception data are threat perception data related to the basic threat perception situation in dimensions such as a safety protection environment and a safety protection strategy. For example, derived threat awareness data for "threat awareness behavior for anomalous tampering" may include "threat awareness behavior for non-validated software updates" and "threat awareness behavior for non-certified information upgrades" and the like.
The data integration service may obtain data derivative services before executing the data processing method based on threat awareness big data and artificial intelligence of this embodiment, and configure the data derivative services into the current data relay area.
Each first past threat perception data derivative database in the data derivative service refers to a derivative database of derivative threat perception data such as data scheduling performed on the first past threat perception data by the training service, for example, scheduling frequency information, scheduling path information, first derivative threat perception data, and other information, where the scheduling frequency information may not only indicate a frequency parameter of the first past threat perception data scheduled by the training service, but also indicate whether a perception service node of the first past threat perception data is a key service node, and may be divided into: a high scheduling frequency parameter, a general scheduling frequency parameter, and a low scheduling frequency parameter.
In a design idea, data derived services can be acquired according to past scheduling records in a target scheduling container, wherein when training services input past scheduling threat perception data in the target scheduling container, and the target scheduling container performs data scheduling according to the past scheduling threat perception data to obtain past scheduling contents, the target scheduling container can record a past scheduling log, wherein the past scheduling log comprises the past scheduling threat perception data and corresponding past scheduling contents, and thus, the past scheduling log recorded by the target scheduling container in a period of time is the past scheduling record.
It should be noted that the target scheduling container and the data integration service currently executing the data processing method flow based on the threat awareness big data and the artificial intelligence may be different execution subjects independent from each other, so that the data integration service may analyze data in other data integration systems (referred to as target scheduling container in this embodiment), and after obtaining the data derivative service by referring to the data in the target scheduling container, configure the data derivative service into the current data integration service, so that the data integration service performs data integration on the obtained basic threat awareness data in real time according to the data derivative service.
For example, the data aggregation service may first obtain a past dispatch log in the target dispatch container, where the past dispatch log includes past dispatch threat awareness data and past dispatch content of the target dispatch container to the past dispatch threat awareness data. And acquiring a past threat perception situation corresponding to the past scheduling threat perception data. According to the past scheduling content corresponding to the past scheduling threat perception data, counting derived threat perception data corresponding to the past threat perception situation and the past scheduling threat perception data respectively, wherein the derived threat perception data corresponding to the past threat perception situation and the past scheduling threat perception data respectively are first derived threat perception data configured in the data derived service, and the past threat perception situation and the past scheduling threat perception data are first past threat perception data configured in the data derived service.
In general, in a data integration process, a target scheduling container may also extract a threat awareness situation from past scheduling threat awareness data to obtain a past threat awareness situation, where the obtained past scheduling content includes a plurality of first past scheduling information obtained by past scheduling based on the whole past scheduling threat awareness data and a plurality of second past scheduling information obtained by extracting past scheduling based on each past threat awareness situation, and when the data integration service in this embodiment counts derived threat awareness data corresponding to the past scheduling threat awareness data, it may be determined that scheduling path information and scheduling frequency information of any one first past scheduling information satisfy a target condition, and when any one first past scheduling information includes derived threat awareness data corresponding to the past scheduling threat awareness data, it may be determined according to any one first past scheduling information that the derived threat awareness data corresponding to the past scheduling threat awareness data corresponds to the past scheduling threat awareness data And (4) data.
The condition that the scheduling path information and the scheduling frequency information of any first past scheduling information meet the target condition means that all first past scheduling information is compared, the scheduling sequence and the scheduling frequency parameter of any first past scheduling information are higher, for example, the scheduling sequence is N before, and the scheduling frequency parameter is M before, so that the first past scheduling information with the scheduling sequence arranged earlier and the scheduling frequency parameter higher can be selected, or the first past scheduling information with the scheduling sequence arranged earlier and the scheduling frequency parameter common is selected.
In addition, when determining whether any first past scheduling information contains derivative threat sensing data corresponding to past scheduling threat sensing data, the past scheduling threat sensing data may be matched with key header information of the first past scheduling information, so as to determine whether the first past scheduling information contains the corresponding derivative threat sensing data. For example, most threat awareness data in past scheduling threat awareness data continuously appears in first past scheduling information, relevant threat awareness data in the first past scheduling information is used as derived threat awareness data corresponding to the past scheduling threat awareness data, for example, the past scheduling threat awareness data is a threat awareness behavior aiming at abnormal tampering, a threat awareness behavior aiming at non-authenticated information upgrading appears in a certain piece of first past scheduling information, and a threat awareness behavior aiming at non-verified software updating appears in another piece of first past scheduling information, so that the threat awareness behavior aiming at non-authenticated information upgrading and the threat awareness behavior aiming at non-verified software updating are derived threat awareness data corresponding to the past scheduling threat awareness data.
Similarly, when the data integration service counts the derived threat perception data corresponding to the past threat perception situation, it is determined that when the scheduling path information and the scheduling frequency information of any one second past scheduling information meet the target condition, and any one second past scheduling information contains the derived threat perception data corresponding to the past threat perception situation, the derived threat perception data corresponding to the past threat perception situation is determined according to any one second past scheduling information.
For another example, the data integration service may further determine scheduling frequency information and scheduling path information corresponding to the past threat awareness situation and the past scheduling threat awareness data, respectively, and the derived database of the first past threat awareness data further includes the scheduling frequency information and the scheduling path information corresponding to the past threat awareness situation and the past scheduling threat awareness data, respectively.
For example, a group of past scheduling logs obtained from the target scheduling container includes past scheduling threat awareness data a, corresponding past threat awareness situations of which are a1 and a2, first past scheduling information corresponding to the past scheduling threat awareness data a includes Bi, second past scheduling information corresponding to a past threat awareness situation T1 is Cj, second past scheduling information corresponding to a past threat awareness situation T2 is Dk, where i, j, and k are natural numbers greater than 0. Therefore, first past scheduling information with a front scheduling sequence and high scheduling frequency parameters is determined from the first past scheduling information Bi, and further derived threat sensing data corresponding to the past scheduling threat sensing data A are obtained based on the selected first past scheduling information. And similarly, obtaining derived threat perception data corresponding to the past threat perception situation T1 according to the second past scheduling information Cj, and obtaining derived threat perception data corresponding to the past threat perception situation T2 according to the second past scheduling information Rk.
Step S103, matching the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data with first past threat perception data and first derived threat perception data in the data derived service acquired in the step S103 respectively to obtain first past threat perception data and first derived threat perception data matched with the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data.
For example, the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data are respectively matched with each first past threat perception data and the first derived threat perception data thereof in the data derived service, so as to obtain the first past threat perception data and the first derived threat perception data which are matched with the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data.
In this process, the data integration service may combine other derivative databases of each first past threat awareness data, such as scheduling path information and scheduling frequency information, and select, from the obtained first past threat awareness data and first derivative threat awareness data that match with each other, the first past threat awareness data and the first derivative threat awareness data whose scheduling path information and scheduling frequency information satisfy the target condition, such as selecting the first past threat awareness data and the first derivative threat awareness data whose scheduling order is ranked earlier and whose scheduling frequency parameter is higher (or whose scheduling frequency parameter is general).
And step S104, performing data integration on the basic threat perception data according to the first past threat perception data and the first derived threat perception data obtained by matching in the step S103.
For example, the data integration service may directly output the first past threat awareness data and the first derived threat awareness data obtained by matching in the foregoing step S103, so as to perform data integration on the basic threat awareness data. Or, the data integration service may apply the first past threat perception data and the first derived threat perception data obtained by the matching to a data integration process of some specific information, for example, data integration of a program, data integration of information such as threat perception data, and the like is performed, so that subsequent AI training is performed after the integration is collected, and the feature quantity of data mining may be improved.
It should be noted that, through the foregoing steps S101 to S104, threat awareness data related to the basic threat awareness data in terms of derived data features and the like can be obtained (i.e., the first past threat awareness data and the first derived threat awareness data obtained by matching in the foregoing step S104), thereby, when data integration is carried out based on the acquired relevant threat perception data, the data integration range is wider, the data precision of the data integration information is also improved, since AI analysis of the underlying threat awareness data and its underlying threat awareness situation itself is not required, and the basic derived threat awareness data directly associated with the basic threat awareness data and the basic threat awareness situation describes the basic threat awareness data and the basic threat awareness situation from the aspects of derived data characteristics and the like, the data range of final data integration is wider, and meanwhile, data isolated islands caused in an AI training process are avoided. For example, when the underlying threat awareness data is new threat awareness data information, the AI analysis may be limited by certain rules and the like.
In order to further improve the precision of the data integration information, the data integration service can also extract corresponding protection environment derivative databases, namely a protection environment derivative database of the first past threat perception data, according to past scheduling threat perception data, past threat perception situations and a preset protection environment network in the target scheduling container, and the corresponding protection environment derivative databases are configured in the data integration service.
When the training service uploads the basic threat perception data in the data integration service in real time, the data integration service acquires the protection environment derivative databases corresponding to the basic threat perception data and the basic threat perception situation respectively according to the basic threat perception data, the basic threat perception situation and the preset protection environment network. And then matching the determined protection environment derived database with a protection environment derived database of first past threat perception data configured in the system, so as to select the first past threat perception data with higher coincidence degree between the protection environment derived database and the protection environment derived database of the basic threat perception data (or the basic threat perception situation). And finally, performing data integration of related information according to the selected first past threat perception data. The protection environment network is mainly used for extracting protection environment characteristics of basic threat perception data and basic threat perception situations, is an AI network obtained through training and is preset into a data integration service.
For the training of the protection environment network, configuration needs to be performed based on collected training samples, where the training samples need to include: the past threat perception data and the corresponding protection environment labeling information of the past threat perception data, so that a large number of training samples with exact protection environments (namely, the past threat perception data with the protection environment labeling information) are needed to train a relatively accurate protection environment network.
It can be seen that, in the method of this embodiment, the data integration service matches the data derivative service by using the first basic derivative threat awareness data and the second basic derivative threat awareness data corresponding to the basic threat awareness data and the basic threat awareness situation thereof, respectively, so as to obtain the first past threat awareness data and the first derivative threat awareness data that are matched, thereby performing data integration. In the process, AI analysis is not needed to be carried out on the threat perception data, but derivative threat perception data related to the threat perception data in the aspects of derivative data characteristics and the like can be considered, so that a data island caused in an AI training process can be avoided, the threat perception data related to the basic threat perception data, namely the matched first past threat perception data and first derivative threat perception data, can be derived abundantly, and data integration is carried out on the basic threat perception data.
In an embodiment that can be implemented independently, another embodiment of the present application provides a data processing method based on threat-aware big data and artificial intelligence, and different from the foregoing embodiment, the method in this embodiment mainly performs data integration according to a threat-aware data cluster, and in the foregoing method, data integration of relevant information of basic threat-aware data is performed according to a data derivation service, and the method in this embodiment includes the following steps:
step S201, basic threat perception data are obtained, threat perception situation extraction is carried out on the basic threat perception data, and basic threat perception situations of the basic threat perception data are obtained.
Step S202, a threat perception data cluster is obtained, wherein the threat perception data cluster comprises a plurality of second past threat perception data and information whether integration and labeling are carried out between any two second past threat perception data based on threat perception categories.
The data integration service may obtain the threat awareness data cluster before executing the data processing method flow based on the threat awareness big data and the artificial intelligence of this embodiment, and configure the threat awareness data cluster into the current data transfer area.
In a design idea, when the data integration service acquires a threat awareness data cluster, a target scheduling container may be obtained by analyzing past scheduling records, for example, the data integration service may acquire a past scheduling log in the target scheduling container, where the past scheduling log includes past scheduling threat awareness data. The method comprises the steps of obtaining a past threat perception situation corresponding to past scheduling threat perception data, obtaining the past scheduling threat perception data and the past threat perception situation, and extracting corresponding derivative databases, wherein the derivative databases specifically comprise scheduling path information, scheduling frequency information, derivative threat perception data and the like. Extracting corresponding derivative databases according to the past scheduling threat perception data and the past threat perception situation to extract the past scheduling threat perception data and the past threat perception situation for threat perception category clustering, and obtaining second past threat perception data of a plurality of threat perception categories. And integrating any two second past threat perception data in the same threat perception category to obtain a threat perception data cluster.
When extracting the corresponding derivative databases according to the past scheduling threat perception data and the past threat perception situation to perform threat perception category clustering on the past scheduling threat perception data and the past threat perception situation, the corresponding derivative databases can be extracted from the past scheduling threat perception data and the past threat perception situation to perform encoding, and then threat perception category clustering is performed by adopting a threat perception category clustering algorithm. For example, expected protection environments of second past threat awareness data may be found, and second past threat awareness data belonging to the same protection environment may be clustered, and threat awareness category clustering may be performed. Wherein the threat awareness category of the second past threat awareness data refers to an attribute of the second past threat awareness data.
For another example, the data consolidation service may also determine a threat awareness intersection between second past threat awareness data for any two of the plurality of threat awareness classes. And integrating second past threat perception data of any two threat perception categories with the threat perception cross degree meeting a target condition (for example, the threat perception cross degree is greater than a preset cross degree). The threat perception cross degree can refer to parameters such as contact degree and the like for describing the similarity between two threat perception data, and therefore second past threat perception data in two threat perception categories with higher similarity can be integrated.
Step S203, the basic threat perception data and the basic threat perception situation are respectively matched with second past threat perception data in the threat perception data cluster, and second past threat perception data associated with the basic threat perception data and the basic threat perception situation are obtained.
For example, if the base threat awareness data (or the base threat awareness situation) matches a second past threat awareness data in the threat awareness data cluster, the second past threat awareness data associated with the base threat awareness data and the base threat awareness situation specifically includes: some second past threat awareness data that matches the underlying threat awareness data or the underlying threat awareness situation, and further comprising: and other second past threat perception data which are subjected to integration labeling based on threat perception categories are arranged between the threat perception data cluster and certain second past threat perception data.
For example, assume that a threat awareness data cluster includes a plurality of threat awareness data nodes, each threat awareness data node corresponds to a second past threat awareness data, and a connection attribute between the threat awareness data nodes represents that the corresponding second past threat awareness data are associated with each other based on a threat awareness category. For example, the second past threat awareness data represented by the threat awareness data nodes A, B, C and D belongs to the threat awareness class 1, the second past threat awareness data represented by the threat awareness data nodes E and F belongs to the threat awareness class 2, the second past threat awareness data represented by the threat awareness data nodes G, H and I belongs to the threat awareness class 3, and for example, the threat awareness cross degree between the threat awareness class 1 and the second past threat awareness data represented by the threat awareness class 2 satisfies the target condition, the two threat awareness classes are integrated, and the threat awareness classes 1 and 2 are not respectively associated with the threat awareness class 3. Thus, when the basic threat awareness data uploaded by the training service or the basic threat awareness situation corresponding to the basic threat awareness data is matched with the second past threat awareness data represented by the threat awareness data node E in the threat awareness category 2, the finally obtained associated second past threat awareness data includes: threat awareness data nodes A, B, C, D, E and F represent second past threat awareness data.
And step S204, integrating relevant threat perception categories of the basic threat perception data according to the second past threat perception data obtained by matching in the step S203.
For example, the data integration service may directly output the second past threat awareness data obtained by matching in step S203 to integrate the relevant threat awareness categories with the basic threat awareness data. Or, the data integration service may apply the second past threat awareness data obtained by the matching to a data integration process of some specific information, for example, perform data integration of a program, perform data integration of information such as threat awareness data, and so on, and perform subsequent AI training after collection, thereby improving the feature quantity of data mining.
Therefore, the basic threat perception data and the basic threat perception situation thereof are respectively matched with the threat perception data cluster, second past threat perception data matched with the basic threat perception data or the basic threat perception situation thereof based on the threat perception categories can be obtained, and the direct protection environment and the expected protection environment of the basic threat perception data and the basic threat perception situation can be found, so that the threat perception category data are integrated comprehensively.
In order to further improve the accuracy of the data integration information, the data integration service can also extract corresponding protection environment derivative databases respectively according to past scheduling threat perception data, past threat perception situations and preset protection environment networks in a target scheduling container, namely a second past threat perception data protection environment derivative database, then perform clustering according to the contact ratio between the second past threat perception data protection environment derivative databases, and then adjust the obtained threat perception data clusters based on the clustering result. In addition, when the training service uploads the basic threat perception data in the data integration service in real time, the data integration service acquires the protection environment derivative databases corresponding to the basic threat perception data and the basic threat perception situation respectively according to the basic threat perception data, the basic threat perception situation and the preset protection environment network. And then second past threat perception data with higher coincidence degree between the protection environment derived database and the protection environment derived databases corresponding to the basic threat perception data and the basic threat perception situation respectively are selected from the threat perception data cluster. The protection environment network is mainly used for extracting protection environment features of basic threat perception data and basic threat perception situations.
A specific application example is used below to describe the data processing method based on threat awareness big data and artificial intelligence, where the method in this embodiment is mainly applied to an application program related to integration of a basic threat awareness data track uploaded according to a training service, and the method in this embodiment mainly includes the following two parts:
and acquiring data derivative services of the threat awareness cloud protection system, and presetting the data derivative services into data integration services.
Step S301, the data integration service acquires past scheduling logs in the target scheduling container, wherein the past scheduling logs comprise past scheduling threat perception data and past scheduling contents of the target scheduling container to the past scheduling threat perception data.
Step S302, the data integration service carries out threat perception situation extraction on past scheduling threat perception data to obtain past threat perception situation extraction, and eliminates the past threat perception situation extraction without practical value in the past threat perception situation extraction.
Step S303, the data integration service acquires the past scheduling threat perception data and the past threat perception situation and extracts the derivative databases corresponding to the past scheduling threat perception data and the past threat perception situation respectively, specifically, the scheduling frequency information, the scheduling path information, the derivative threat perception data and the like can be included, then, the data derivative service can be obtained, the derivative database of the first past threat perception data included in the data derivative service is the derivative database corresponding to the past scheduling threat perception data and the past threat perception situation and extracted respectively, and the first past threat perception data included in the data derivative service is the past scheduling threat perception data and the past threat perception situation extracted.
The data integration service acquires the past scheduling threat perception data and the corresponding derivative threat perception data extracted from the past threat perception situation according to the past scheduling content of the target scheduling container to the past scheduling threat perception data, and the acquiring method is described in the foregoing embodiments and is not described herein.
Step S304, the data integration service extracts the protection environment features, such as the protection environment features, from the past scheduling threat awareness data and the past threat awareness situation, and configures extracted protection environment derivative databases, which include protection environment derivative databases corresponding to the past scheduling threat awareness data and the past threat awareness situation extraction (i.e., the first past threat awareness data). In this way, the derivative database of each first past threat awareness data in the data derivative service may further include a protection environment derivative database corresponding to each first past threat awareness data.
(2) And integrating the basic threat perception data input by the training service in real time and the acquired data derivative service.
In step S305, the data integration service provides a training service upload interface, so that the training service can upload basic threat awareness data on the training service upload interface, and the training service can select a threat awareness data type of the data integration service data integration information on the training service upload interface.
Step S306, the data integration service extracts the threat perception situation of the basic threat perception data to obtain the basic threat perception situation, and obtains first basic derived threat perception data and second basic derived threat perception data and the like corresponding to the basic threat perception data and the basic threat perception situation respectively.
For example, when the data integration service obtains the first and second basic derived threat awareness data, the data integration service may be implemented by using a deriver, and the basic threat awareness data and the basic threat awareness situation are generally complementarily derived, for example, the derived threat awareness data of "live threat awareness behavior verified against information abnormality before access" is "live jump threat awareness data of threat awareness behavior verified against information abnormality before access", "live bullet screen threat awareness data of threat awareness behavior verified against information abnormality before access", and the like.
Step S307, the data integration service matches the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data with the acquired data derived service to obtain first derived threat perception data and first past threat perception data which are matched with the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data.
Step S308, the data integration service may further determine, according to the basic threat perception data, the basic threat perception situation, and the preset protection environment network, protection environment derivative databases corresponding to the basic threat perception data and the basic threat perception situation, respectively, and then match the determined protection environment derivative databases with the protection environment derivative database of the first past threat perception data obtained in step S304, to obtain first past threat perception data with higher coincidence between the protection environment derivative databases corresponding to the basic threat perception data and the basic threat perception situation, respectively.
Step S309, the data integration service performs data integration on the basic threat perception data according to the first derivative threat perception data and the first past threat perception data acquired in step S307 and the first past threat perception data acquired in step S308, for example, directly outputs the first past threat perception data and the first derivative threat perception data to the training service.
The data processing method based on threat awareness big data and artificial intelligence is described as another specific application example, and the method in this embodiment is mainly applied to an application program related to integration of a basic threat awareness data track uploaded according to a training service, and then the method in this embodiment mainly includes the following two parts:
(1) and acquiring a threat perception data cluster, and presetting the threat perception data cluster into a data integration service.
Step S401, the data integration service acquires past scheduling logs in the target scheduling container, wherein the past scheduling logs comprise past scheduling threat perception data.
Step S402, the data integration service extracts threat perception situation of past scheduling threat perception data to obtain past threat perception situation extraction, and eliminates the past threat perception situation extraction without actual value in the past threat perception situation extraction.
Step S403, the data integration service obtains the past scheduling threat awareness data and the past threat awareness situation to extract the derivative databases respectively corresponding to the past scheduling threat awareness data and the past threat awareness situation, which may specifically include scheduling frequency information, scheduling path information, derivative threat awareness data, and the like.
Step S404, the data integration service extracts corresponding derivative databases according to the past scheduling and the past threat perception situation of the training service to perform threat perception category clustering on the past scheduling threat perception data and the past threat perception situation, and second past threat perception data of a plurality of threat perception categories are obtained.
For another example, in order to more accurately obtain a threat awareness data cluster, in the process of executing step S404, the data integration service may perform a "pruning" operation on some data in the protection environment space after mapping the past scheduling threat awareness data and the quantized features extracted from the past threat awareness situation to the protection environment space, for example, if the number of elements having a value of zero in the protection environment space corresponding to a certain past scheduling threat awareness data or a past threat awareness situation is greater than a preset cross degree, the protection environment space feature value extracted from the past scheduling threat awareness data or the past threat awareness situation is removed.
In addition, after the data integration service obtains second past threat perception data of a plurality of threat perception categories, a removing operation can be carried out, namely, some second past threat perception data in some threat perception categories are removed. For example, second past threat awareness data included in any threat awareness category that is inconsistent with the threat awareness category may be removed.
Step S405, the data integration service integrates any two second past threat awareness data in the same threat awareness category, and also determines a threat awareness intersection between the second past threat awareness data of any two threat awareness categories in the plurality of threat awareness categories. And integrating second past threat perception data of any two threat perception categories with the threat perception cross degree meeting a target condition (for example, the threat perception cross degree is greater than a preset cross degree), further obtaining a threat perception data cluster, and configuring the threat perception data cluster into a data integration service.
In step S406, the data integration service further extracts the protection environment features from the past scheduling threat perception data and the past threat perception situation, for example, extracts the protection environment features, groups the past scheduling threat perception data and the past threat perception situation extraction based on the contact ratio between the protection environment derivative databases, and adjusts the threat perception data cluster obtained in step S405 according to the grouping result.
(2) And performing data integration on the basic threat perception data input by the training service in real time and the obtained threat perception data cluster.
In step S407, the data integration service provides a training service upload interface, so that the training service can upload basic threat awareness data on the training service upload interface, and the training service can select a threat awareness data type of the data integration service data integration information on the training service upload interface.
Step S408, the data integration service carries out threat perception situation extraction on the basic threat perception data to obtain a basic threat perception situation.
Step S409, the data integration service matches the basic threat perception data and the basic threat perception situation with the obtained threat perception data cluster respectively to obtain second past threat perception data matched with the basic threat perception data or the basic threat perception situation.
Step S410, the data integration service performs program data integration of relevant threat perception categories on the basic threat perception data according to the second past threat perception data acquired in step S409.
Therefore, by the method of the embodiment, protective environment learning can be reasonably performed on each threat perception data, and the effect is good when the data are integrated.
In a separately implementable embodiment, the present application further provides another data processing method based on threat-aware big data and artificial intelligence, comprising the following steps.
Step S501, basic threat perception data and subordinate perception data which are integrated with the basic threat perception data are obtained, and the basic threat perception data and the subordinate perception data are summarized into target perception data.
Step S502, obtaining a key perception data group corresponding to each safety protection simulation strategy of the target perception data in a first safety protection upgrading stage, wherein the first safety protection upgrading stage comprises at least two safety protection simulation strategies, and the key perception data group corresponding to each safety protection simulation strategy comprises threat attack perception data of a target threat attack channel sensed by a threat perception microservice in the target perception data in the corresponding safety protection simulation strategy.
Step S503, determining a sensing data node network between key sensing data clusters corresponding to each safety protection simulation strategy in the first safety protection upgrading stage.
Step S504, determining a threat perception thermodynamic diagram of the target perception data in the first safety protection upgrading stage according to the perception data node networks among the key perception data clusters corresponding to the safety protection simulation strategies in the first safety protection upgrading stage.
And step S505, determining perception capability evaluation information of the target perception data in a first safety protection upgrading stage according to the threat perception thermodynamic diagram.
Based on the steps, key perception data cliques corresponding to all safety protection simulation strategies of target perception data in a first safety protection upgrading stage are obtained firstly, perception data node networks among the key perception data cliques corresponding to all safety protection simulation strategies in the first safety protection upgrading stage are determined secondly, threat perception thermodynamic diagrams of the target perception data in the first safety protection upgrading stage are determined secondly, and perception capability evaluation information of the target perception data in the first safety protection upgrading stage is determined finally. By the design, on one hand, global analysis of key perception data cliques can be achieved based on a perception data node network, and on the other hand, deep identification of threat attack perception data can be achieved based on a threat perception thermodynamic diagram. Therefore, various threat perception thermal units existing in the target perception data can be rapidly judged according to the key perception data group, and the key perception data can be analyzed and identified in a self-adaptive mode.
In an embodiment, step S502 can be implemented by the following steps.
Step S5021, threat and attack sensing data of a target threat and attack channel captured in a set data area after a first safety protection simulation strategy is started by a threat sensing micro-service in target sensing data is obtained, a key sensing data group corresponding to the first safety protection simulation strategy is determined according to the threat and attack sensing data of the target threat and attack channel captured in the set data area after the first safety protection simulation strategy is started by the threat sensing micro-service in the target sensing data, and the first safety protection simulation strategy is any safety protection simulation strategy in a first safety protection upgrading stage.
Step S5022, under the condition that the threat perception microservice in the target perception data sets a data area without capturing a target threat attack channel after a second safety protection simulation strategy is started, according to the threat attack perception data of the target threat attack channel received by the threat perception microservice in the target perception data, a key perception data group corresponding to a second safety protection simulation strategy is determined, and the second safety protection simulation strategy is any safety protection simulation strategy except the first safety protection simulation strategy in a first safety protection upgrading stage.
In this embodiment, a target threat attack channel capture request may also be sent to the threat awareness microserver under the condition that the threat awareness microserver in the target awareness data does not capture the target threat attack channel in the set data region after the third security protection simulation policy is enabled, and the key awareness data groups corresponding to the consecutive first-target-number security protection simulation policies before the third security protection simulation policy are determined according to the threat attack awareness data of the target threat attack channel received by the threat awareness microserver, so that the threat awareness microserver responds to the target threat attack channel capture request to capture the target threat attack channel, and the third security protection simulation policy is any one of the security protection simulation policies other than the first security protection simulation policy and the second security protection simulation policy in the first security protection upgrade stage.
Therefore, threat attack sensing data of the target threat attack channel captured by the threat perception microservice response target threat attack channel capturing request can be obtained, and a key sensing data group corresponding to the third safety protection simulation strategy is determined according to the threat attack sensing data of the target threat attack channel captured by the threat perception microservice response target threat attack channel capturing request.
In an embodiment that can be implemented independently, determining a sensing data node network between key sensing data cliques corresponding to each security protection simulation policy in a first security protection upgrade stage may specifically be: and determining a dynamic threat attack perception data cluster from key perception data clusters corresponding to each safety protection simulation strategy in the first safety protection upgrading stage. And then, respectively determining each key sensing data cluster except the dynamic threat attack sensing data cluster in the key sensing data clusters corresponding to each safety protection simulation strategy in the first safety protection upgrading stage and a sensing data node network between the dynamic threat attack sensing data cluster. Or respectively determining the sensing data node networks between the key sensing data cliques corresponding to each two related safety protection simulation strategies in the first safety protection upgrading stage.
The sensing data node network can comprise a plurality of sensing data nodes and sensing relation data among the sensing data nodes, the sensing data nodes comprise sensing data areas and threat sensing data tracks corresponding to the sensing data areas, and the sensing relation data comprise sensing relation attributes among the sensing data nodes.
In an embodiment that can be implemented independently, the key sensing data clusters corresponding to each security protection simulation policy in the first security protection upgrade stage include migratable key sensing data clusters and non-migratable key sensing data clusters, the threat sensing thermodynamic diagram includes a first threat sensing thermodynamic diagram determined according to the sensing data node networks corresponding to the migratable key sensing data clusters of each security protection simulation policy specified in the first security protection upgrade stage, and a second threat sensing thermodynamic diagram determined according to the sensing data node networks corresponding to the non-migratable key sensing data clusters of each security protection simulation policy specified in the first security protection upgrade stage.
On the basis, the perception capability evaluation information of the target perception data in the first security protection upgrading stage is determined according to the threat perception thermodynamic diagrams, and specifically, the perception capability evaluation information of the target perception data in the first security protection upgrading stage can be determined according to the first threat perception thermodynamic diagrams and the second threat perception thermodynamic diagrams.
In an embodiment, step S504 can be implemented as follows.
Step S5041, determining, from the key perceptual data clusters corresponding to the security protection simulation policies in the first security protection upgrade stage, at least one target migratable key perceptual data cluster in which the perceptual capability value corresponding to the perceptual capability evaluation information of the derived data features of the target threat attack channel is higher than the first value threshold, and at least one target non-migratable key perceptual data cluster in which the perceptual capability value corresponding to the perceptual capability evaluation information of the derived data features of the target threat attack channel is higher than the second value threshold.
Step S5042, determining a first threat awareness thermodynamic diagram according to the sensing data node network corresponding to the at least one target migratable key sensing data group, and determining a second threat awareness thermodynamic diagram according to the sensing data node network corresponding to the at least one target non-migratable key sensing data group.
The method comprises the following steps of determining perception capability evaluation information of target perception data in a first security protection upgrading stage according to a first threat perception thermodynamic diagram and a second threat perception thermodynamic diagram, and specifically may be: under the condition that the sparsity of the thermal regions of the first threat perception thermodynamic diagram (which may represent the number of threat perception thermodynamic regions with category differences) is not less than a preset first target sparsity, and the sparsity of the thermal regions of the second threat perception thermodynamic diagram is not less than a preset second target sparsity, determining the perception capability evaluation information of the target perception data in the first security protection upgrade stage as first perception capability evaluation information (namely, the first threat perception thermodynamic diagram and the second threat perception thermodynamic diagram are included). And under the condition that the sparsity of the thermodynamic region of the first threat perception thermodynamic diagram is not less than the sparsity of the first target, and the sparsity of the thermodynamic region of the second threat perception thermodynamic diagram is less than the sparsity of the second target, determining the perception capability evaluation information of the target perception data in the first safety protection upgrading stage as second perception capability evaluation information (namely, the first threat perception thermodynamic diagram is included). And under the condition that the sparsity of the thermodynamic region of the first threat perception thermodynamic diagram is less than the sparsity of the first target, and the sparsity of the thermodynamic region of the second threat perception thermodynamic diagram is less than the sparsity of the second target, determining perception capability evaluation information of target perception data in a first security protection upgrading stage as third perception capability evaluation information (namely, an associated threat perception thermodynamic diagram which is beyond the first threat perception thermodynamic diagram and the second threat perception thermodynamic diagram and can refer to predicted possibly related threat perception thermodynamic diagrams).
In an embodiment that can be implemented independently, if the perception capability assessment information is third perception capability assessment information, N threat perception intelligence sets corresponding to the third perception capability assessment information and an intelligence tag cluster corresponding to each threat perception intelligence set can be obtained, each threat perception intelligence set includes M different key threat perception intelligence, and N and M are positive integers greater than or equal to 1. Then, determining current frequent intelligence labels corresponding to the threat perception intelligence sets in intelligence label clusters corresponding to the threat perception intelligence sets, extracting the intelligence label characteristics by adopting the current frequent intelligence labels corresponding to the threat perception intelligence sets to obtain the intelligence label characteristics of each key threat perception intelligence in the threat perception intelligence sets, conducting extension on the current frequent intelligence labels corresponding to the threat perception intelligence sets based on the intelligence label characteristics of each key threat perception intelligence in the N kinds of threat perception intelligence sets to obtain real-time extension intelligence labels corresponding to the threat perception intelligence sets, and adding the real-time extension intelligence labels corresponding to the threat perception intelligence sets into the intelligence label clusters corresponding to the threat perception intelligence sets.
Therefore, the steps are returned and executed to determine the current frequent information labels corresponding to the threat perception information sets in the information label clusters corresponding to the threat perception information sets until the global perception heat power corresponding to the N threat perception information sets is larger than the set perception heat power, and the update information of the threat perception information intervals corresponding to the N threat perception information sets is obtained according to the global perception heat power.
Wherein, the current frequent intelligence labels corresponding to the threat perception intelligence sets are determined in the intelligence label clusters corresponding to the threat perception intelligence sets, which can be specifically: determining an associated frequent information label corresponding to the threat perception information set, current threat perception information interval information and current threat perception information interval information corresponding to the target threat perception information set, and comparing the current threat perception information interval information corresponding to the threat perception information set with the current threat perception information interval information corresponding to the target threat perception information set to obtain first coverage range information of the current threat perception information interval information corresponding to the threat perception information set, wherein the target threat perception information set is all threat perception information sets including the threat perception information sets in the N kinds of threat perception information sets. And then, comparing the current threat perception information interval information corresponding to the threat perception information set with the associated frequent information labels corresponding to the threat perception information set to obtain second coverage range information of the current threat perception information interval information of the threat perception information set, and determining the associated frequent information labels corresponding to the threat perception information set or the current threat perception information interval information corresponding to the threat perception information set as the information labels corresponding to the current time sequence node of the threat perception information set based on the second coverage range information and the first coverage range information.
Fig. 3 is a schematic diagram illustrating a hardware structure of an AI system 100 for implementing the above-mentioned data processing method based on threat awareness big data and artificial intelligence according to an embodiment of the present application, where, as shown in fig. 3, the AI system 100 may include a processing chip 110 and a machine-readable storage medium 120; wherein the machine-readable storage medium 120 has stored thereon executable code, which when executed by the processing chip 110, causes the processing chip 110 to perform the steps of the above embodiments of the data processing method based on threat-aware big data and artificial intelligence.
Actually, the AI system 100 may further include a communication interface 140, and the processing chip 110, the machine-readable storage medium 120, and the communication interface 140 are connected through the bus 130, and the communication interface 140 is used for communication with other devices.
Additionally, the present application provides a non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform at least the steps of an embodiment of a threat awareness big data and artificial intelligence based data processing method as described above.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (10)

1. A data processing method based on threat awareness big data and artificial intelligence is applied to an AI system, the AI system carries out data interaction with a plurality of threat awareness cloud protection systems, and the method comprises the following steps:
acquiring basic threat perception data of the threat perception cloud protection system, and extracting threat perception situation of the basic threat perception data to obtain a basic threat perception situation corresponding to the basic threat perception data;
acquiring first basic derived threat perception data corresponding to the basic threat perception data and second basic derived threat perception data corresponding to a basic threat perception situation, and acquiring data derived services of the threat perception cloud protection system, wherein a plurality of first past threat perception data and derived databases corresponding to the first past threat perception data are configured in the data derived services, and the derived databases comprise first derived threat perception data corresponding to the first past threat perception data;
matching the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data with first past threat perception data and first derived threat perception data in the data derived service respectively to obtain the first past threat perception data and the first derived threat perception data which are matched with the basic threat perception data, the basic threat perception situation, the first basic derived threat perception data and the second basic derived threat perception data;
and performing data integration on the basic threat perception data according to the first past threat perception data and the first derived threat perception data obtained by matching.
2. The threat awareness big data and artificial intelligence based data processing method according to claim 1, wherein the step of obtaining data derivative services of the threat awareness cloud protection system comprises:
obtaining past scheduling logs in a target scheduling container of the threat awareness cloud protection system, wherein the past scheduling logs comprise past scheduling threat awareness data and past scheduling contents of the target scheduling container to the past scheduling threat awareness data;
acquiring a past threat perception situation corresponding to the past scheduling threat perception data;
according to past scheduling contents corresponding to the past scheduling threat perception data, counting derived threat perception data corresponding to the past scheduling threat perception data and the past threat perception situation respectively, wherein the derived threat perception data are configured in the data derived service, and the derived threat perception data corresponding to the past threat perception situation and the past scheduling threat perception data respectively are configured in the data derived service.
3. The threat awareness big data and artificial intelligence based data processing method of claim 2, wherein the past scheduling content comprises: the step of counting the past scheduling threat perception data and the past threat perception situation and extracting corresponding derivative threat perception data according to the past scheduling content corresponding to the past scheduling threat perception data includes:
when the scheduling path information and the scheduling frequency information of any one piece of first past scheduling information meet a target condition and the any one piece of first past scheduling information contains derivative threat perception data corresponding to the past scheduling threat perception data, determining the derivative threat perception data corresponding to the past scheduling threat perception data according to the any one piece of first past scheduling information;
and if the scheduling path information and the scheduling frequency information of any one second past scheduling information meet the target condition and the any one second past scheduling information contains the derived threat perception data corresponding to the past threat perception situation, determining the derived threat perception data corresponding to the past threat perception situation according to the any one second past scheduling information.
4. The data processing method based on threat awareness big data and artificial intelligence according to claim 2, wherein the derivative database further includes scheduling frequency information and scheduling path information corresponding to the first past threat awareness data, respectively, and the step of obtaining the data derivative service of the threat awareness cloud protection system further includes:
determining scheduling frequency information and scheduling path information respectively corresponding to the past threat awareness situation and the past scheduling threat awareness data, and then the derived database of the first past threat awareness data further includes scheduling frequency information and scheduling path information respectively corresponding to the past scheduling threat awareness data.
5. The data processing method based on threat awareness big data and artificial intelligence according to claim 4, wherein the step of matching the basic threat awareness data, the obtained basic threat awareness situation, the first basic derived threat awareness data, and the second basic derived threat awareness data with first past threat awareness data and first derived threat awareness data in the data derived service, respectively, to obtain the first past threat awareness data and first derived threat awareness data that match the basic threat awareness data, the basic threat awareness situation, and the first basic derived threat awareness data and the second basic derived threat awareness data, comprises:
matching the basic threat perception data, the basic threat perception situation and the basic derived threat perception data with each first past threat perception data and the first derived threat perception data in data derived services respectively to obtain matched first past threat perception data and first derived threat perception data;
and selecting first past threat perception data and first derivative threat perception data of which scheduling path information and scheduling frequency information meet target conditions from the matched first past threat perception data and first derivative threat perception data.
6. The threat awareness big data and artificial intelligence based data processing method according to any one of claims 1 to 5, wherein the method further comprises:
acquiring a threat perception data cluster, wherein the threat perception data cluster comprises a plurality of second past threat perception data and information whether integration and labeling are carried out between any two second past threat perception data based on threat perception categories;
matching the basic threat perception data and the basic threat perception situation thereof with second past threat perception data in the threat perception data cluster respectively to obtain second past threat perception data associated with the basic threat perception data and the basic threat perception situation;
and integrating relevant threat perception categories of the basic threat perception data according to the second past threat perception data obtained by matching.
7. The threat awareness big data and artificial intelligence based data processing method according to claim 6, wherein the step of obtaining the threat awareness data cluster comprises:
obtaining past scheduling logs in a target scheduling container, wherein the past scheduling logs comprise past scheduling threat perception data;
acquiring a past threat perception situation corresponding to the past scheduling threat perception data, and acquiring the past scheduling threat perception data and the past threat perception situation to extract derivative databases respectively corresponding to the past scheduling threat perception data and the past threat perception situation;
extracting corresponding derivative databases according to the past scheduling threat perception data and the past threat perception situation to extract the past scheduling threat perception data and the past threat perception situation for threat perception category clustering to obtain second past threat perception data of a plurality of threat perception categories;
and integrating any two second past threat perception data in the same threat perception category to obtain the threat perception data cluster.
8. The data processing method based on threat awareness big data and artificial intelligence according to claim 7, wherein after obtaining second past threat awareness data of a plurality of threat awareness categories, further comprising:
determining a threat awareness intersection between second past threat awareness data of any two threat awareness classes of the plurality of threat awareness classes;
and integrating second past threat perception data of any two threat perception categories with the threat perception cross degree meeting the target condition.
9. The threat awareness big data and artificial intelligence based data processing method according to any one of claims 1 to 8, wherein the method further comprises:
acquiring the basic threat perception data and subordinate perception data which is subjected to data integration with the basic threat perception data, and summarizing the basic threat perception data and the subordinate perception data into target perception data;
obtaining a key perception data group corresponding to each security protection simulation strategy of the target perception data in a first security protection upgrading stage, wherein the first security protection upgrading stage comprises at least two security protection simulation strategies, and the key perception data group corresponding to each security protection simulation strategy comprises threat attack perception data of a target threat attack channel sensed by threat perception microservices in the target perception data in the corresponding security protection simulation strategies;
determining a sensing data node network between key sensing data clusters corresponding to each safety protection simulation strategy in the first safety protection upgrading stage;
determining a threat perception thermodynamic diagram of the target perception data in the first security protection upgrading stage according to perception data node networks among key perception data clusters corresponding to each security protection simulation strategy in the first security protection upgrading stage;
determining perception capability evaluation information of the target perception data in the first security protection upgrading stage according to the threat perception thermodynamic diagram;
the step of obtaining the key perception data clusters corresponding to the security protection simulation strategies of the target perception data in the first security protection upgrading stage includes:
acquiring threat attack sensing data of a target threat attack channel captured in a set data area after a first security protection simulation strategy is started by a threat sensing micro-service in the target sensing data, and determining a key sensing data group corresponding to the first security protection simulation strategy according to the threat attack sensing data of the target threat attack channel captured in the set data area after the first security protection simulation strategy is started by the threat sensing micro-service in the target sensing data, wherein the first security protection simulation strategy is any one security protection simulation strategy in a first security protection upgrading stage;
under the condition that a threat perception microservice in the target perception data sets a data area without capturing a target threat attack channel after a second security protection simulation strategy is started, determining a key perception data group corresponding to the second security protection simulation strategy according to threat attack perception data of the target threat attack channel received by the threat perception microservice in the target perception data, wherein the second security protection simulation strategy is any one security protection simulation strategy except the first security protection simulation strategy in the first security protection upgrading stage;
wherein the method further comprises:
the threat-aware microservices in the target-aware data do not capture a target threat attack channel in a set data area after the third security protection simulation strategy is enabled, and the key perception data cliques corresponding to the security protection simulation strategies of the first target quantity which are continuous before the third security protection simulation strategy are all determined according to the threat attack perception data of the target threat attack channel received by the threat perception micro-service, a target threat attack channel capturing request is sent to the threat perception micro-service, such that the threat-aware microservice captures a target threat attack channel in response to the target threat attack channel capture request, the third security protection simulation strategy is any one of the first security protection simulation strategy and the second security protection simulation strategy in the first security protection upgrading stage;
and obtaining threat attack perception data of the target threat attack channel captured by the threat perception micro-service responding to the target threat attack channel capturing request, and determining a key perception data group corresponding to the third security protection simulation strategy according to the threat attack perception data of the target threat attack channel captured by the threat perception micro-service responding to the target threat attack channel capturing request.
10. An AI system, comprising:
a machine readable storage medium for storing a computer program;
a processing chip for executing the computer program to perform the data processing method based on threat-aware big data and artificial intelligence of any one of claims 1 to 9.
CN202110770377.5A 2021-07-08 2021-07-08 Data processing method and AI system based on threat perception big data and artificial intelligence Withdrawn CN113434869A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110770377.5A CN113434869A (en) 2021-07-08 2021-07-08 Data processing method and AI system based on threat perception big data and artificial intelligence

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110770377.5A CN113434869A (en) 2021-07-08 2021-07-08 Data processing method and AI system based on threat perception big data and artificial intelligence

Publications (1)

Publication Number Publication Date
CN113434869A true CN113434869A (en) 2021-09-24

Family

ID=77759444

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110770377.5A Withdrawn CN113434869A (en) 2021-07-08 2021-07-08 Data processing method and AI system based on threat perception big data and artificial intelligence

Country Status (1)

Country Link
CN (1) CN113434869A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114697128A (en) * 2022-04-13 2022-07-01 石家庄汇勤网络科技有限公司 Big data denoising method and big data acquisition system through artificial intelligence decision

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114697128A (en) * 2022-04-13 2022-07-01 石家庄汇勤网络科技有限公司 Big data denoising method and big data acquisition system through artificial intelligence decision
CN114697128B (en) * 2022-04-13 2023-10-24 广东数字金服信息技术有限公司 Big data denoising method and big data acquisition system through artificial intelligence decision

Similar Documents

Publication Publication Date Title
CN111177714A (en) Abnormal behavior detection method and device, computer equipment and storage medium
CN109241223B (en) Behavior track identification method and system
CN112308001A (en) Data analysis method and personnel tracking method and system for smart community
CN110198303A (en) Threaten the generation method and device, storage medium, electronic device of information
CN110969215A (en) Clustering method and device, storage medium and electronic device
CN109063984B (en) Method, apparatus, computer device and storage medium for risky travelers
CN112463859B (en) User data processing method and server based on big data and business analysis
CN109784220B (en) Method and device for determining passerby track
CN106294219A (en) A kind of equipment identification, data processing method, Apparatus and system
CN113486983A (en) Big data office information analysis method and system for anti-fraud processing
CN112437034B (en) False terminal detection method and device, storage medium and electronic device
CN113434869A (en) Data processing method and AI system based on threat perception big data and artificial intelligence
CN111046705A (en) Image recognition method, device and system and computing equipment
CN112925805A (en) Big data intelligent analysis application method based on network security
CN113434868A (en) Information generation method based on threat perception big data and artificial intelligence perception system
CN111368128A (en) Target picture identification method and device and computer readable storage medium
CN112292671A (en) Device recognition apparatus and device recognition method
CN113343004B (en) Object identification method and device, storage medium and electronic device
CN115439928A (en) Operation behavior identification method and device
CN114356712A (en) Data processing method, device, equipment, readable storage medium and program product
CN113936157A (en) Abnormal information processing method and device, storage medium and electronic device
CN113254672A (en) Abnormal account identification method, system, equipment and readable storage medium
CN113098884A (en) Network security monitoring method based on big data, cloud platform system and medium
CN113724059A (en) Federal learning model training method and device and electronic equipment
CN111475380A (en) Log analysis method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20210924

WW01 Invention patent application withdrawn after publication