CN111181982B - Abnormal data identification method and device, computing equipment and medium - Google Patents
Abnormal data identification method and device, computing equipment and medium Download PDFInfo
- Publication number
- CN111181982B CN111181982B CN201911424692.1A CN201911424692A CN111181982B CN 111181982 B CN111181982 B CN 111181982B CN 201911424692 A CN201911424692 A CN 201911424692A CN 111181982 B CN111181982 B CN 111181982B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- period information
- determining
- interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
- G06F21/562—Static detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/145—Countermeasures against malicious traffic the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Virology (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present disclosure provides a method for identifying abnormal data, including: acquiring target data, wherein the target data comprises data generated when data interaction is carried out between first equipment and second equipment; determining whether the target data comprises target period information, wherein the target period information is used for representing that the first equipment and the second equipment perform periodic abnormal data interaction; and in response to determining that the target data includes the target period information, determining that the data interaction between the first device and the second device is an abnormal interaction. The disclosure also provides an abnormal data identification device, a computing device, a medium and a computer program product.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to an abnormal data identification method, an abnormal data identification device, a computing device, and a computer-readable storage medium.
Background
Most of the current common Trojan horse detection software identifies Trojan horse programs based on a program feature code detection technology. The security analyst extracts their sample features by collecting malicious samples and analyzing their host behavior. And the detection and protection of the Trojan horse by the detection software are realized by upgrading the feature library. Due to the faster Trojan horse variation, the transmission and attack modes are more diversified. Particularly, the remote control trojan horse is particularly prominent, and the efficiency of discovering malicious program samples and analyzing the host behaviors is reduced due to stronger concealment and persistence.
Therefore, how to quickly acquire malicious data from massive data to analyze the malicious data becomes an urgent problem to be solved.
Disclosure of Invention
In view of the above, the present disclosure provides an optimized abnormal data identification method, an abnormal data identification device, a computing device, and a computer-readable storage medium.
One aspect of the present disclosure provides a method for identifying abnormal data, including: the method comprises the steps of obtaining target data, wherein the target data comprise data generated when a first device and a second device perform data interaction, determining whether the target data comprise target period information or not, wherein the target period information is used for representing that the first device and the second device perform periodic abnormal data interaction, and determining that the data interaction between the first device and the second device is abnormal interaction in response to the fact that the target data comprise the target period information.
According to an embodiment of the present disclosure, the method further includes: in response to determining that data interaction between the first device and the second device is abnormal interaction, respectively determining address data of the first device and address data of the second device, and acquiring data to be identified, wherein the data to be identified comprises data generated when a plurality of devices perform data interaction, the plurality of devices at least comprise the first device and the second device, and determining abnormal data in the data to be identified based on the address data of the first device and the address data of the second device, wherein the abnormal data comprises interaction data, related to the first device and the second device, in the data to be identified.
According to an embodiment of the present disclosure, the determining whether the target data includes the target period information includes: processing the target data to obtain a plurality of initial period information in the target data, determining a weight of each initial period information in the plurality of initial period information, obtaining a plurality of weights corresponding to the plurality of initial period information one to one, determining whether at least one weight meeting a preset condition is included in the plurality of weights, and determining that the target period information is included in the target data in response to determining that at least one weight meeting the preset condition is included in the plurality of weights, wherein at least one initial period information corresponding to the at least one weight is the target period information.
According to an embodiment of the present disclosure, the target data includes time domain data. Wherein the processing the target data to obtain a plurality of initial period information in the target data includes: and processing the target data by utilizing a Fourier transform algorithm to obtain frequency domain data, and determining the initial period information in the target data based on the frequency domain data.
According to an embodiment of the present disclosure, the determining the weight of each of the plurality of initial period information includes: determining a plurality of period curves corresponding to the plurality of initial period information, respectively determining the matching degree of the target data and the plurality of period curves, wherein the target data comprises a plurality of discrete data, the matching degree represents the distance between the plurality of discrete data and the period curves, and the weight of each initial period information in the plurality of initial period information is determined based on the matching degree.
According to an embodiment of the present disclosure, the processing the target data to obtain a plurality of pieces of initial period information in the target data includes: determining time information in the target data, wherein the time information includes a time range of the target data generated when the first device and the second device perform data interaction, determining time granularity based on the time information, performing compression processing on the target data based on the time granularity, and obtaining the plurality of initial period information based on the compressed target data.
According to an embodiment of the present disclosure, the first device and the second device are regarded as a set of devices, and the address data includes address data of multiple sets of devices. Wherein the determining abnormal data in the data to be identified based on the address data comprises: the method comprises the steps of processing address data of multiple groups of equipment respectively to obtain multiple groups of characteristic values corresponding to the address data of the multiple groups of equipment, processing the multiple groups of characteristic values to obtain target characteristic values, processing multiple pieces of address data to be identified in the data to be identified to obtain multiple pieces of characteristic values to be identified, filtering the multiple pieces of characteristic values to be identified by using the target characteristic values to obtain at least one characteristic value to be identified matched with the target characteristic values, and determining data corresponding to the at least one characteristic value to be identified in the data to be identified as abnormal data.
According to an embodiment of the present disclosure, the method further includes: the method comprises the steps of obtaining initial data and obtaining white list data, wherein the white list data comprise address data of a plurality of safety devices, filtering the initial data based on the white list data to obtain target data, and enabling the target data not to comprise interactive data of the plurality of safety devices.
Another aspect of the present disclosure provides an apparatus for identifying abnormal data, including: the device comprises a first obtaining module, a first determining module and a second determining module. The first acquisition module acquires target data, wherein the target data comprises data generated when the first equipment and the second equipment perform data interaction. The first determining module is used for determining whether the target data comprises target period information, wherein the target period information is used for representing that the first equipment and the second equipment perform periodic abnormal data interaction. And the second determining module is used for determining that the data interaction between the first equipment and the second equipment is abnormal interaction in response to the fact that the target data comprises the target period information.
According to the embodiment of the present disclosure, the apparatus further includes: the device comprises a third determining module, a fourth obtaining module and a fourth determining module. The third determining module is used for respectively determining the address data of the first device and the address data of the second device in response to the fact that the data interaction between the first device and the second device is abnormal. The fourth acquisition module acquires data to be identified, wherein the data to be identified comprises data generated when a plurality of devices perform data interaction, and the plurality of devices at least comprise the first device and the second device. The fourth determining module is used for determining abnormal data in the data to be identified based on the address data of the first device and the address data of the second device, wherein the abnormal data comprises interaction data between the first device and the second device in the data to be identified.
According to an embodiment of the present disclosure, the determining whether the target data includes the target period information includes: processing the target data to obtain a plurality of initial period information in the target data, determining a weight of each initial period information in the plurality of initial period information, obtaining a plurality of weights corresponding to the plurality of initial period information one to one, determining whether at least one weight meeting a preset condition is included in the plurality of weights, and determining that the target period information is included in the target data in response to determining that at least one weight meeting the preset condition is included in the plurality of weights, wherein at least one initial period information corresponding to the at least one weight is the target period information.
According to an embodiment of the present disclosure, the target data includes time domain data. Wherein the processing the target data to obtain a plurality of initial period information in the target data includes: and processing the target data by utilizing a Fourier transform algorithm to obtain frequency domain data, and determining the initial period information in the target data based on the frequency domain data.
According to an embodiment of the present disclosure, the determining the weight of each of the plurality of initial period information includes: determining a plurality of period curves corresponding to the plurality of initial period information, respectively determining the matching degree of the target data and the plurality of period curves, wherein the target data comprises a plurality of discrete data, the matching degree represents the distance between the plurality of discrete data and the period curves, and the weight of each initial period information in the plurality of initial period information is determined based on the matching degree.
According to an embodiment of the present disclosure, the processing the target data to obtain a plurality of pieces of initial period information in the target data includes: determining time information in the target data, wherein the time information includes a time range of the target data generated when the first device and the second device perform data interaction, determining time granularity based on the time information, performing compression processing on the target data based on the time granularity, and obtaining the plurality of initial period information based on the compressed target data.
According to an embodiment of the present disclosure, the first device and the second device are regarded as a set of devices, and the address data includes address data of multiple sets of devices. Wherein the determining abnormal data in the data to be identified based on the address data comprises: the method comprises the steps of processing address data of multiple groups of equipment respectively to obtain multiple groups of characteristic values corresponding to the address data of the multiple groups of equipment, processing the multiple groups of characteristic values to obtain target characteristic values, processing multiple pieces of address data to be identified in the data to be identified to obtain multiple pieces of characteristic values to be identified, filtering the multiple pieces of characteristic values to be identified by using the target characteristic values to obtain at least one characteristic value to be identified matched with the target characteristic values, and determining data corresponding to the at least one characteristic value to be identified in the data to be identified as abnormal data.
According to the embodiment of the present disclosure, the apparatus further includes: the device comprises a second acquisition module, a third acquisition module and a filtering module. And the second acquisition module acquires the initial data. And the third acquisition module acquires white list data, wherein the white list data comprises address data of a plurality of safety devices. And the filtering module is used for filtering the initial data based on the white list data to obtain the target data, so that the target data does not comprise the interactive data of the plurality of safety devices.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.
Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.
Another aspect of the disclosure provides a computer program product comprising computer readable instructions, wherein the computer readable instructions, when executed, are for implementing the method as described above.
According to the embodiments of the present disclosure, the problem of low Trojan horse detection efficiency in the related art may be at least partially solved, and thus a technical effect of improving the detection efficiency of danger information may be achieved.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
fig. 1 schematically illustrates an application scenario of an identification method of abnormal data and an identification apparatus of abnormal data according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a method of identification of anomalous data in accordance with an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart of a method of identifying anomalous data in accordance with another embodiment of the present disclosure;
FIG. 4 schematically illustrates a schematic diagram of a periodic curve according to an embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of an apparatus for identification of anomalous data in accordance with an embodiment of the present disclosure;
FIG. 6 schematically shows a block diagram of an apparatus for identification of anomalous data in accordance with another embodiment of the present disclosure; and
FIG. 7 schematically illustrates a block diagram of a computer system adapted for identification of anomalous data in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides an identification method of abnormal data, which comprises the following steps: target data are obtained, wherein the target data comprise data generated when the first equipment and the second equipment perform data interaction. And then, determining whether the target data comprises target period information or not, wherein the target period information is used for representing that the first equipment and the second equipment perform periodic abnormal data interaction, and determining that the data interaction between the first equipment and the second equipment is abnormal interaction in response to the fact that the target period information is included in the target data.
Fig. 1 schematically shows a due scenario of an abnormal data identification method and an abnormal data identification device according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a proper scenario in which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the application scenario 100 includes, for example, a first device 110, a second device 120, and a firewall device 130.
According to an embodiment of the present disclosure, the first device 110 may be, for example, a device in an intranet, and the second device 120 may be, for example, a device in an extranet. In order to ensure the security of data interaction between the intranet device and the extranet device, data monitoring is usually performed through the firewall device 130.
For example, if the first device 110 needs to access the extranet device, the firewall device 130 may detect whether there is a danger in the extranet device to be accessed, and if there is a danger, the access of the first device 110 is prohibited, otherwise, the access of the first device 110 is permitted. If the second device 110 needs to access a device in the intranet, the firewall device 130 may detect whether there is a danger in the second device 120, and if there is a danger, prohibit the second device 120 from accessing the device in the intranet, otherwise, allow the second device 120 to access the device in the intranet.
In the embodiment of the present disclosure, the firewall device 130 stores, for example, data generated during data interaction between the first device 110 and the second device 120. Therefore, the embodiment of the present disclosure may obtain data in the firewall device 130, and analyze the obtained data to obtain abnormal data in the data interaction process between the first device 110 and the second device 120, so as to perform subsequent security analysis based on the abnormal data, for example, for Trojan horse virus analysis.
The method for identifying abnormal data according to the embodiment of the present disclosure is described below with reference to fig. 1 and 2.
Fig. 2 schematically shows a flow chart of an identification method of abnormal data according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S210 to S230.
In operation S210, target data is acquired, where the target data includes data generated when the first device performs data interaction with the second device.
According to an embodiment of the present disclosure, the target data may be, for example, data in a firewall device. The target data may be, for example, traffic data generated when data interaction is performed between the first device and the second device, where the traffic data includes, for example, a traffic log and a service log, and the traffic data includes, for example, detailed information of a data interaction process.
According to the embodiment of the disclosure, the traffic log and the service log can be unified and standardized, and the processed traffic log and the service log are uniformly stored in a JSON format, for example. For example, a plurality of fields in each traffic log or service log are obtained and standardized, and the plurality of fields include, for example, a device unique identifier SN, a Session identifier, address information, a Host field in a URL, log time, and the like. Wherein, a plurality of traffic logs or service logs may be from different firewall devices, so the device unique identifier SN may be, for example, the device identifier of the firewall device from which the traffic log or service log comes. The address information may be, for example, five-tuple data including, for example, addresses of the first device and the second device. The log time may include, for example, a timestamp for each log, which may, for example, characterize the time at which the firewall device stored the traffic log or the traffic log.
In operation S220, it is determined whether target period information is included in the target data, where the target period information is used to characterize the first device performing periodic abnormal data interaction with the second device.
According to the embodiment of the disclosure, if the target period information is included in the target data, the interaction process of the first device and the second device may be characterized as periodic interaction. For example, the first device periodically interacts with the second device for data. For example, the second device accesses the first device at 00:00, 06:00, 12:00, 18:00 of each day, where the target period information included in the access time may be, for example, a period of 6 hours. Or the second device accesses the first device every N days or every M months, wherein the target period information may be, for example, N days or M months. If the target data comprises the period information, the data interaction between the first device and the second device is abnormal data interaction, for example, the second device accesses the first device regularly, so that the Trojan horse virus is injected into the first device, and the purpose of attacking the first device is achieved.
In operation S230, in response to determining that the target period information is included in the target data, it is determined that the data interaction between the first device and the second device is an abnormal interaction.
The disclosed embodiment further comprises: and respectively determining the address data of the first device and the address data of the second device in response to the fact that the data interaction between the first device and the second device is abnormal. Then, data to be identified is obtained, the data to be identified comprises data generated when a plurality of devices perform data interaction, and the plurality of devices at least comprise a first device and a second device. And then, determining abnormal data in the data to be identified based on the address data of the first device and the address data of the second device, wherein the abnormal data comprises the interaction data between the first device and the second device in the data to be identified.
According to the embodiment of the disclosure, when it is determined that the target data has the target period information, it may be determined that abnormal data interaction is performed between the first device and the second device, and then the first device and the second device may be used as subsequent key monitoring objects. Then, the address data of the first device and the address data of the second device may be obtained, facilitating to subsequently find the interaction data between the first device and the second device from the mass data based on the address data.
According to the embodiment of the disclosure, the data to be identified may be, for example, data obtained from a plurality of firewall devices, and the data to be identified may include, in addition to the interaction data between the first device and the second device, interaction data between other devices, or may include interaction data between the first device and other devices and interaction data between the second device and other devices.
According to the embodiment of the disclosure, if periodic information exists between the first device and the second device, the interaction between the first device and the second device can be characterized as abnormal interaction. Because the periodic information can reflect abnormal interaction more directly, the embodiment of the disclosure determines deep risks implied during device interaction by acquiring the periodic information, and improves the effect of risk detection.
According to the embodiment of the disclosure, the interaction between the first device and the second device is an abnormal interaction. Therefore, it is necessary to obtain abnormal data between the first device and the second device from the mass data to be identified, so as to perform subsequent security analysis based on the abnormal data. According to the embodiment of the disclosure, after the abnormal interaction between the first device and the second device is determined from the target data, the abnormal data between the first device and the second device is further obtained from the mass data to be identified, so that the computing resources are greatly reduced. Namely, whether abnormal interaction equipment exists in massive data to be identified is not required to be directly analyzed, the abnormal interaction equipment is determined from target data with small data quantity, and then the abnormal data of the abnormal interaction equipment is further acquired from the massive data to be identified based on the determined abnormal interaction equipment, so that the calculation amount is greatly reduced.
Fig. 3 schematically shows a flow chart of an identification method of abnormal data according to another embodiment of the present disclosure.
As shown in fig. 3, the method includes, for example, operations S210 to S230 and operations S310 to S330. Operations S210 to S230 are the same as or similar to the operations in fig. 2, and are not described herein again.
In operation S310, initial data is acquired. The initial data includes, for example, target data. Since the data size of the initial data is too large, the initial data needs to be preprocessed to obtain the target data. For example, a part of normal data in the initial data is filtered first, so that the data volume of the remaining target data is reduced, which is convenient for reducing the calculation amount in the process of processing the target data to obtain the target period information.
In operation S320, white list data is acquired, wherein the white list data includes address data of a plurality of security devices. According to the embodiment of the disclosure, the security device may be, for example, a security device known from historical experience, and address data of the security device is stored in the white list data, so that normal data in the initial data is filtered based on the white list data.
In operation S330, the initial data is filtered based on the white list data to obtain target data, such that the target data does not include interaction data of the plurality of security devices.
According to the embodiment of the disclosure, because the interactive data of the plurality of safety devices are normal data, the data volume of the target data obtained by filtering the normal data from the initial data is greatly reduced, the calculation amount in the process of obtaining the target period information by subsequently processing the target data is reduced, and the processing efficiency is improved.
According to the embodiment of the present disclosure, the above-mentioned operation S220 of determining whether the target period information is included in the target data includes the following steps (1) to (4), for example.
(1) And processing the target data to obtain a plurality of pieces of initial period information in the target data.
For example, time information in the target data is first determined, where the time information includes, for example, a time range in which the target data is generated when the first device performs data interaction with the second device. For example, the time range may include 1 day, 7 days, 30 days, and so forth. For ease of understanding, the time range is exemplified by 1 day, for example. For example, the target data is data within the past 1 day.
Then, a temporal granularity is determined based on the temporal information. For example, different time information corresponds to different time granularities. For example, a time range of 1 day corresponds to a time granularity of 5 seconds, 7 days corresponds to a time granularity of 60 seconds, 30 days corresponds to a time granularity of 300 seconds, and so forth.
Thereafter, the target data may be compressed based on the time granularity. That is, multiple pieces of data with the same data interaction objects in the time granularity are compressed into one piece of data, so that the calculation amount of the target period information determined subsequently is reduced.
For example, time period 00, such as 5 seconds: 01-00: the target data in 05 includes 4 logs. The 4 logs are respectively: log 1 of interaction of the first device with the second device, log 2 of interaction of the second device with the third device, log 3 of interaction of the first device with the second device, and log 4 of interaction of the fourth device with the fifth device. At this time, the log 1 and the log 3 may be compressed into the log 1', and the log 2 and the log 4 may not be compressed. The finally obtained processed target data includes, for example, log 1 ', log 2, and log 4, and the timestamps of log 1', log 2, and log 4 are all initial time 00:01 in the time period.
Further, a plurality of pieces of initial cycle information may be obtained based on the target data after the compression processing. For example, a plurality of compressed target data are obtained within 1 day of the time range, and the target data for the interactive devices, which are the first device and the second device, are processed to obtain a plurality of initial periods for the interaction between the first device and the second device. It can be understood that, for different devices of the two interacting parties, the target data of the different devices are processed respectively to obtain a plurality of initial period information of the different interacting devices. For example, the first device and the second device correspond to a plurality of pieces of initial cycle information, and the third device and the fourth device also correspond to a plurality of pieces of initial cycle information. For convenience of understanding, the first device and the second device correspond to a plurality of pieces of initial period information, which are exemplified in the embodiments of the present disclosure.
According to the embodiment of the disclosure, the processed target data of the interaction between the first device and the second device is, for example, time domain data. The time domain data is, for example, sequence data, for example, logs corresponding to the target data at a plurality of time instants, for example, at least one log at each of 06:00, 08:00, 12:00, 16:00, 18:00, 24:00, and the like.
According to the embodiment of the present disclosure, the target data may be processed by using a fourier transform algorithm, for example, to obtain frequency domain data. In other words, the target data in the time domain, whose period information in the frequency domain is significant, is converted into the frequency domain, and thus a plurality of initial period information in the target data can be determined based on the frequency domain data. The obtained plurality of initial period information include, for example, a period of 6 hours (a log at time 06:00, 12:00, 18:00, 24: 00), a period of 8 hours (a log at time 08:00, 16:00, 24: 00), and the like.
(2) Determining the weight of each initial period information in the plurality of initial period information to obtain a plurality of weights corresponding to the plurality of initial period information one by one.
FIG. 4 schematically shows a schematic of a periodic curve according to an embodiment of the disclosure.
As shown in fig. 4, for example, a plurality of cycle curves corresponding to a plurality of initial cycle information are determined. And respectively determining the matching degree of the target data and the plurality of periodic curves, wherein the target data comprises a plurality of discrete data, and the matching degree represents the distance between the plurality of discrete data and the periodic curves. Then, based on the degree of matching, a weight of each of the plurality of initial period information is determined.
For example, FIG. 4 illustrates a cycle of 6 hours. A plurality of discrete target data are respectively distributed on or beside the period curve corresponding to the period of 6 hours. For example, if 1000 discrete data are included, and the distance between 800 discrete data and the periodic curve is smaller than the preset distance, it may indicate that the 800 discrete data are matched with the periodic curve, that is, the weight is, for example, 0.8, and the weight is, for example, the ratio of the number of matched discrete data to the total number of discrete data.
Similarly, a period of 8 hours, for example, corresponds to a weight of 0.4, for example.
(3) It is determined whether at least one weight satisfying a preset condition is included in the plurality of weights. For example, the weight satisfying the preset condition is, for example, a weight value greater than a preset weight value, and the preset weight value may be, for example, 0.5. Therefore, the weight 0.8 corresponding to a period of 6 hours satisfies the preset condition.
(4) And determining that the target period information is included in the target data in response to determining that at least one weight satisfying a preset condition is included in the plurality of weights. Wherein, at least one initial period information corresponding to at least one weight is the target period information. That is, a cycle corresponding to a weight of 0.8 may be 6 hours as the target cycle information. Therefore, the target period information is contained in the target data, and the interaction between the first device and the second device can be characterized as abnormal interaction.
Since the period information of the target data is more obvious in the frequency domain, the embodiment of the disclosure converts the target data in the time domain into the data in the frequency domain through the fourier transform algorithm, thereby improving the accuracy of the acquired period information. In addition, the target period information is determined by determining the matching degree of the target data and the plurality of period curves, the period information with high matching degree is determined as the target period information, the determined target period information is more accurate, and the identification accuracy degree of abnormal interaction is improved.
According to the embodiment of the disclosure, because the interaction between the first device and the second device is abnormal interaction, abnormal data related to the first device and the second device can be associated from the massive data to be identified based on the address data of the first device and the second device, and the associated abnormal data can be used for subsequent security analysis.
In one case, when the devices with abnormal interaction include multiple groups of devices, the interaction data of each group of devices is to be acquired from the massive data to be identified according to the address data of each group of devices in the multiple groups of devices, which causes the problems of excessive calculation and low efficiency. For example, if the devices of the abnormal interaction include multiple groups of devices, for example, the first group of devices includes a first device and a second device, the second group of devices includes a third device and a fourth device, and the third group of devices includes a fifth device and a sixth device. If the interactive data about the multiple groups of devices needs to be acquired from the massive data to be identified, the address data of the first group of devices, the address data of the second group of devices and the address data of the third group of devices need to be used for correlating the massive data to be identified in sequence, so that the problem of overlarge calculation amount exists.
Therefore, in order to improve the efficiency of matching interactive data of multiple sets of devices from massive data to be identified, the embodiment of the present disclosure constructs a bloom filter based on address data including the multiple sets of devices, and filters the massive data to be identified by using the constructed bloom filter to obtain required data.
For example, first, address data of a plurality of sets of devices are processed, and a plurality of sets of feature values corresponding to the address data of the plurality of sets of devices are obtained. The hash value is obtained, for example, by calculating address data for each group of devices, i.e., each group of devices has a corresponding hash value. The hash values of the multiple sets of devices are used as multiple sets of characteristic values.
And then, processing the multiple groups of characteristic values to obtain target characteristic values. That is, a plurality of sets of feature values are combined into a target feature value, which is the required bloom filter.
And then, processing a plurality of address data to be identified in the data to be identified to obtain a plurality of characteristic values to be identified, and filtering the plurality of characteristic values to be identified by using the target characteristic value to obtain at least one characteristic value to be identified matched with the target characteristic value. In other words, at least one feature value to be identified that matches the target feature value is a filtering result of the bloom filter, and the at least one feature value to be identified is, for example, a feature value corresponding to address data of a plurality of groups of devices.
Finally, data corresponding to at least one characteristic value to be identified in the data to be identified can be determined as abnormal data. The abnormal data is the abnormal data interacted by multiple groups of devices successfully associated in the massive data to be identified. Because the multiple groups of equipment are abnormal equipment obtained through periodic calculation, abnormal data obtained by correlation in massive data to be identified can be used for subsequent security analysis.
Fig. 5 schematically shows a block diagram of an apparatus for identifying anomaly data according to an embodiment of the present disclosure.
As shown in fig. 5, the abnormal data identification apparatus 500 includes, for example, a first obtaining module 510, a first determining module 520, and a second determining module 530.
The first obtaining module 510 may be configured to obtain target data, where the target data includes data generated when the first device performs data interaction with the second device. According to an embodiment of the present disclosure, the first obtaining module 510 may perform, for example, the operation S210 described above with reference to fig. 2, which is not described herein again.
The first determining module 520 may be configured to determine whether target period information is included in the target data, where the target period information is used to characterize the first device performing periodic abnormal data interaction with the second device. According to the embodiment of the present disclosure, the first determining module 520 may perform, for example, operation S220 described above with reference to fig. 2, which is not described herein again.
The second determination module 530 may be configured to determine that the data interaction between the first device and the second device is an abnormal interaction in response to determining that the target period information is included in the target data. According to an embodiment of the present disclosure, the second determining module 530 may perform, for example, the operation S230 described above with reference to fig. 2, which is not described herein again.
Fig. 6 schematically shows a block diagram of an apparatus for identifying abnormal data according to another embodiment of the present disclosure.
As shown in fig. 6, the abnormal data identification apparatus 600 includes, for example, a first obtaining module 510, a first determining module 520, a second determining module 530, a second obtaining module 610, a third obtaining module 620, and a filtering module 630. The first obtaining module 510, the first determining module 520, and the second determining module 530 are, for example, the same as or similar to the modules described above with reference to fig. 4, and are not repeated herein.
The second obtaining module 610 may be used to obtain initial data. According to the embodiment of the present disclosure, the second obtaining module 610 may, for example, perform the operation S310 described above with reference to fig. 3, which is not described herein again.
The third obtaining module 620 may be configured to obtain white list data, where the white list data includes address data of a plurality of security devices. According to the embodiment of the present disclosure, the third obtaining module 620 may, for example, perform operation S320 described above with reference to fig. 3, which is not described herein again.
The filtering module 630 may be configured to filter the initial data based on the white list data to obtain the target data such that the target data does not include interaction data of the plurality of security devices. According to the embodiment of the present disclosure, the filtering module 630 may perform, for example, the operation S330 described above with reference to fig. 3, which is not described herein again.
Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
FIG. 7 schematically illustrates a block diagram of a computer system adapted for identification of anomalous data in accordance with an embodiment of the present disclosure. The computer system illustrated in FIG. 7 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 7, a computer system 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.
In the RAM 703, various programs and data necessary for the operation of the system 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. The processor 701 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. It is noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the system 700 may also include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The system 700 may also include one or more of the following components connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program, when executed by the processor 701, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a computer-non-volatile computer-readable storage medium, which may include, for example and without limitation: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 702 and/or the RAM 703 and/or one or more memories other than the ROM 702 and the RAM 703 described above.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (10)
1. A method for identifying abnormal data comprises the following steps:
acquiring target data, wherein the target data comprises data generated when data interaction is carried out between first equipment and second equipment;
processing the target data to obtain a plurality of pieces of initial period information in the target data;
determining the weight of each initial period information in the plurality of initial period information to obtain a plurality of weights corresponding to the plurality of initial period information one by one;
determining whether at least one weight satisfying a preset condition is included in the plurality of weights;
in response to that at least one weight meeting a preset condition is included in the plurality of weights, determining that the target data includes target period information, wherein at least one initial period information corresponding to the at least one weight is the target period information, and the target period information is used for representing that the first equipment and the second equipment perform periodic abnormal data interaction; and
and in response to determining that the target data comprises target period information, determining that the data interaction between the first device and the second device is abnormal interaction.
2. The method of claim 1, further comprising:
respectively determining address data of the first device and address data of the second device in response to determining that the data interaction between the first device and the second device is abnormal interaction;
acquiring data to be identified, wherein the data to be identified comprises data generated when a plurality of devices perform data interaction, and the plurality of devices at least comprise the first device and the second device; and
determining abnormal data in the data to be identified based on the address data of the first device and the address data of the second device, wherein the abnormal data comprises interaction data between the first device and the second device in the data to be identified.
3. The method of claim 1, wherein the target data comprises time domain data;
wherein the processing the target data to obtain a plurality of initial period information in the target data includes:
processing the target data by utilizing a Fourier transform algorithm to obtain frequency domain data; and
determining the plurality of initial period information in the target data based on the frequency domain data.
4. The method of claim 1, wherein the determining a weight for each of the plurality of initial period information comprises:
determining a plurality of period curves corresponding to the plurality of initial period information;
determining the matching degree of the target data and the plurality of periodic curves respectively, wherein the target data comprises a plurality of discrete data, and the matching degree represents the distance between the plurality of discrete data and the periodic curves; and
determining a weight of each of the plurality of initial period information based on the degree of matching.
5. The method according to any one of claims 2 to 4, wherein the processing the target data to obtain a plurality of initial period information in the target data comprises:
determining time information in the target data, wherein the time information comprises a time range of the target data generated when the first device and the second device perform data interaction;
determining a temporal granularity based on the temporal information;
compressing the target data based on the time granularity; and
and obtaining the plurality of initial period information based on the target data after the compression processing.
6. The method of claim 2, wherein the first device and the second device are a set of devices, the address data comprising address data for multiple sets of devices;
wherein determining abnormal data in the data to be identified based on the address data comprises:
respectively processing the address data of the multiple groups of equipment to obtain multiple groups of characteristic values corresponding to the address data of the multiple groups of equipment;
processing the multiple groups of characteristic values to obtain target characteristic values;
processing a plurality of address data to be identified in the data to be identified to obtain a plurality of characteristic values to be identified;
filtering the plurality of characteristic values to be identified by using the target characteristic value to obtain at least one characteristic value to be identified matched with the target characteristic value; and
and determining data corresponding to the at least one characteristic value to be identified in the data to be identified as the abnormal data.
7. The method of claim 1, further comprising:
acquiring initial data;
acquiring white list data, wherein the white list data comprises address data of a plurality of safety devices; and
and filtering the initial data based on the white list data to obtain the target data, so that the target data does not comprise the interaction data of the plurality of safety devices.
8. An apparatus for identifying abnormal data, comprising:
the first acquisition module is used for acquiring target data, wherein the target data comprises data generated when the first equipment and the second equipment perform data interaction;
the first determining module is to: processing the target data to obtain a plurality of pieces of initial period information in the target data; determining the weight of each initial period information in the plurality of initial period information to obtain a plurality of weights corresponding to the plurality of initial period information one by one; determining whether at least one weight satisfying a preset condition is included in the plurality of weights; in response to that at least one weight meeting a preset condition is included in the plurality of weights, determining that the target data includes target period information, wherein at least one initial period information corresponding to the at least one weight is the target period information, and the target period information is used for representing that the first equipment and the second equipment perform periodic abnormal data interaction; and
and the second determining module is used for determining that the data interaction between the first equipment and the second equipment is abnormal interaction in response to the fact that the target data comprises the target period information.
9. A computing device, comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.
10. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911424692.1A CN111181982B (en) | 2019-12-31 | 2019-12-31 | Abnormal data identification method and device, computing equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911424692.1A CN111181982B (en) | 2019-12-31 | 2019-12-31 | Abnormal data identification method and device, computing equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111181982A CN111181982A (en) | 2020-05-19 |
| CN111181982B true CN111181982B (en) | 2022-03-25 |
Family
ID=70649145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911424692.1A Active CN111181982B (en) | 2019-12-31 | 2019-12-31 | Abnormal data identification method and device, computing equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111181982B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114666090A (en) * | 2022-02-11 | 2022-06-24 | 广州理工学院 | Fire-proof wall |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635658A (en) * | 2009-08-26 | 2010-01-27 | 中国科学院计算技术研究所 | Method and system for detecting abnormality of network secret stealing behavior |
| CN107223345A (en) * | 2014-08-22 | 2017-09-29 | 弗劳恩霍夫应用研究促进协会 | FIR filter coefficient for beamforming filter is calculated |
| CN107454109A (en) * | 2017-09-22 | 2017-12-08 | 杭州安恒信息技术有限公司 | A kind of network based on HTTP flow analyses is stolen secret information behavioral value method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8555388B1 (en) * | 2011-05-24 | 2013-10-08 | Palo Alto Networks, Inc. | Heuristic botnet detection |
| US10592372B2 (en) * | 2017-07-18 | 2020-03-17 | Vmware, Inc. | Confidence-controlled sampling methods and systems to analyze high-frequency monitoring data and event messages of a distributed computing system |
-
2019
- 2019-12-31 CN CN201911424692.1A patent/CN111181982B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635658A (en) * | 2009-08-26 | 2010-01-27 | 中国科学院计算技术研究所 | Method and system for detecting abnormality of network secret stealing behavior |
| CN107223345A (en) * | 2014-08-22 | 2017-09-29 | 弗劳恩霍夫应用研究促进协会 | FIR filter coefficient for beamforming filter is calculated |
| CN107454109A (en) * | 2017-09-22 | 2017-12-08 | 杭州安恒信息技术有限公司 | A kind of network based on HTTP flow analyses is stolen secret information behavioral value method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111181982A (en) | 2020-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110351280B (en) | Method, system, equipment and readable storage medium for extracting threat information | |
| US9208323B1 (en) | Classifier-based security for computing devices | |
| US8805995B1 (en) | Capturing data relating to a threat | |
| CN111178760B (en) | Risk monitoring method, risk monitoring device, terminal equipment and computer readable storage medium | |
| CN111131320B (en) | Asset identification method, device, system and medium | |
| KR102260417B1 (en) | Method and apparatus for detecting traffic | |
| WO2015062541A1 (en) | Cloud checking and killing method, device and system for combating anti-antivirus test | |
| EP2881877A1 (en) | Program execution device and program analysis device | |
| CN109815702B (en) | Software behavior safety detection method, device and equipment | |
| CN114024764A (en) | Monitoring method, monitoring system, equipment and storage medium for abnormal access of database | |
| CN111181982B (en) | Abnormal data identification method and device, computing equipment and medium | |
| CN114826639B (en) | Application attack detection method and device based on function call chain tracking | |
| US11695793B2 (en) | Vulnerability scanning of attack surfaces | |
| CN117201273A (en) | Automatic analysis and noise reduction method and device for safety alarm and server | |
| CN113656314A (en) | Pressure test processing method and device | |
| CN115563617A (en) | Source code vulnerability detection method and device | |
| CN113656313A (en) | Automatic test processing method and device | |
| CN114218283A (en) | Abnormality detection method, apparatus, device, and medium | |
| CN114422186A (en) | Attack detection method and device, electronic equipment and storage medium | |
| KR20210076455A (en) | Method and apparatus for automated verifying of xss attack | |
| US11042634B2 (en) | Determining information leakage of computer-readable programs | |
| CN112541183B (en) | Data processing method and device, edge computing equipment and storage medium | |
| US20240193075A1 (en) | Automate Load Test Scenario Based On Mapping With Real-Time Data Monitoring | |
| EP3640830B1 (en) | Method and system for determining risk in automotive ecu components | |
| CN111628984B (en) | Information processing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Room 332, 3 / F, Building 102, 28 xinjiekouwei street, Xicheng District, Beijing 100088 Patentee after: Qianxin Technology Group Co.,Ltd. Patentee after: Qianxin Wangshen information technology (Beijing) Co.,Ltd. Address before: Room 332, 3 / F, Building 102, 28 xinjiekouwei street, Xicheng District, Beijing 100088 Patentee before: Qianxin Technology Group Co.,Ltd. Patentee before: LEGENDSEC INFORMATION TECHNOLOGY (BEIJING) Inc. |