CN113315792B - Object extraction method and device of network data, electronic equipment and storage medium - Google Patents
Object extraction method and device of network data, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113315792B CN113315792B CN202110875253.3A CN202110875253A CN113315792B CN 113315792 B CN113315792 B CN 113315792B CN 202110875253 A CN202110875253 A CN 202110875253A CN 113315792 B CN113315792 B CN 113315792B
- Authority
- CN
- China
- Prior art keywords
- data
- data stream
- legal
- detected
- network data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 claims description 25
- 230000015654 memory Effects 0.000 claims description 18
- 230000014509 gene expression Effects 0.000 claims description 15
- 230000006399 behavior Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 238000013075 data extraction Methods 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a network data object extraction method, a network data object extraction device, electronic equipment and a storage medium, and relates to the field of computer information security, wherein the network data object extraction method comprises the following steps: acquiring to-be-detected network data sent by a server; performing access control on network data to be detected to obtain a legal data stream; extracting object knowledge characteristics from the legal data stream to obtain characteristics of the object to be detected; extracting object features to be verified of the network data to be verified from a preset object feature library according to the selection conditions; comparing the object characteristics to be detected with the object characteristics to be verified, and if the object characteristics to be detected accord with the object characteristics to be verified, acquiring object message data of a legal data stream; and grouping the object message data and extracting the object data of the legal data stream. The object extraction method of the network data can accurately screen the object data, improves the identification efficiency and realizes the object extraction of the network data message in the complex network environment.
Description
Technical Field
The present invention relates to the field of computer information security, and in particular, to a method and an apparatus for extracting an object from network data, an electronic device, and a storage medium.
Background
With the continuous development of the internet, the types of network services are more and more abundant, and the safety problem of information becomes a main problem facing the development of enterprises while the convenience and the rapidness brought by the information-based network are achieved.
Classifying and extracting objects in a network is an important part of research in network security level protection, and due to the diversity of data types, researchers often have difficulty in accurately identifying the classes of the objects in actual research work. In the conventional classification method, the object is mainly specified manually, and identification is usually performed according to online device function characteristics. However, because of the wide variety of network devices, the classification and identification are performed manually, which is time-consuming and labor-consuming, and the accuracy is poor. At present, the methods for automatically extracting objects are few, and typically, the methods extract the functional characteristics of the objects, and the types of the objects are identified through the characteristics. However, in a complex network environment, similar features exist among objects, and objects of the same type are different, which easily causes misjudgment when identifying and extracting objects in the network.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the embodiment of the invention provides an object extraction method for network data, which can accurately screen object data, improve the identification efficiency and realize object extraction of network data messages in a complex network environment.
The embodiment of the invention also provides an object extraction device of the network data.
The embodiment of the invention also provides the electronic equipment.
The embodiment of the invention also provides a computer readable storage medium.
An object extraction method for network data according to an embodiment of a first aspect of the present invention includes:
acquiring to-be-detected network data sent by a server;
performing access control on the network data to be detected to obtain a legal data stream;
carrying out object knowledge characteristic extraction on the legal data stream to obtain object characteristics to be detected corresponding to the legal data stream;
extracting object features to be verified of the network data to be verified from a preset object feature library according to the selection conditions;
comparing the object characteristics to be detected with the object characteristics to be verified, and if the object characteristics to be detected accord with the object characteristics to be verified, acquiring object message data of the legal data stream;
and grouping the object message data and extracting the object data of the legal data stream.
The object extraction method of the network data according to the embodiment of the first aspect of the invention has at least the following beneficial effects: the method comprises the steps of performing access control on network data to be detected, performing object knowledge feature extraction on an obtained legal data stream to obtain object features to be detected, extracting the object features to be verified of the network data to be verified from a preset object feature library according to selection conditions, comparing the object features to be detected with the object features to be verified, obtaining object message data of the network data to be detected if the object features to be detected meet the features, performing grouping processing on the object message data, extracting the object data of the network data to be detected, being capable of accurately screening the object data, improving identification efficiency and achieving object extraction of network data messages in a complex network environment.
According to some embodiments of the invention, the method further comprises: and if the object characteristics to be detected do not accord with the object characteristics to be verified, performing release processing on the legal data stream.
According to some embodiments of the invention, after the passing the legal data stream, the method further includes: acquiring the incidence relation between the legal data stream and the object feature library; and detecting the legal data stream and the characteristics of the object to be detected according to the incidence relation to obtain object data corresponding to the legal data stream.
According to some embodiments of the present invention, the grouping the object packet data and extracting the object data of the legal data stream includes: reading message protocol data and service message data of the legal data stream; grouping the object message data according to the message protocol data and the service message data to obtain a plurality of types of data; respectively constructing regular expressions for the plurality of types of data to generate local features; and combining the same parts and the local features according to a preset sequence to obtain the object data.
According to some embodiments of the present invention, the performing object knowledge feature extraction on the legal data stream to obtain an object feature to be detected corresponding to the legal data stream includes: carrying out named object identification on the legal data stream to obtain named object data; performing object relation extraction on the legal data stream according to the named object data to obtain object relation data; performing attribute extraction on the legal data stream according to the object relation data to obtain object information; and positioning the object information, classifying and abstracting to obtain the characteristics of the object to be detected.
According to some embodiments of the present invention, the performing access control on the network data to be detected to obtain a legal data stream includes: acquiring preset data configuration; and performing access control on the network data to be detected according to the preset data configuration to obtain the legal data stream.
According to some embodiments of the present invention, the method further includes establishing the object feature library, specifically including: further comprising establishing the object feature library, specifically comprising: acquiring a sample network data stream, and extracting a feature template of the sample network data stream to obtain a sample feature label and a sample service behavior; matching the sample network data stream and the sample characteristic label by using a regular expression to obtain a regular character string; confirming a service sequence between the sample service behaviors according to a preset session; and forming the object feature library according to the rule character strings, the service sequence and the sample network data stream.
An object extracting apparatus for network data according to a second aspect of the present invention includes:
the acquisition module is used for acquiring the to-be-detected network data sent by the server;
the access control module is used for carrying out access control on the network data to be detected to obtain legal data flow;
the identification module is used for extracting object knowledge characteristics of the legal data stream to obtain characteristics of the object to be detected corresponding to the legal data stream;
the extraction module is used for extracting the object characteristics to be verified of the network data to be verified from a preset object characteristic library according to the selection condition;
the comparison module is used for comparing the object characteristics to be detected with the object characteristics to be verified, and if the object characteristics to be detected accord with the object characteristics to be verified, object message data of the legal data stream is acquired;
and the extraction module is used for grouping the object message data and extracting the object data of the legal data stream.
The object extraction device of network data according to the embodiment of the second aspect of the present invention has at least the following beneficial effects: by implementing the object extraction method of the network data in the embodiment of the first aspect of the invention, the object data can be accurately screened out, the identification efficiency is improved, and the object extraction of the network data message in a complex network environment is realized.
An electronic device according to an embodiment of the third aspect of the invention includes: at least one processor, and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions, and the instructions are executed by the at least one processor, so that the at least one processor implements the object extraction method for network data according to the first aspect when executing the instructions.
According to the electronic device of the embodiment of the third aspect of the invention, at least the following beneficial effects are achieved: by implementing the object extraction method of the network data in the embodiment of the first aspect of the invention, the object data can be accurately screened out, the identification efficiency is improved, and the object extraction of the network data message in a complex network environment is realized.
According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the object extracting method for network data according to the first aspect.
The computer-readable storage medium according to the fourth aspect of the present invention has at least the following advantages: by implementing the object extraction method of the network data in the embodiment of the first aspect of the invention, the object data can be accurately screened out, the identification efficiency is improved, and the object extraction of the network data message in a complex network environment is realized.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of an object extraction method of network data according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a brain-like computing platform according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an object extracting apparatus for network data according to an embodiment of the present invention;
fig. 4 is a functional block diagram of an electronic device according to an embodiment of the invention.
Reference numerals:
the system comprises an acquisition module 300, an access control module 310, an identification module 320, an extraction module 330, a comparison module 340, an extraction module 350, a processor 400, a memory 410, a data transmission module 420, a camera 430 and a display screen 440.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
Classifying and extracting objects in a network is an important part of research in network security level protection, and due to the diversity of data types, researchers often have difficulty in accurately identifying the classes of the objects in actual research work. In the conventional classification method, the object is mainly specified manually, and identification is usually performed according to online device function characteristics. However, because of the wide variety of network devices, the classification and identification are performed manually, which is time-consuming and labor-consuming, and the accuracy is poor. At present, the methods for automatically extracting objects are few, and typically, the methods extract the functional characteristics of the objects, and the types of the objects are identified through the characteristics. However, in a complex network environment, similar features exist among objects, and objects of the same type are different, which easily causes misjudgment when identifying and extracting objects in the network.
Based on this, the embodiment of the invention provides an object extraction method and device for network data, an electronic device and a storage medium, which can accurately screen object data, improve recognition efficiency and realize object extraction of network data messages in a complex network environment.
Referring to fig. 1, an object extraction method for network data according to an embodiment of a first aspect of the present invention includes:
and step S100, acquiring the network data to be detected sent by the server.
The network data to be detected is a network data message sent by the server, and object extraction is performed on the network data to be detected sent by the server to obtain object data. And acquiring a new network data packet through the network data packet sent by the server, so as to obtain the network data to be detected. The network data to be detected may include structured data, semi-structured data, and unstructured data.
Step S110, access control is carried out on the network data to be detected, and legal data flow is obtained.
Optionally, an authorized object may configure an access control policy, where the access control policy specifies an access rule of the object to the object, and the access control rule in network security is to handle a validity problem set by the object to the access authority of the object, and may configure a user ip, a port type, a protocol, and the like in network data to be detected, and determine validity of the user ip, the port type, the protocol, and the like. The network data to be detected which accords with the configuration can be used as a legal data stream for further feature extraction; and not extracting the characteristics of the network data to be detected which do not conform to the configuration.
And step S120, carrying out object knowledge characteristic extraction on the legal data stream to obtain the characteristics of the object to be detected corresponding to the legal data stream.
The object characteristics to be detected can be object abstract characteristics of legal data streams; the named object recognition can be information extraction of semi-structured data and unstructured data in a legal data stream; object relation extraction, also called relation classification, namely classifying the relation between objects in legal data stream; the attribute extraction may be the extraction of attributes of objects and relationships. Optionally, the granularity of object feature extraction is mainly embodied in a file and database table level. Corresponding data operation is needed after legal data flow is acquired, knowledge extraction is the key part of the data operation on the legal data flow, and the knowledge extraction on the legal data flow mainly comprises three steps: naming object identification, object relation extraction and attribute extraction, extracting object knowledge characteristics of legal data streams in the above mode, further positioning the object information of the legal data streams, classifying and abstracting to obtain object abstract characteristics of the legal data streams, and obtaining the characteristics of the object to be detected. Referring to fig. 2, the brain-like computing platform shown in fig. 2 can provide computing power for extracting object knowledge features and establishing an object feature library, and due to the number of work flows and flow states of the business system and the complexity of conversion, a large computing power platform is required to support the real-time screening and analysis effect without interfering with the operation of the system business. The brain-like computer system adopts a parallel computing hypercube architecture system integrating computing, storage and communication, has super computing power, has the characteristics of convenient deployment and installation, and can provide great computing power for capturing and analyzing numerous network data packets in a complex network environment to construct a data object feature library. The brain-like computer system adopts a parallel computing hypercube architecture system integrating computing, storage and communication, uses the human brain information processing mode for reference, uses the neural system structure and the information processing mode of the human brain for simulation, simulation and learning, constructs a novel ultra-low power consumption type computing system with learning capacity, provides strong computing power for mass user access operation, state machine detection, data flow identification and judgment, and achieves the purposes of data analysis and extraction.
Step S120, extracting the object characteristics to be verified of the network data to be verified from a preset object characteristic library according to the selection conditions.
Optionally, the selection condition may be set according to requirements; the object feature library can be a preset database storing object features of network data packets; the network data to be verified can be a network data message stored in the object feature library; the object feature to be verified may be an abstract feature of the network data to be verified. Optionally, the plurality of network packets and the object characteristics thereof may form an object characteristic library by confirming the relationship between different object characteristics. In order to verify the abstract features of the legal data stream, pre-stored network data to be verified can be extracted from the object feature library, and corresponding abstract features of the network data can be extracted, where the abstract features include, but are not limited to: and comparing and analyzing the characteristics of the object to be verified and the characteristics of the object to be detected by using a TCP protocol, a ticket TDS protocol, a ticket service monitoring port and the like.
Step S130, comparing the object feature to be detected and the object feature to be verified, and if the object feature to be detected conforms to the object feature to be verified, obtaining object message data of a legal data stream.
The object message data may include a message protocol header of a legal data stream and a service message common portion. Optionally, the object characteristics to be detected of the legal data stream may be compared with the object characteristics to be verified, and whether the object characteristics to be detected meet the characteristics is determined. In some specific embodiments, if the characteristics of the object to be verified are the specified packet address and the port, all legal data streams outside the specified address and the port which do not conform to are removed, and only the legal data streams conforming to the specified address and the port are reserved; if the object feature to be verified specifies the transmission protocol, the legal data stream which does not conform to the specified protocol is removed, and only the legal data stream which conforms to the specified protocol is reserved. For example, let the object to be verified be characterized by TCP protocol, the characteristic information of TCP packet be structure, 16 bits of source and destination port, 32 bits of data sequence number, 32 bits of acknowledgement sequence, offset bit, reserved bit, etc. If the extracted characteristics of the object to be detected meet the characteristic information, if the characteristics meet the corresponding characteristic requirements of the address and the port of the designated data packet, the legal data flow meeting the designated address and the port is reserved, the legal data flow meeting the characteristic conditions is released, the legal data flow is normally sent to the destination server, and the network data flow to be detected meeting the characteristics of the object to be verified is sent to a passenger ticket system for continuous analysis and processing so as to extract the object information and carry out deep analysis.
Step S140, performing packet processing on the object message data, and extracting the object data of the legal data stream.
The object data may include a source address of the legal data stream and payload data, where the payload data is data content of the legal data stream. Optionally, for a legal data stream satisfying the characteristics, key information positioning, depth analysis, object extraction, and an exponential-level data packet filtering function (the exponential-level filtering is performed by the brain-like platform) may be performed on the legal data stream according to the brain-like computing algorithm platform shown in fig. 2. Specifically, object message data of a legal data stream can be read, the object message data includes a message protocol header and a service message common portion, the object message data is grouped, different types are marked out, regular expressions are respectively constructed in the different types to generate local features, and then the same portions and the local features are combined in sequence to obtain message features of an object, for example, a service message of the object may be: and responding id | object parameter 1| object parameter 2.. the request id can be a source address of legal data flow, and the object parameter 1, the object parameter 2 and the like can be payload data, so that the object data are extracted. In some embodiments, the object relationship is what the subject and the object are made of, and how the subject and the object are related to each other. The relationship between the subject and the object can be constructed and grasped through a large amount of data analysis and characteristic analysis.
According to the object data extraction method of the network data, access control is performed on the network data to be detected, object knowledge feature extraction is performed on an obtained legal data stream to obtain object features to be detected, object features to be verified of the network data to be verified are extracted from a preset object feature library according to selection conditions, the object features to be detected and the object features to be verified are compared, object message data of the network data to be detected are obtained if the object features to be detected meet the features, finally, grouping processing is performed on the object message data, the object data of the network data to be detected are extracted, the object data can be accurately screened, recognition efficiency is improved, and object extraction of network data messages in a complex network environment is achieved.
In some embodiments of the present invention, the object extracting method for network data further includes:
and if the object characteristics to be detected do not accord with the object characteristics to be verified, performing release processing on the legal data stream. Optionally, if the object feature to be detected does not conform to the object feature to be verified, for example, the object feature to be detected is a UDP packet, the object feature to be verified is a TCP data packet, and the legal data stream does not conform to the feature requirement, the legal data stream that does not conform to the object feature to be verified may be released, that is, the released service data packet may be screened out according to the configuration of the service system. Specifically, as shown in fig. 2, the feature comparison has a plurality of nodes, and when the data stream reaches the entry, the data stream is compared and screened by the first-level node, and then reaches the next-level node after the condition is satisfied. Each level of nodes can analyze and judge some characteristics of the data stream, such as whether the characteristics of a TCP packet are met, whether the characteristics of an HTTP protocol, a TDS protocol and the like are met, the data stream meeting the white list characteristics is released, and the data stream meeting the black list characteristics is discarded or intercepted.
In some embodiments of the present invention, after the passing the legal data stream, the method further includes:
and acquiring the association relation between the legal data stream and the object feature library. The association relationship may be an association relationship between a feature chain of a legal data stream and an object feature library. Optionally, the association refers to association between features, such as SYN in TCP packet, and association between ACK field and data transmission. In the TCP protocol, in order to ensure that data can be stably transmitted, the protocol uses two fields of SYN and ACK in a data packet to monitor whether the data is correctly transmitted and received. The association relation among all messages can be collated and learned by analyzing the object characteristics in the knowledge base.
And detecting the legal data stream and the characteristics of the object to be detected according to the association relation to obtain object data corresponding to the legal data stream. Optionally, as shown in fig. 2, the egress node performs corresponding processing on the data meeting the condition, releases the data stream meeting the white list feature, discards or intercepts the data stream meeting the black list feature, and performs feature sampling on the data stream outside the feature library (i.e., the released data stream) and adds the feature sample to the feature library. Specifically, the characteristics of the legal data stream and the data stream can be synchronously detected according to the association relationship between the characteristic chain of the legal data stream and the object characteristic library, and corresponding object content in the legal data stream is extracted, so that the object data is obtained. The method has the advantages that the legal data flow which does not meet the characteristics is released, the object data corresponding to the legal data flow are detected according to the incidence relation between the legal data flow and the object characteristic library, released data packets can be extracted and analyzed additionally, the object data are obtained, and the efficiency is improved.
In some embodiments of the present invention, performing packet processing on object packet data, and extracting object data of a legal data stream includes:
and reading the message protocol data and the service message data of the legal data stream. The message protocol data and the service message data may be a common part of a message protocol header and a service message of a legal data stream, and the format of the common part of the object service message in the object message data may be: the service message of the object may be a response id object parameter 1 object parameter 2.
And grouping the object message data according to the message protocol data and the service message data to obtain a plurality of types of data. The type data may be different types of data obtained by grouping object message data of the legal data stream. Optionally, the object message data of the legal data stream may include a message protocol header and a service message common part, and a format of the service message common part in the object message data may be: the response id | object parameter 1| object parameter 2 … … can group the object message data as required, and mark out different types of data, i.e. multiple types of data.
And constructing regular expressions for the plurality of types of data respectively to generate local features. The regular expression is also called as regular expression, and the regular expression is used for retrieving and replacing texts conforming to a certain mode (rule); the local features may be features corresponding to each type of data. Optionally, for different types of data, regular expressions may be respectively constructed to generate local features, that is, local features corresponding to each type of data are generated.
And combining the same parts and the local characteristics according to a preset sequence to obtain object data. Wherein the preset order may be an order of combining local features set in advance. Optionally, the preset sequence may be set as required, and the same part and the local feature may be combined according to the preset sequence to obtain the message feature, so as to extract the object data. The object message data are grouped to obtain a plurality of types of data, then the plurality of types of data are respectively constructed into regular expressions to generate local features, finally the same parts and the local features are combined according to a preset sequence to obtain object data, and objects in a legal data stream are classified and extracted by using a deep learning method, so that the efficiency is improved, the accuracy is achieved, and the time and the resources can be effectively saved.
In some embodiments of the present invention, performing object knowledge feature extraction on a legal data stream to obtain an object feature to be detected corresponding to the legal data stream, includes:
and carrying out named object identification on the legal data stream to obtain named object data. Optionally, named object recognition is the first step of information extraction for semi-structured data and unstructured data, and often the object is the main carrier of information. The object may be a person, a place name, or some concept. The required object can be extracted through character string matching or manual operation, and the object can be extracted through natural language processing and machine learning to obtain named object information.
And performing object relation extraction on the legal data stream according to the named object data to obtain object relation data. Optionally, the object relationship extraction is also called relationship classification, and in order to determine an object-relationship-object triple, the relationship between objects needs to be classified, that is, extraction of semantic information. In some specific embodiments, according to named object data, a sentence containing two objects with a relationship is subjected to relationship labeling through feature engineering, so that supervised learning is realized, and the accuracy of a model is improved to a certain extent.
And performing attribute extraction on the legal data stream according to the object relation data to obtain object information. Optionally, after the "object-relationship-object" triple is constructed according to the object relationship data, the attributes of the object and the relationship need to be extracted, and the attribute extraction may be directly obtained through a network, and the attributes may also be regarded as the object relationship.
And positioning the object information, classifying and abstracting to obtain the characteristics of the object to be detected. Optionally, the object information of the legal data stream may be located according to the object data features that the user needs to extract, and the classification and abstraction are performed, that is, the characteristic analysis is performed on the legal data stream to obtain the abstract features of the legal data stream, that is, the features of the object to be detected are obtained, and the brain-like computing platform shown in fig. 2 may be used to support the calculation power for the feature analysis of the legal data stream. The object characteristics to be detected are obtained by performing characteristic extraction on the legal data stream and then positioning, classifying and abstracting the legal data stream, and the unsupervised or semi-supervised characteristic learning and layered characteristic extraction efficient algorithm replaces manual characteristic acquisition, so that the object extraction of the network data message in the complex network environment is realized.
In some embodiments of the present invention, performing access control on network data to be detected to obtain a legal data stream includes:
and acquiring preset data configuration. The preset data configuration may be a preset relevant configuration of the network data to be detected. Optionally, the preset data configuration may be set according to a requirement, for example, the preset data configuration may be a configuration such as preset user ip information, a port type, or a protocol.
And performing access control on the network data to be detected according to the preset data configuration to obtain a legal data stream. Optionally, the network data to be detected may be screened through preset data configuration, for example, assuming that the preset data configuration is user ip information, port type or protocol, and the like, if the network data to be detected meets the configuration, the network data to be detected is used as a legal data stream for further feature analysis; if the data to be detected does not meet the configuration, the legality of the network data to be detected does not meet the preset requirement, so that further feature extraction of the data to be detected is not needed, and new network data to be detected can be obtained again for access control. The network data to be detected is subjected to access control through preset data configuration, only the legal data flow is allowed to be subjected to further feature extraction, and the legal data flow meeting the requirement can be screened out.
In some embodiments of the present invention, the method further includes establishing an object feature library, specifically including:
and acquiring a sample network data stream, and extracting a feature template of the sample network data stream to obtain a sample feature label and a sample service behavior. Wherein, the sample network data flow may be a data flow (i.e. a data flow on the white list) which is pre-screened and satisfies the legal condition; the sample feature tags and sample traffic behavior may be tag data material and traffic behavior data material corresponding to the sample network data stream. Optionally, if the preset condition is configuration of a user ip, a port, a protocol, and the like of the data stream, the data stream conforming to the configuration is taken as a legal data stream, and the data stream not conforming to the configuration is removed. And generating a sample feature chain corresponding to the sample network data stream according to the sample feature label and the sample business behavior to construct an object feature library.
And matching the sample network data stream and the sample characteristic label by using a regular expression to obtain a regular character string. Where regular strings may be used to express a filtering logic for the strings. Optionally, the feature template extraction may be performed on the requested sample network data stream, and then the extraction result is matched by using a regular expression, where the regular expression is a logic formula for operating the sample network data stream and the sample feature tag, and is a rule character string formed by using some specific characters defined in advance and a combination of the specific characters, and is used for describing a filtering logic of the object feature library.
And confirming the service sequence among the sample service behaviors according to the preset session. The preset session may be a preset reliable communication mode facing to the connection. Optionally, the sequence relationship between the sample service behaviors may be confirmed through a session, that is, a service sequence is obtained and used for describing a data arrangement condition in the object feature library.
And forming an object feature library according to the rule character strings, the service sequence and the sample network data stream. Optionally, the sample network data stream, the sample feature tag corresponding to the sample network data stream, and the sample business behavior may be imported into the object feature library according to a rule character string for describing a filtering logic of the object feature library and a business sequence for describing a data arrangement condition in the object feature library, so as to construct the object feature library. Computational support for feature extraction may be provided by a brain-like computing platform as shown in fig. 2. The object feature library is constructed through the sample network data stream, so that the accurate comparison and acquisition of the data entity content can be realized, and the purposes of data analysis and data extraction are achieved.
Referring to fig. 3, an object extracting apparatus for network data according to a second embodiment of the present invention includes:
the acquiring module 300 is configured to acquire network data to be detected sent by a server;
the access control module 310 is configured to perform access control on network data to be detected to obtain a legal data stream;
the identification module 320 is configured to perform object knowledge feature extraction on the legal data stream to obtain an object feature to be detected corresponding to the legal data stream;
the extracting module 330 is configured to extract an object feature to be verified of the network data to be verified from a preset object feature library according to a selection condition;
the comparison module 340 is configured to compare the object feature to be detected with the object feature to be verified, and if the object feature to be detected conforms to the object feature to be verified, obtain object message data of a legal data stream;
the extraction module 350 is configured to perform packet processing on the object packet data, and extract object data of a legal data stream.
By implementing the object extraction method of the network data in the embodiment of the first aspect of the invention, the object extraction device of the network data can accurately screen the object data, improve the identification efficiency and realize the object extraction of the network data message in a complex network environment.
Referring to fig. 4, an embodiment of the third aspect of the present invention further provides a functional module diagram of an electronic device, including: at least one processor 400, and a memory 410 communicatively coupled to the at least one processor 400; and the system also comprises a data transmission module 420, a camera 430 and a display screen 440.
The processor 400 is configured to execute the object extraction method of the network data in the first aspect embodiment by calling the computer program stored in the memory 410.
The data transmission module 420 is connected to the processor 400, and is used for implementing data interaction between the data transmission module 420 and the processor 400.
The cameras 430 may include a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera 430 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.
The display screen 440 may be used to display information entered by the user or provided to the user. The Display screen 440 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel may cover the display panel, and when the touch panel detects a touch operation thereon or nearby, the touch panel transmits the touch operation to the processor 400 to determine the type of the touch event, and then the processor 400 provides a corresponding visual output on the display panel according to the type of the touch event. In some embodiments, the touch panel may be integrated with the display panel to implement input and output functions.
The memory, as a non-transitory storage medium, may be used to store a non-transitory software program and a non-transitory computer-executable program, such as the object extraction method of network data in the embodiment of the first aspect of the present invention. The processor implements the object extraction method of network data in the above-described first embodiment by executing a non-transitory software program and instructions stored in the memory.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store an object extraction method for executing the network data in the embodiment of the first aspect. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and these remote memories may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Non-transitory software programs and instructions required to implement the object extraction method for network data in the first aspect of the embodiment described above are stored in a memory, and when executed by one or more processors, perform the object extraction method for network data in the first aspect of the embodiment described above.
Embodiments of the fourth aspect of the present invention also provide a computer-readable storage medium storing computer-executable instructions for: the object extraction method of network data in the first aspect embodiment is performed.
In some embodiments, the storage medium stores computer-executable instructions, which are executed by one or more control processors, for example, by one of the processors in the electronic device of the third aspect, and may cause the one or more processors to execute the object extraction method of network data in the first aspect.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
The above-described embodiments of the apparatus are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may also be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims (9)
1. An object extraction method for network data, comprising:
acquiring to-be-detected network data sent by a server;
performing access control on the network data to be detected to obtain a legal data stream;
carrying out object knowledge characteristic extraction on the legal data stream to obtain object characteristics to be detected corresponding to the legal data stream;
extracting object features to be verified of the network data to be verified from a preset object feature library according to the selection conditions;
comparing the object characteristics to be detected with the object characteristics to be verified, and if the object characteristics to be detected accord with the object characteristics to be verified, acquiring object message data of the legal data stream;
reading message protocol data and service message data of the legal data stream;
grouping the object message data according to the message protocol data and the service message data to obtain a plurality of types of data;
respectively constructing regular expressions for the plurality of types of data to generate local features;
and combining the same parts and the local features according to a preset sequence to obtain object data.
2. The method of claim 1, further comprising:
and if the object characteristics to be detected do not accord with the object characteristics to be verified, performing release processing on the legal data stream.
3. The method of claim 2, further comprising, after said passing said legal data stream,:
acquiring the incidence relation between the legal data stream and the object feature library;
and detecting the legal data stream and the characteristics of the object to be detected according to the incidence relation to obtain object data corresponding to the legal data stream.
4. The method according to claim 1, wherein the performing object knowledge feature extraction on the legal data stream to obtain object features to be detected corresponding to the legal data stream comprises:
carrying out named object identification on the legal data stream to obtain named object data;
performing object relation extraction on the legal data stream according to the named object data to obtain object relation data;
performing attribute extraction on the legal data stream according to the object relation data to obtain object information;
and positioning the object information, classifying and abstracting to obtain the characteristics of the object to be detected.
5. The method according to claim 1, wherein the performing access control on the network data to be detected to obtain a legal data flow comprises:
acquiring preset data configuration;
and performing access control on the network data to be detected according to the preset data configuration to obtain the legal data stream.
6. The method according to claim 1, further comprising establishing the object feature library, specifically comprising:
acquiring a sample network data stream, and extracting a feature template of the sample network data stream to obtain a sample feature label and a sample service behavior;
matching the sample network data stream and the sample characteristic label by using a regular expression to obtain a regular character string;
confirming a service sequence between the sample service behaviors according to a preset session;
and forming the object feature library according to the rule character strings, the service sequence and the sample network data stream.
7. An object extraction device for network data, comprising:
the acquisition module is used for acquiring the to-be-detected network data sent by the server;
the access control module is used for carrying out access control on the network data to be detected to obtain legal data flow;
the identification module is used for extracting object knowledge characteristics of the legal data stream to obtain characteristics of the object to be detected corresponding to the legal data stream;
the extraction module is used for extracting the object characteristics to be verified of the network data to be verified from a preset object characteristic library according to the selection condition;
the comparison module is used for comparing the object characteristics to be detected with the object characteristics to be verified, and if the object characteristics to be detected accord with the object characteristics to be verified, object message data of the legal data stream is acquired;
the extraction module is used for reading message protocol data and service message data of the legal data stream, grouping the object message data according to the message protocol data and the service message data to obtain a plurality of types of data, respectively constructing a regular expression for the plurality of types of data to generate local features, and combining the same part and the local features according to a preset sequence to obtain the object data.
8. An electronic device, comprising:
at least one processor, and,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions for execution by the at least one processor to cause the at least one processor, when executing the instructions, to implement the object extraction method for network data according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that the storage medium stores computer-executable instructions for causing a computer to execute the object extraction method of network data according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110875253.3A CN113315792B (en) | 2021-07-30 | 2021-07-30 | Object extraction method and device of network data, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110875253.3A CN113315792B (en) | 2021-07-30 | 2021-07-30 | Object extraction method and device of network data, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113315792A CN113315792A (en) | 2021-08-27 |
| CN113315792B true CN113315792B (en) | 2021-11-30 |
Family
ID=77382397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110875253.3A Active CN113315792B (en) | 2021-07-30 | 2021-07-30 | Object extraction method and device of network data, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113315792B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347165A (en) * | 2019-08-08 | 2021-02-09 | 腾讯科技（深圳）有限公司 | Log processing method and device, server and computer readable storage medium |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101252441B (en) * | 2008-02-20 | 2010-06-02 | 深圳市永达电子股份有限公司 | Acquired safety control method and system based on target capable of setting information safety |
| CN102413198A (en) * | 2011-09-30 | 2012-04-11 | 山东中创软件工程股份有限公司 | Security-marker-based access control method and related system |
| CN102394885B (en) * | 2011-11-09 | 2015-07-15 | 中国人民解放军信息工程大学 | Information classification protection automatic verification method based on data stream |
| CN105719033B (en) * | 2014-12-02 | 2019-12-13 | 阿里巴巴集团控股有限公司 | Method and device for identifying object risk |
| KR101708491B1 (en) * | 2015-04-03 | 2017-02-20 | 삼성에스디에스 주식회사 | Method for recognizing object using pressure sensor |
| CN107992758B (en) * | 2017-11-29 | 2020-01-14 | 中国人民解放军信息工程大学 | Dynamic management method and device for security mechanism |
| US11307541B2 (en) * | 2019-09-06 | 2022-04-19 | Intelligent Fusion Technology, Inc. | Decision support method and apparatus for machinery control |
| CN111178075A (en) * | 2019-12-19 | 2020-05-19 | 厦门快商通科技股份有限公司 | Online customer service log analysis method, device and equipment |
| CN112199488B (en) * | 2020-11-04 | 2023-09-26 | 国网江苏省电力有限公司营销服务中心 | Incremental knowledge graph entity extraction method and system for power customer service question and answer |
-
2021
- 2021-07-30 CN CN202110875253.3A patent/CN113315792B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347165A (en) * | 2019-08-08 | 2021-02-09 | 腾讯科技（深圳）有限公司 | Log processing method and device, server and computer readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 一种网络访问控制语义相容性推理规则自动生成技术;韦丽娟;《软件导刊》;20180123(第02期);全文 * |
| 信息系统内部威胁检测技术研究;王振辉等;《计算机系统应用》;20191215(第12期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113315792A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11301778B2 (en) | Method and system for training and validating machine learning in network environments | |
| US10795992B2 (en) | Self-adaptive application programming interface level security monitoring | |
| US9922287B2 (en) | Identification and classification of web traffic inside encrypted network tunnels | |
| US20080259919A1 (en) | Method for Dynamic Sensor Network Processing | |
| CN115600128A (en) | Semi-supervised encrypted traffic classification method and device and storage medium | |
| CN114422271B (en) | Data processing method, device, equipment and readable storage medium | |
| EP3348026A1 (en) | Optimized complex event processing in a software-defined network | |
| CN110225009A (en) | It is a kind of that user's detection method is acted on behalf of based on communication behavior portrait | |
| CN113315792B (en) | Object extraction method and device of network data, electronic equipment and storage medium | |
| CN116828087B (en) | Information security system based on block chain connection | |
| CN105100246A (en) | Network flow management and control method based on downloaded resource name | |
| CN112436969A (en) | Internet of things equipment management method, system, equipment and medium | |
| Wan et al. | DevTag: A benchmark for fingerprinting IoT devices | |
| CN113313216B (en) | Method and device for extracting main body of network data, electronic equipment and storage medium | |
| Qu et al. | A novel method for network traffic classification based on robust support vector machine | |
| CN116192527A (en) | Attack flow detection rule generation method, device, equipment and storage medium | |
| CN116248346A (en) | Smart city-oriented CPS network security situation awareness establishing method and system | |
| CN116055587A (en) | Method and device for realizing hierarchical classification of API (application program interface) assets | |
| CN109194756A (en) | Application features information extracting method and device | |
| Kumar et al. | Machine learning based traffic classification using low level features and statistical analysis | |
| Yichiet et al. | A semantic-aware log generation method for network activities | |
| Cheng et al. | Identify IoT devices through web interface characteristics | |
| Zhang et al. | IoTminer: Semantic Information Extraction in the Packet Payloads | |
| Yuchao et al. | The Construction and Experimental Approach of Anonymous Network Analysis and Control Platform | |
| CN112367326B (en) | Method and device for identifying traffic of Internet of vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |