CN114760369A - Protocol metadata extraction method, device, equipment and storage medium - Google Patents
Protocol metadata extraction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114760369A CN114760369A CN202210393134.9A CN202210393134A CN114760369A CN 114760369 A CN114760369 A CN 114760369A CN 202210393134 A CN202210393134 A CN 202210393134A CN 114760369 A CN114760369 A CN 114760369A
- Authority
- CN
- China
- Prior art keywords
- protocol
- metadata
- item
- extraction
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 149
- 238000012545 processing Methods 0.000 claims abstract description 69
- 238000000034 method Methods 0.000 claims abstract description 48
- 230000006870 function Effects 0.000 claims description 42
- 230000015654 memory Effects 0.000 claims description 30
- 230000006798 recombination Effects 0.000 claims description 22
- 238000005215 recombination Methods 0.000 claims description 22
- 238000013507 mapping Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 17
- 239000000284 extract Substances 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 description 17
- 230000008569 process Effects 0.000 description 17
- 238000004891 communication Methods 0.000 description 10
- 230000003993 interaction Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/12—Protocol engines
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9057—Arrangements for supporting packet reassembly or resequencing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/03—Protocol definition or specification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer And Data Communications (AREA)
- Communication Control (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for extracting protocol metadata. The method comprises the following steps: receiving a data packet and identifying a target protocol corresponding to the data packet; calling a protocol processing function registered by the target protocol, and determining a metadata extraction mode; if the feature matching extraction is carried out, an extraction rule engine corresponding to the target protocol identification is called to scan the load, and the metadata is extracted; if the syntax is extracted by parsing, a syntax rule engine corresponding to the target protocol identification is called, and the metadata is extracted according to the description parsing load of the syntax rule protocol item corresponding to the target protocol identification. According to the technical scheme of the embodiment of the invention, only part of flow related to the metadata is processed by calling the extraction rule engine or the grammar rule engine, so that the code quality and the processing efficiency of metadata extraction are improved.
Description
Technical Field
The present invention relates to the field of data communication technologies, and in particular, to a method, an apparatus, a device, and a storage medium for extracting protocol metadata.
Background
The metadata is also called medium data, and is data for describing data attribute information, and the analysis service at the back end usually performs correlation processing of different dimensions based on the reported metadata.
In the prior art, writing codes for each protocol realizes all interactive realization of the protocol, and metadata under each protocol is extracted by processing all traffic under the protocol. However, this metadata extraction method requires a large amount of written code and low development efficiency, and requires a large amount of data to be processed, resulting in low metadata extraction efficiency.
Disclosure of Invention
The invention provides a method, a device, equipment and a storage medium for extracting protocol metadata, which only process part of flow related to the metadata by calling an extraction rule engine or a grammar rule engine, thereby improving the code quality and the processing efficiency of metadata extraction.
According to an aspect of the present invention, there is provided a protocol metadata extraction method, including:
receiving a data packet and identifying a target protocol corresponding to the data packet;
calling a protocol processing function registered by the target protocol, and determining a metadata extraction mode;
if the feature matching extraction is carried out, an extraction rule engine corresponding to the target protocol identification is called to scan the load, and the metadata is extracted;
If the syntax analysis extraction is performed, a syntax rule engine corresponding to the target protocol identification is called, and the metadata is extracted according to the description analysis load of the syntax rule protocol item corresponding to the target protocol identification.
Optionally, before the invoking of the extraction rule engine corresponding to the target protocol identifier scans the load and extracts the metadata, the method further includes:
loading and analyzing metadata extraction rule files of each protocol, creating extraction rule engines corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the extraction rule engines;
and synchronizing the extraction rule engine and the mapping relation between the protocol identifier and the extraction rule engine to each Non-Uniform Memory Access (NUMA) node.
By adopting the technical scheme, the metadata extraction rule engine is constructed based on the NUMA architecture, the multi-core characteristic of the CPU is fully utilized, and the processing efficiency of metadata extraction according to feature matching is improved.
Optionally, before the invoking of the grammar rule engine corresponding to the target protocol identifier, parsing the load according to the description of the grammar rule protocol item corresponding to the target protocol identifier, and extracting the metadata, the method further includes:
Loading and analyzing a metadata grammar rule description file of each protocol, creating grammar rule protocol items corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the grammar rule protocol items;
and synchronizing the grammar rule protocol item and the mapping relation between the protocol identification and the grammar rule protocol item to each NUMA node.
By adopting the technical scheme, the grammar rule engine based on the grammar rule protocol item is constructed based on the NUMA architecture, the multi-core characteristic of the CPU is fully utilized, and the processing efficiency of extracting the metadata according to grammar analysis is improved.
Optionally, parsing the load according to the description of the grammatical rule protocol item corresponding to the target protocol identifier, and extracting metadata includes:
traversing a basic item linked list in the grammar rule protocol item;
judging whether the load is consistent with the condition item description information in the basic item;
and if the basic items are consistent, acquiring a field item linked list in the basic items to analyze the load, and extracting the metadata.
By adopting the technical scheme, a universal grammar rule is defined aiming at binary streams with dependence among protocol metadata, whether the load needs to extract the metadata is judged according to condition items in the grammar rule, if so, the load is processed according to field items to extract the metadata, and the problem that the metadata can not be obtained from the binary streams with dependence among the metadata through characteristic rule matching is solved.
Optionally, the determining whether the load is consistent with the condition item description information in the basic item includes:
extracting condition data from the load according to the offset information and the length information in the condition item description information;
determining a matching condition according to numerical value information and operation information in the condition item description information, and if the condition data meets the matching condition, determining that the load is consistent with the condition item description information;
otherwise, determining that the load is inconsistent with the condition item description information.
By adopting the technical scheme, the offset and field of the protocol can be described according to the definition of the metadata grammar rule without developing experience of a worker by putting the parsing offset logic into the grammar rule, so that the development efficiency is greatly improved. Meanwhile, the problem that the program is hung due to the fact that offset value taking codes are not strict can be solved.
Optionally, the obtaining a field item linked list in the basic item to analyze the load and extract metadata includes:
traversing a field item linked list in the basic item;
processing the load according to the field name, the field type and the field length in the field item description information to obtain binary information corresponding to the field item;
And processing the binary information according to the field result description and the field operation description in the field description information to obtain the metadata corresponding to the field item.
By adopting the technical scheme, according to the description of the metadata in the field item description information, the binary information of the metadata is extracted from the load, and the extracted data is subjected to data type conversion and other processing, so that the protocol metadata which can be directly used is obtained, and further processing of the protocol metadata in subsequent use is avoided.
Optionally, the calling the protocol processing function registered by the target protocol to determine a metadata extraction manner includes:
executing a protocol processing function registered by the target protocol, and judging whether the current data stream is recombined or not;
if yes, recombining the received data packet according to the set recombined correlation information;
if the recombination is finished, calling a recombination finishing callback function registered by the target protocol, and determining the type of the engine used for extracting the metadata.
By adopting the technical scheme, functions such as a protocol processing function, a recombination ending callback function and the like are registered according to the analysis processing flow of the protocol metadata by each protocol, so that when the metadata required by the output protocol needs to be recombined, the message is recombined and then the metadata is extracted, and the correct extraction of the metadata is ensured.
According to another aspect of the present invention, there is provided a protocol metadata extraction apparatus including:
the protocol identification module is used for receiving the data packet and identifying a target protocol corresponding to the data packet;
the protocol processing module is used for executing a protocol processing function for calling the target protocol registration and determining a metadata extraction mode;
the characteristic matching module is used for executing extraction for characteristic matching, calling an extraction rule engine corresponding to the target protocol identifier to scan the load and extracting the metadata;
and the grammar parsing module is used for calling a grammar rule engine corresponding to the target protocol identifier if the grammar parsing extraction is carried out, and extracting the metadata according to the description parsing load of the grammar rule protocol item corresponding to the target protocol identifier.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the protocol metadata extraction method according to any of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the protocol metadata extraction method according to any one of the embodiments of the present invention when the computer instructions are executed.
According to the technical scheme of the embodiment of the invention, a data packet is received, and a target protocol corresponding to the data packet is identified; calling a protocol processing function registered by the target protocol, and determining a metadata extraction mode; if the feature matching extraction is carried out, an extraction rule engine corresponding to the target protocol identification is called to scan the load, and the metadata is extracted; if the syntax is extracted by syntax analysis, a syntax rule engine corresponding to the target protocol identification is called, the metadata is extracted according to the description analysis load of the syntax rule protocol item corresponding to the target protocol identification, the problems that in the prior art, the metadata extraction mode needs more codes to be written and the metadata extraction efficiency is low are solved, the method only processes the flow of part of the metadata by calling the extraction rule engine or the syntax rule engine, and the beneficial effects of improving the code quality and the processing efficiency of metadata extraction are achieved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a protocol metadata extraction method according to an embodiment of the present invention;
fig. 2 is a flowchart of a protocol metadata extraction method according to a second embodiment of the present invention;
FIG. 3 is a flow diagram of a metadata grammar rules engine process adapted according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a protocol metadata extraction apparatus according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device implementing the protocol metadata extraction method according to the embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, shall fall within the protection scope of the present invention.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of this invention and the above-described drawings are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1 is a flowchart of a protocol metadata extraction method according to an embodiment of the present invention, where this embodiment is applicable to a situation where protocol metadata extraction is performed on multiple protocols quickly and accurately, and the method may be performed by a protocol metadata extraction apparatus, which may be implemented in hardware and/or software and may be configured in an electronic device. As shown in fig. 1, the method includes:
S110, receiving the data packet, and identifying a target protocol corresponding to the data packet.
The data packets are units of binary format transmitted in the network, and each data packet comprises two parts: a message header consisting of control information and a message payload consisting of data information. Specifically, the message contains the complete data information to be transmitted, and during the transmission process, in order to provide communication performance and reliability, the message is divided into a plurality of smaller parts, and a message header consisting of some necessary control information is added in front of each part to be packaged into a data packet.
In this embodiment, since metadata extraction manners corresponding to different protocol packets may be different, after receiving a current data packet, a target protocol corresponding to the data packet may be identified according to a specified field in a header of the data packet, or a protocol type corresponding to the data packet may also be determined by using other protocol identification techniques.
And S120, calling a protocol processing function registered by the target protocol, and determining a metadata extraction mode.
In this embodiment, since the metadata parsing process flows of each protocol are different, the basic process flow of the protocol metadata may be predefined, a registration interface may be provided, and different protocols may implement their own interface processing functions, registration protocol processing functions, a reassembly end callback function, and a timeout processing function to perform metadata processing. Therefore, after the protocol type of the current data packet is determined, the protocol processing function registered by the protocol can be called to process the message, and the engine type used for extracting the metadata is determined.
The metadata refers to some field information focused by a user in the protocol traffic, and may be understood as a group of information used for describing data, and all information in the group of information is used for describing some aspect feature of some data.
Optionally, the calling the protocol processing function registered by the target protocol to determine a metadata extraction manner includes: executing a protocol processing function registered by the target protocol, and judging whether the current data stream is recombined or not; if yes, recombining the received data packet according to the set recombined correlation information; if the recombination is finished, calling a recombination finishing callback function registered by the target protocol, and determining the type of the engine used for extracting the metadata.
The data stream may include a plurality of data packets into which the same packet is segmented and encapsulated. In this embodiment, by executing a protocol processing function registered by a target protocol, it is determined whether to output metadata required by the target protocol and to perform packet reassembly on a data stream where a current data packet is located. If the message recombination is needed, the recombination associated information is set, including the information of the direction of the recombination, the sequence number of the starting data packet, the recombination ending mode and the like, and the recombination process is carried out according to the information. And after the message recombination is finished, calling a recombination finishing callback function registered by a target protocol, and further determining an engine type used for extracting the metadata. If the recombination is not needed, the engine type used by the metadata is directly obtained through a protocol processing function.
By adopting the technical scheme, functions such as protocol processing functions, restructuring ending callback functions and the like are registered according to the protocol metadata analysis processing flow by each protocol, so that when metadata required by an output protocol needs to be restructured, the message is firstly restructured and then metadata is extracted, correct extraction of the metadata is ensured, and missing extraction of the metadata is avoided.
And S130, if the feature matching extraction is carried out, calling an extraction rule engine corresponding to the target protocol identification to scan the load, and extracting the metadata.
In this embodiment, if the protocol packet may obtain the start position, the length, or the end position of metadata extraction through feature string matching, the type of the engine obtained from the protocol processing function is an extraction rule engine. Therefore, the extraction rule engine corresponding to the target protocol identifier can be called, the information related to the metadata in the packet load of the data packet can be acquired through regular rule matching based on the regular expression matching library, and meanwhile, the extracted information can be correspondingly processed according to data result description in the regular rule, such as data type conversion, decoding and the like, so that the protocol metadata which can be directly used can be acquired.
Optionally, before the invoking of the extraction rule engine corresponding to the target protocol identifier scans the load and extracts the metadata, the method further includes: loading and analyzing metadata extraction rule files of each protocol, creating extraction rule engines corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the extraction rule engines; and synchronizing the extraction rule engine and the mapping relation between the protocol identification and the extraction rule engine to each NUMA node.
The inside of one NUMA node consists of a processor core and all local memories of the processor core, and the speed of directly accessing the memory managed by each NUMA node is higher than the speed of accessing the memories of other NUMA nodes through channels.
In this embodiment, metadata extraction rule files for analyzing different protocols may be loaded in the main thread, and an extraction rule engine corresponding to each protocol identifier may be created. The extraction rule comprises two parts of matching feature strings and operations executed after matching. And then each thread applies for a memory from a local NUMA node, the related data structure of the extraction rule engine in the main thread is copied to the NUMA memory of each NUMA node, and a thread-level extraction rule engine is created. When the thread processes the message, the local extraction rule engine can be called to execute extraction rule matching.
By adopting the technical scheme, the multi-core processor can simultaneously run a plurality of tasks, so that one metadata extraction task can be divided into a plurality of threads to be executed in order to improve the metadata extraction efficiency. In order to fully utilize the multi-core characteristics of the CPU, a thread-level metadata extraction rule engine may be constructed based on a NUMA architecture, so that when each thread processes a packet, the local extraction rule engine may be invoked to extract metadata by accessing a local NUMA memory. Therefore, according to the characteristic that the speed of the processor core accessing the NUMA node memory is higher than the speed of the processor core accessing other node memories, the processing efficiency of extracting the metadata according to the feature matching is improved.
And S140, if the syntax is extracted by syntax analysis, calling a syntax rule engine corresponding to the target protocol identifier, and extracting the metadata according to the description analysis load of the syntax rule protocol item corresponding to the target protocol identifier.
In this embodiment, if the protocol packet is a binary stream and there is a dependency between the protocol metadata, the engine type obtained from the protocol processing function is a syntax rule engine. Therefore, when a message load is received, a grammar rule engine corresponding to the target protocol identification is called, namely, a corresponding grammar rule protocol item is found according to the target protocol identification, condition judgment is carried out according to the description of the grammar rule protocol item, if the condition is met, the message load sequence is analyzed downwards according to the field item, and metadata information is extracted. Wherein the syntax rules define condition entries and field entries.
Optionally, before the invoking the grammar rule engine corresponding to the target protocol identifier, parsing the load according to the description of the grammar rule protocol item corresponding to the target protocol identifier, and extracting the metadata, the method further includes: loading and analyzing a metadata grammar rule description file of each protocol, creating grammar rule protocol items corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the grammar rule protocol items; and synchronizing the grammar rule protocol item and the mapping relation between the protocol identification and the grammar rule protocol item to each NUMA node.
In this embodiment, in the main thread, the metadata grammar rule description file of each protocol may be loaded and parsed, grammar rule protocol items corresponding to each protocol identifier, that is, combination relationship items of the matching condition and the field item, may be created, and a mapping relationship between the protocol identifier and the grammar rule protocol items may be established. And then each thread applies for a memory from the local NUMA node, the related data structure of the grammar rule protocol item in the main thread is copied to the NUMA memory of each NUMA node, and a thread-level grammar rule engine is created. When the thread processes the message, the local grammar rule engine can be called to extract the metadata.
By adopting the technical scheme, the multi-core processor can simultaneously run a plurality of tasks, so that one metadata extraction task can be divided into a plurality of threads to be executed in order to improve the metadata extraction efficiency. In order to fully utilize the multi-core characteristics of the CPU, a thread-level metadata grammar rule engine can be constructed based on a NUMA architecture, so that when each thread processes a message, the local NUMA memory can be accessed to invoke the local grammar rule engine to extract metadata. Therefore, according to the characteristic that the speed of the processor core accessing the NUMA node memory is higher than the speed of accessing other node memories, the processing efficiency of extracting the metadata according to the syntax analysis is improved.
According to the technical scheme of the embodiment of the invention, a data packet is received, and a target protocol corresponding to the data packet is identified; calling a protocol processing function registered by the target protocol, and determining a metadata extraction mode; if the feature matching extraction is carried out, an extraction rule engine corresponding to the target protocol identification is called to scan the load, and the metadata is extracted; if the syntax is extracted by syntax analysis, a syntax rule engine corresponding to the target protocol identification is called, the metadata is extracted according to the description analysis load of the syntax rule protocol item corresponding to the target protocol identification, the problems that in the prior art, the metadata extraction mode needs more codes to be written and the metadata extraction efficiency is low are solved, the method only processes the flow of part of the metadata by calling the extraction rule engine or the syntax rule engine, and the beneficial effects of improving the code quality and the processing efficiency of metadata extraction are achieved.
Example two
Fig. 2 is a flowchart of a protocol metadata extraction method according to a second embodiment of the present invention, and this embodiment further provides specific steps of parsing a load according to a description of a syntax rule protocol item corresponding to a target protocol identifier and extracting metadata, on the basis of the foregoing embodiment. As shown in fig. 2, the method includes:
s210, constructing an extraction rule engine and a grammar rule engine based on grammar rule protocol items based on the NUMA architecture.
In this embodiment, metadata is obtained according to whether a message of protocol interaction can be matched through a feature rule, and metadata extraction is divided into two implementation modes, namely feature matching extraction and syntax parsing extraction. Loading and analyzing a metadata extraction rule file of each protocol according to a feature matching extraction mode, creating an extraction rule engine corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the extraction rule engine; and synchronizing the relevant data structure of the extraction rule engine to a NUMA memory applied by each thread in a local NUMA node according to the characteristics of the NUMA architecture to create a thread-level extraction rule engine.
And aiming at a grammar parsing and extracting mode, a set of grammar rule description is defined by analyzing an organization mode of a protocol transmission load. Loading a metadata grammar rule description file of each protocol, creating grammar rule protocol items corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the grammar rule protocol items; and according to the NUMA architecture characteristics, synchronizing the data structure related to the grammar rule protocol item to a NUMA memory applied by each thread in the local NUMA node, and creating a thread-level grammar rule engine.
By adopting the technical scheme, the interaction logic and the organization mode of the protocol are separated, all interaction realization of the protocol is completed without compiling codes for each protocol, all flow of the protocol is processed, the extraction rule engine or the grammar rule engine is called to share the code logic, each protocol only needs to complete the logic code realization of some interaction parts according to the requirement of actually extracting metadata, and the code quality is greatly improved.
S220, receiving the data packet and identifying a target protocol corresponding to the data packet.
And S230, calling a protocol processing function registered by the target protocol, and judging whether the current data stream is recombined.
In this embodiment, if yes, step S240 is executed to set reassembly association information and reassemble the received data packet according to the reassembly association information; otherwise, step S250 is executed to determine the metadata extraction manner directly through the protocol processing function.
S240, the received data packet is recombined according to the set recombination association information, and a recombination ending callback function registered by the target protocol is called after the recombination is ended.
And S250, determining a metadata extraction mode of the target protocol.
In this embodiment, if the target protocol performs metadata extraction through the rule extraction engine, that is, a feature matching extraction manner is adopted, step S260 is executed; if the target protocol performs metadata extraction through the grammar rule engine, that is, a grammar parsing extraction manner is adopted, step S270 is executed.
And S260, if the feature matching extraction is carried out, calling an extraction rule engine corresponding to the target protocol identification to scan the load, and extracting the metadata.
And S270, if the syntax is extracted by syntax analysis, calling a syntax rule engine corresponding to the target protocol identifier, and extracting the metadata according to the description analysis load of the syntax rule protocol item corresponding to the target protocol identifier.
Wherein, a grammar rule protocol item includes a linked list of basic items, and the metadata required to be extracted by a protocol can be described by a plurality of basic items. The basic item is the combination of the condition item list and the field item linked list, and is used for indicating that a certain condition is met, and then the message load can be subjected to offset processing according to the field item linked list corresponding to the condition item.
Optionally, parsing the load according to the description of the syntax rule protocol item corresponding to the target protocol identifier, and extracting metadata includes: traversing a basic item linked list in the grammar rule protocol item; judging whether the load is consistent with the condition item description information in the basic item; and if the basic items are consistent, acquiring a field item linked list in the basic items to analyze the load, and extracting the metadata.
In this embodiment, as shown in fig. 3, after the syntax rule protocol item is obtained, each basic item is sequentially used as a target basic item for current processing by traversing a basic item linked list in the syntax rule protocol item. Acquiring condition item linked list information in a target basic item, judging whether the message load is consistent with the condition item description information, if not, processing the next basic item until all basic items are traversed, and ending the jump-out cycle; and if the field item is consistent with the field item, acquiring a field item linked list in the target basic item. And analyzing the message load by traversing the field item linked list, and extracting the metadata.
For example, assuming that a tls protocol interaction process involves a Client _ Hello packet, a Server Hello packet, a Certificate packet, and the like, different types of packets in the interaction process respectively correspond to different basic items in syntax protocol items of the tls protocol. And the condition item linked list in each basic item is used for judging whether the currently processed message load is a message needing to be processed. Assume that protocol version information, session ID information, cipher suite information, server _ name information, and the like to be extracted in the Client _ Hello packet correspond to respective fields, and may be represented by field entries in the basic entry.
By adopting the technical scheme, a universal grammar rule is defined aiming at binary streams with dependence among protocol metadata, whether the load needs to extract the metadata is judged according to condition items in the grammar rule, if so, the load is processed according to field items to extract the metadata, and the problem that the metadata can not be obtained from the binary streams with dependence among the metadata through characteristic rule matching is solved.
Optionally, the determining whether the load is consistent with the condition item description information in the basic item includes: extracting condition data from the load according to the offset information and the length information in the condition item description information; determining a matching condition according to numerical value information and operation information in the condition item description information, and if the condition data meets the matching condition, determining that the load is consistent with the condition item description information; otherwise, determining that the load is inconsistent with the condition item description information.
For example, it is assumed that in the description information of the syntax rule condition item, the offset information is 4 bytes, the length information is 2 bytes, the numerical value information is 100, and the operation information is to determine whether the data obtained by offset is equal to the numerical value information. Then, starting from the beginning position offset by 4 bytes of the packet payload, data of length 2 bytes is extracted. If the extracted data value is equal to 100, the load is determined to be consistent with the description information of the condition item, that is, the message load is considered to meet the condition, and the metadata extraction processing can be performed on the message load according to the field item linked list corresponding to the condition item. If the extracted data value is not equal to 100, determining that the load is inconsistent with the description information of the condition item, namely, the message load is considered not to meet the condition, and performing metadata extraction processing on the message load according to the field item linked list corresponding to the condition item.
By adopting the technical scheme, the offset and field of the protocol can be described according to the definition of the metadata grammar rule without developing experience of a worker by putting the parsing offset logic into the grammar rule, so that the development efficiency is greatly improved. Meanwhile, the problem that the program is hung due to the fact that offset value taking codes are not strict can be solved.
Optionally, the obtaining a field item linked list in the basic item to analyze the load and extract metadata includes: traversing a field item linked list in the basic item; processing the load according to the field name, the field type and the field length in the field item description information to obtain binary information corresponding to the field item; and processing the binary information according to the field result description and the field operation description in the field description information to obtain the metadata corresponding to the field item.
In this embodiment, as shown in fig. 3, field names, field types, and field length information used for extracting data in the field item description information are obtained, for example, "field _ name": session ID len ", which indicates that the field name of the current field is session ID length," value ": { "type": "direct" indicates that the current field is binary type in the message load, "van _ len": { "fix _ len": 8}, indicates that the current field occupies 8 bits, and the length is 1 byte. Other required information can be obtained for data extraction, such as "exist": type ": "fix" indicates that the current field appears in the current position of the message load.
And after binary information corresponding to the field items is obtained according to the information, processing the binary information according to field result description and field operation description in the field description information to obtain metadata corresponding to the field items. For example, "data _ decode": the { "func _ name": decode _ bin2hex "} refers to calling a decode _ bin2hex function to the extracted data and outputting according to a 16-system; wherein, different registration functions can be called according to different output scene requirements; actions { "func _ name": action _ set _ index "}, which indicates that the extracted field value is saved into an array and provided for other fields for use, represents the processing operation to be performed on the data result.
By adopting the technical scheme, the specified data is extracted from the load according to the description of the metadata in the field item description information, and the extracted data is subjected to data type conversion and other processing, so that the protocol metadata which can be directly used is obtained, and further processing of the protocol metadata during subsequent use is avoided.
The technical scheme of the embodiment of the invention is based on NUMA architecture, and an extraction rule engine and a grammar rule engine based on grammar rule protocol items are constructed; after receiving the data packet, identifying a target protocol corresponding to the data packet, and if the data packet is subjected to feature matching extraction, calling an extraction rule engine corresponding to a target protocol identifier to extract metadata; if the metadata extraction is performed for syntax analysis, a syntax rule engine is called, the load is analyzed according to the description of the corresponding syntax rule protocol item, the metadata is extracted, the problems that more codes need to be written and the metadata extraction efficiency is low in a metadata extraction mode in the prior art are solved, different protocols share the code logic of the extraction rule engine or the syntax rule engine through separating the interaction logic and the organization mode of the protocols, only part of traffic related to the metadata is processed, each protocol only needs to extract the metadata according to the actual requirement, the logic code implementation of the interaction part is completed, and the code quality and the metadata extraction efficiency are improved.
EXAMPLE III
Fig. 4 is a schematic structural diagram of a protocol metadata extraction apparatus according to a third embodiment of the present invention. As shown in fig. 4, the apparatus includes:
a protocol identification module 410, configured to receive a data packet and identify a target protocol corresponding to the data packet;
a protocol processing module 420, configured to execute a protocol processing function for invoking the target protocol registration, and determine a metadata extraction manner;
the feature matching module 430 is configured to, if feature matching extraction is performed, invoke an extraction rule engine corresponding to the target protocol identifier to scan a load, and extract metadata;
and the grammar parsing module 440 is used for calling a grammar rule engine corresponding to the target protocol identifier if the grammar parsing extraction is performed, and extracting the metadata according to the description parsing load of the grammar rule protocol item corresponding to the target protocol identifier.
Optionally, the apparatus further comprises: an extraction rules engine creation module for executing, prior to said invoking an extraction rules engine scan load corresponding to the target protocol identification, extracting metadata,
loading and analyzing the metadata extraction rule files of all the protocols, creating extraction rule engines corresponding to all the protocol identifications, and establishing a mapping relation between the protocol identifications and the extraction rule engines;
And synchronizing the extraction rule engine and the mapping relation between the protocol identifier and the extraction rule engine to each non-uniform memory access NUMA node.
Optionally, the apparatus further comprises: a grammar rule engine creating module for executing, before the calling of the grammar rule engine corresponding to the target protocol identification, the parsing of the load according to the description of the grammar rule protocol item corresponding to the target protocol identification, the extraction of the metadata,
loading and analyzing a metadata grammar rule description file of each protocol, creating grammar rule protocol items corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the grammar rule protocol items;
and synchronizing the grammar rule protocol item and the mapping relation between the protocol identification and the grammar rule protocol item to each NUMA node.
Optionally, the syntax parsing module 440 includes:
the traversal module is used for executing traversal of a basic item linked list in the grammar rule protocol item;
the judging module is used for judging whether the load is consistent with the condition item description information in the basic item or not;
and the analysis module is used for acquiring a field item linked list in the basic item to analyze the load and extracting the metadata if the field item linked list is consistent with the basic item.
Optionally, the parsing module is configured to perform:
extracting condition data from the load according to the offset information and the length information in the condition item description information;
determining a matching condition according to numerical value information and operation information in the condition item description information, and if the condition data meets the matching condition, determining that the load is consistent with the condition item description information;
otherwise, determining that the load is inconsistent with the condition item description information.
Optionally, the parsing module is configured to perform:
traversing a field item linked list in the basic item;
processing the load according to the field name, the field type and the field length in the field item description information to obtain binary information corresponding to the field item;
and processing the binary information according to the field result description and the field operation description in the field description information to obtain the metadata corresponding to the field item.
Optionally, the protocol processing module 420 is configured to execute:
executing a protocol processing function registered by the target protocol, and judging whether the current data stream is recombined or not;
if yes, recombining the received data packet according to the set recombination association information;
and if the recombination is finished, calling a recombination finishing callback function registered by the target protocol, and determining the type of the engine used for extracting the metadata.
The protocol metadata extraction device provided by the embodiment of the invention can execute the protocol metadata extraction method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Example four
FIG. 5 illustrates a block diagram of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM)12, a Random Access Memory (RAM)13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM)12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 may also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to the bus 14.
A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. Processor 11 performs the various methods and processes described above, such as the protocol metadata extraction method.
In some embodiments, the protocol metadata extraction method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the protocol metadata extraction method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the protocol metadata extraction method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for extracting protocol metadata, comprising:
receiving a data packet and identifying a target protocol corresponding to the data packet;
calling a protocol processing function registered by the target protocol, and determining a metadata extraction mode;
if the feature matching extraction is carried out, an extraction rule engine corresponding to the target protocol identification is called to scan the load, and the metadata is extracted;
if the syntax is extracted by parsing, a syntax rule engine corresponding to the target protocol identification is called, and the metadata is extracted according to the description parsing load of the syntax rule protocol item corresponding to the target protocol identification.
2. The method of claim 1, wherein before the invoking an extraction rules engine corresponding to the target protocol identification scans the load to extract the metadata, further comprising:
loading and analyzing metadata extraction rule files of each protocol, creating extraction rule engines corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the extraction rule engines;
and synchronizing the extraction rule engine and the mapping relation between the protocol identifier and the extraction rule engine to each non-uniform memory access NUMA node.
3. The method of claim 1, before said invoking a grammar rule engine corresponding to the target protocol identification, parsing the load according to the description of the grammar rule protocol item corresponding to the target protocol identification, and extracting the metadata, further comprising:
loading and analyzing a metadata grammar rule description file of each protocol, creating grammar rule protocol items corresponding to each protocol identifier, and establishing a mapping relation between the protocol identifier and the grammar rule protocol items;
and synchronizing the grammar rule protocol item and the mapping relation between the protocol identification and the grammar rule protocol item to each NUMA node.
4. The method of claim 1, wherein parsing the payload according to the description of the grammar rule protocol item corresponding to the target protocol identification to extract metadata comprises:
traversing a basic item linked list in the grammar rule protocol item;
judging whether the load is consistent with the condition item description information in the basic item;
and if the basic items are consistent, acquiring a field item linked list in the basic items to analyze the load, and extracting the metadata.
5. The method of claim 4, wherein the determining whether the load is consistent with the condition item description information in the base item comprises:
extracting condition data from the load according to the offset information and the length information in the condition item description information;
determining a matching condition according to numerical value information and operation information in the condition item description information, and if the condition data meets the matching condition, determining that the load is consistent with the condition item description information;
otherwise, determining that the load is inconsistent with the condition item description information.
6. The method of claim 4, wherein the obtaining the linked list of field items in the base item to parse the load and extract the metadata comprises:
Traversing a field item linked list in the basic item;
processing the load according to the field name, the field type and the field length in the field item description information to obtain binary information corresponding to the field item;
and processing the binary information according to the field result description and the field operation description in the field description information to obtain the metadata corresponding to the field item.
7. The method of claim 1, wherein the calling the protocol processing function of the target protocol registration to determine the metadata extraction manner comprises:
executing a protocol processing function registered by the target protocol, and judging whether the current data stream is recombined or not;
if yes, recombining the received data packet according to the set recombination association information;
and if the recombination is finished, calling a recombination finishing callback function registered by the target protocol, and determining the type of the engine used for extracting the metadata.
8. A protocol metadata extraction apparatus, comprising:
the protocol identification module is used for receiving the data packet and identifying a target protocol corresponding to the data packet;
the protocol processing module is used for executing a protocol processing function for calling the target protocol registration and determining a metadata extraction mode;
The characteristic matching module is used for calling an extraction rule engine corresponding to the target protocol identifier to scan the load and extracting the metadata if the characteristic matching extraction is carried out;
and the grammar parsing module is used for calling a grammar rule engine corresponding to the target protocol identifier if the grammar parsing extraction is carried out, and extracting the metadata according to the description parsing load of the grammar rule protocol item corresponding to the target protocol identifier.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the protocol metadata extraction method of any one of claims 1-7.
10. A computer-readable storage medium having stored thereon computer instructions for causing a processor to execute the method for extracting protocol metadata according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210393134.9A CN114760369B (en) | 2022-04-14 | 2022-04-14 | Protocol metadata extraction method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210393134.9A CN114760369B (en) | 2022-04-14 | 2022-04-14 | Protocol metadata extraction method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114760369A true CN114760369A (en) | 2022-07-15 |
| CN114760369B CN114760369B (en) | 2023-12-19 |
Family
ID=82331053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210393134.9A Active CN114760369B (en) | 2022-04-14 | 2022-04-14 | Protocol metadata extraction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114760369B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115314570A (en) * | 2022-10-12 | 2022-11-08 | 深圳华锐分布式技术股份有限公司 | Data issuing method, device, equipment and medium based on protocol development framework |
| CN116796782A (en) * | 2023-05-04 | 2023-09-22 | 北京百度网讯科技有限公司 | Two-dimensional code generation method, two-dimensional code identification device, two-dimensional code generation equipment and storage medium |
| WO2024086973A1 (en) * | 2022-10-24 | 2024-05-02 | 华为技术有限公司 | Data processing method and apparatus, and intelligent driving device |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104038389A (en) * | 2014-06-19 | 2014-09-10 | 高长喜 | Multiple application protocol identification method and device |
| US20150324947A1 (en) * | 2014-03-13 | 2015-11-12 | Verance Corporation | Metadata acquisition using embedded watermarks |
| US20180102997A1 (en) * | 2016-10-06 | 2018-04-12 | Sap Se | Payload description for computer messaging |
| US20180287947A1 (en) * | 2016-01-07 | 2018-10-04 | Trend Micro Incorporated | Metadata extraction |
| US20190018870A1 (en) * | 2017-07-11 | 2019-01-17 | International Business Machines Corporation | Custom metadata extraction across a heterogeneous storage system environment |
| US20190147103A1 (en) * | 2017-11-13 | 2019-05-16 | Accenture Global Solutions Limited | Automatic Hierarchical Classification and Metadata Identification of Document Using Machine Learning and Fuzzy Matching |
| CN110336798A (en) * | 2019-06-19 | 2019-10-15 | 南京中新赛克科技有限责任公司 | Message matching filtering method and device based on DPI |
| CN111580931A (en) * | 2020-05-10 | 2020-08-25 | 江苏省互联网行业管理服务中心 | Matching rule engine supporting combined expression of multiple protocol variables |
| CN112511551A (en) * | 2020-12-08 | 2021-03-16 | 中国船舶重工集团公司第七一六研究所 | Communication application layer protocol analysis method and system for multiple types of data streams |
| CN112751845A (en) * | 2020-12-28 | 2021-05-04 | 北京恒光信息技术股份有限公司 | Network protocol analysis method, system and device |
| CN112860727A (en) * | 2021-02-20 | 2021-05-28 | 平安科技（深圳）有限公司 | Data query method, device, equipment and medium based on big data query engine |
| CN112887280A (en) * | 2021-01-13 | 2021-06-01 | 中国人民解放军国防科技大学 | Network protocol metadata extraction system and method based on automaton |
| US20210194894A1 (en) * | 2019-12-23 | 2021-06-24 | Cisco Technology, Inc. | Packet metadata capture in a software-defined network |
-
2022
- 2022-04-14 CN CN202210393134.9A patent/CN114760369B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150324947A1 (en) * | 2014-03-13 | 2015-11-12 | Verance Corporation | Metadata acquisition using embedded watermarks |
| CN104038389A (en) * | 2014-06-19 | 2014-09-10 | 高长喜 | Multiple application protocol identification method and device |
| US20180287947A1 (en) * | 2016-01-07 | 2018-10-04 | Trend Micro Incorporated | Metadata extraction |
| US20180102997A1 (en) * | 2016-10-06 | 2018-04-12 | Sap Se | Payload description for computer messaging |
| US20190018870A1 (en) * | 2017-07-11 | 2019-01-17 | International Business Machines Corporation | Custom metadata extraction across a heterogeneous storage system environment |
| US20190147103A1 (en) * | 2017-11-13 | 2019-05-16 | Accenture Global Solutions Limited | Automatic Hierarchical Classification and Metadata Identification of Document Using Machine Learning and Fuzzy Matching |
| CN110336798A (en) * | 2019-06-19 | 2019-10-15 | 南京中新赛克科技有限责任公司 | Message matching filtering method and device based on DPI |
| US20210194894A1 (en) * | 2019-12-23 | 2021-06-24 | Cisco Technology, Inc. | Packet metadata capture in a software-defined network |
| CN111580931A (en) * | 2020-05-10 | 2020-08-25 | 江苏省互联网行业管理服务中心 | Matching rule engine supporting combined expression of multiple protocol variables |
| CN112511551A (en) * | 2020-12-08 | 2021-03-16 | 中国船舶重工集团公司第七一六研究所 | Communication application layer protocol analysis method and system for multiple types of data streams |
| CN112751845A (en) * | 2020-12-28 | 2021-05-04 | 北京恒光信息技术股份有限公司 | Network protocol analysis method, system and device |
| CN112887280A (en) * | 2021-01-13 | 2021-06-01 | 中国人民解放军国防科技大学 | Network protocol metadata extraction system and method based on automaton |
| CN112860727A (en) * | 2021-02-20 | 2021-05-28 | 平安科技（深圳）有限公司 | Data query method, device, equipment and medium based on big data query engine |
Non-Patent Citations (1)
| Title |
|---|
| 高妮;周明全;耿国华;贺毅岳;: "网络科技资源平台中元数据的设计与实现", 计算机工程与应用, no. 25 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115314570A (en) * | 2022-10-12 | 2022-11-08 | 深圳华锐分布式技术股份有限公司 | Data issuing method, device, equipment and medium based on protocol development framework |
| CN115314570B (en) * | 2022-10-12 | 2023-01-03 | 深圳华锐分布式技术股份有限公司 | Data issuing method, device, equipment and medium based on protocol development framework |
| WO2024086973A1 (en) * | 2022-10-24 | 2024-05-02 | 华为技术有限公司 | Data processing method and apparatus, and intelligent driving device |
| CN116796782A (en) * | 2023-05-04 | 2023-09-22 | 北京百度网讯科技有限公司 | Two-dimensional code generation method, two-dimensional code identification device, two-dimensional code generation equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114760369B (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114760369B (en) | Protocol metadata extraction method, device, equipment and storage medium | |
| CN112154420A (en) | Automatic intelligent cloud service testing tool | |
| CN113238912B (en) | Aggregation processing method for network security log data | |
| US11681606B2 (en) | Automatic configuration of logging infrastructure for software deployments using source code | |
| CN115567589B (en) | Compression transmission method, device and equipment of JSON data and storage medium | |
| CN110727417A (en) | Data processing method and device | |
| CN111078573A (en) | Test message generation method and device | |
| CN112883088B (en) | Data processing method, device, equipment and storage medium | |
| CN116126719A (en) | Interface testing method and device, electronic equipment and storage medium | |
| CN112130860B (en) | JSON object analysis method and device, electronic equipment and storage medium | |
| CN110896391B (en) | Message processing method and device | |
| CN114374745A (en) | Protocol format processing method and system | |
| CN113779018A (en) | Data processing method and device | |
| CN116827411B (en) | Load data analysis method and device, electronic equipment and storage medium | |
| CN114760256B (en) | Data processing method, device, equipment and storage medium | |
| CN113132480B (en) | Data transmission method, device and system | |
| CN107798060B (en) | Real-time streaming data processing application software feature recognition method | |
| CN109032724B (en) | Method, device, terminal and storage medium for processing user-defined bullet screen message | |
| CN117056222A (en) | Interface test file generation method and device, electronic equipment and storage medium | |
| CN117149633A (en) | Script processing method, script processing device, electronic equipment and storage medium | |
| CN115114890A (en) | Data analysis method and device, electronic equipment and readable medium | |
| CN117979210A (en) | Audio transmission method, device, electronic equipment and storage medium | |
| CN116862388A (en) | Production method, device, software material clearing end, continuous integration end and medium | |
| CN115859300A (en) | Vulnerability detection method and device, electronic equipment and storage medium | |
| CN111291299A (en) | Method for directly obtaining local command execution result and local server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |