CN114610958B - Processing method and device of transmission resources and electronic equipment - Google Patents
Processing method and device of transmission resources and electronic equipment Download PDFInfo
- Publication number
- CN114610958B CN114610958B CN202210500274.1A CN202210500274A CN114610958B CN 114610958 B CN114610958 B CN 114610958B CN 202210500274 A CN202210500274 A CN 202210500274A CN 114610958 B CN114610958 B CN 114610958B
- Authority
- CN
- China
- Prior art keywords
- transmission resources
- source heterogeneous
- matching
- group
- keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
- G06F16/90344—Query processing by using string matching techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a processing method and a processing device of transmission resources and electronic equipment, wherein the method comprises the following steps: receiving at least two groups of multi-source heterogeneous transmission resources; performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources; and after carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission. The embodiment of the invention realizes high-speed and accurate multi-mode matching of transmission resources, and improves the matching precision and reduces the cost while keeping the power consumption at a lower level through two-stage filtering matching.
Description
Technical Field
The present invention relates to the field of computer networks and information processing technologies, and in particular, to a method and an apparatus for processing transmission resources, and an electronic device.
Background
The multi-source heterogeneous real-time data multi-mode matching is the first step of transmission resource acquisition and analysis and is also a crucial step. The transmission rate of the backbone network reaches 40Gpbs/100Gbps at present, and with the development of technology, networks with Tbps will appear in the future. How to perform real-time and accurate transmission resource acquisition and analysis at such a high rate without affecting the existing transmission service in the network is a difficult problem in the field of information transmission systems.
In order to solve the problem of fast matching, an AC (Aho-coral) multimode matching algorithm is proposed, which is mainly used in a network intrusion detection system, and since each jump of an automaton can only process one character, and the jump is too frequent, an external memory needs to be constantly accessed to read corresponding jump information, which causes a great access delay and seriously affects the improvement of matching speed.
In order to further improve the efficiency of the AC multi-mode matching algorithm, an AC multi-mode matching algorithm-based improved K-step state machine algorithm is provided. At present, the software is simple and common to implement, but the data real-time processing speed can only reach 100 Kbps.
With the coming of the multi-source heterogeneous data era, more and more data are processed, data flow is larger and larger, and a matching mode of software implementation is long-felt. A pattern matching engine implemented by hardware is also widely researched at present, and mainly includes multi-mode matching based on an FPGA (Field Programmable Gate Array), but because resources of the FPGA are limited, it is difficult to store too many patterns, and it is unable to cope with complex scenes, and extra ROM (Read-Only Memory) or SDRAM (synchronous dynamic random access Memory) support must be added, thereby reducing acquisition and analysis speed and real-time performance of mass data. In order to deal with the problem, multi-mode matching based on the FPGA and the TCAM is proposed, and the TCAM (ternary content addressable memory) provides mode storage and parallel access, so that the speed of mode matching is improved. In order to solve the problem of overlarge power consumption of the TCAM, a Bloom filter (Bloom filter) + FPGA matching mode is generally adopted, but the Bloom filter has the defect of false alarm in matching, so that the matching precision is reduced.
Disclosure of Invention
The invention provides a processing method and device of transmission resources and electronic equipment. The multi-mode matching of high-speed and accurate transmission resources is realized, and the matching precision is improved and the cost is reduced while the power consumption is kept at a lower level through two-stage filtering matching.
To solve the above technical problem, an embodiment of the present invention provides the following solutions:
a processing method of transmission resources is applied to a pattern matching engine, and the method comprises the following steps:
receiving at least two groups of multi-source heterogeneous transmission resources;
performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources;
and after carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission.
Optionally, the filtering and matching at least two levels on each group of multi-source heterogeneous transmission resources to obtain a target transmission resource corresponding to each group of multi-source heterogeneous transmission resources includes:
performing first-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain a first matching result;
and according to the first matching result, performing second-stage filtering matching on the multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources.
Optionally, the performing a first-stage filtering matching on each group of the multi-source heterogeneous transmission resources to obtain a first matching result includes:
analyzing a first multi-source heterogeneous transmission resource encapsulation type which is larger than a first preset data length and smaller than a second preset data length in the multi-source heterogeneous transmission resources to obtain a multi-source heterogeneous data message; the first preset data length is smaller than the second preset data length;
extracting data fields of the data message to obtain a first matching result; the first matching result comprises at least one keyword.
Optionally, performing second-stage filtering matching on the multi-source heterogeneous transmission resources according to the first matching result to obtain a target transmission resource corresponding to each group of the multi-source heterogeneous transmission resources, including:
preprocessing at least one keyword in the first matching result through a secondary filter to obtain an effective keyword;
carrying out Hash compression processing on the effective keywords to obtain compressed keywords;
and searching in a preset memory according to the compressed keyword to obtain a target transmission resource corresponding to each group of multi-source heterogeneous transmission resources.
Optionally, preprocessing at least one keyword in the first matching result through a secondary filter to obtain an effective keyword, including:
screening at least one keyword in the first matching result to obtain a prefix subset;
according to the prefix subset, performing parallel operation on the prefix of the at least one keyword through a secondary filter to obtain a matching vector;
and determining effective keywords according to the matching vectors.
Optionally, performing hash compression processing on the valid keyword to obtain a compressed keyword, including:
carrying out Hash compression processing on the effective keywords to obtain a target address;
and obtaining the storage address of the table entry in a preset storage according to the target address to obtain the compressed keyword.
Optionally, after performing protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission, where the protocol conversion processing includes:
performing service control on the target transmission resource to obtain a service control result;
combining the service control result with the multi-source heterogeneous transmission resource to obtain a combined processing result;
performing protocol conversion processing on the combined processing result to obtain a resource to be transmitted;
and sending the resource to be transmitted to a high-speed channel for transmission.
The invention also provides a processing device of transmission resources, comprising:
the interface adaptation module is used for receiving at least two groups of multi-source heterogeneous transmission resources;
the filtering module is used for performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources;
and the protocol conversion processing module is used for carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources and then sending the target transmission resources and the multi-source heterogeneous transmission resources to a high-speed channel for transmission.
The invention also provides an electronic device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the processing method of the transmission resource as described above.
The present invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the processing method of a transmission resource as described above.
The scheme of the invention at least comprises the following beneficial effects:
according to the scheme, at least two groups of multi-source heterogeneous transmission resources are received; performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources; and after carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission. The scheme of the invention realizes high-speed and accurate multi-mode matching of transmission resources, and improves the matching precision and reduces the cost while keeping the power consumption at a lower level through two-stage filtering matching.
Drawings
Fig. 1 is a flowchart illustrating a method for processing transmission resources according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the architecture of a chip-level pattern matching engine CPME according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a processing apparatus for transmission resources according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
As shown in fig. 1, the present invention provides a method for processing transmission resources, which is applied to a pattern matching engine, and the method includes:
and step 13, carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, and sending the target transmission resources and the multi-source heterogeneous transmission resources to a high-speed channel for transmission.
In this embodiment, the Pattern Matching Engine is preferably a Chip-level Pattern Matching Engine (CPME), and the Pattern Matching Engine is configured to perform at least two-level filtering Matching on transmission resources; receiving at least two groups of multi-source heterogeneous transmission resources, performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources through a pattern matching engine to obtain corresponding target transmission resources, and transmitting each group of multi-source heterogeneous transmission resources and the corresponding target transmission resources to a high-speed channel for transmission after performing protocol conversion processing on each group of multi-source heterogeneous transmission resources and the corresponding target transmission resources; the multi-mode matching of high-speed and accurate transmission resources is realized, and the matching precision is improved and the cost is reduced while the power consumption is kept at a lower level through two-stage filtering matching.
It is worth mentioning that the processing processes of each group of multi-source heterogeneous transmission resources are parallel, and when the number of groups of multi-source heterogeneous transmission resources is larger, the speed of concurrent processing of the multi-source heterogeneous transmission resources is higher, so that support can be provided for software definition matching.
In an alternative embodiment of the present invention, step 12 includes:
step 121, performing a first-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain a first matching result;
and step 122, performing second-stage filtering matching on the multi-source heterogeneous transmission resources according to the first matching result to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources.
In the embodiment, each group of multi-source heterogeneous transmission resources is subjected to at least two-stage filtering matching through a pattern matching engine to obtain corresponding target transmission resources; wherein, the matching fails when carrying out the filtration of first order and matches, then no longer carries out the filtration matching of second order, has reduced the execution number of times of the filtration matching of second order like this to reduced the consumption, simultaneously, the matching succeeds when carrying out the filtration matching of first order, carries out the filtration matching of second order, can eliminate the false positive of the filtration matching of first order, thereby guarantee the precision of matching.
In an alternative embodiment of the present invention, step 121 includes:
step 1211, analyzing a first multi-source heterogeneous transmission resource encapsulation type, which is greater than a first preset data length and smaller than a second preset data length, in the multi-source heterogeneous transmission resources to obtain a multi-source heterogeneous data message; the first preset data length is smaller than the second preset data length;
step 1212, performing data field extraction on the data message to obtain a first matching result; the first matching result comprises at least one keyword.
In this embodiment, the encapsulation type of a first multi-source heterogeneous transmission resource which is greater than a first preset data length and less than a second preset data length in each group of multi-source heterogeneous transmission resources is analyzed, a second multi-source heterogeneous transmission resource which is less than the first preset data length or greater than the second preset data length in the remaining multi-source heterogeneous transmission resources is sent to a configuration layer, and the configuration layer determines deletion or forwarding of the second multi-source heterogeneous transmission resource;
firstly, judging the encapsulation type of a first multi-source heterogeneous transmission resource, wherein the encapsulation type comprises the following steps:
PPP (Point to Point Protocol) encapsulation;
IP (Internet Protocol ) packet encapsulation;
non-IP packet encapsulation;
MPLS (Multi-Protocol Label Switching) layer 1, 2, 3, 4 encapsulation.
Analyzing the packaging type of the first multi-source heterogeneous transmission resource to obtain a multi-source heterogeneous data message;
further, a Protocol type of data in the data message is determined, where the Protocol type includes a Transmission Control Protocol (TCP) and a User Datagram Protocol (UDP), at least one keyword in the data message is extracted according to the Protocol type, and the first matching result includes the at least one keyword.
In an alternative embodiment of the present invention, step 122 includes:
1221, preprocessing at least one keyword in the first matching result through a secondary filter to obtain an effective keyword;
step 1222, performing hash compression processing on the effective keywords to obtain compressed keywords;
and 1223, searching in a preset memory according to the compressed keyword to obtain a target transmission resource corresponding to each group of multi-source heterogeneous transmission resources.
In this embodiment, the secondary filter is preferably a bloom filter (Bloomfilter), and performs preprocessing on at least one keyword in the first matching result to obtain an effective keyword, where the preprocessing is to screen the keyword according to a prefix of the keyword; further, hash compression processing is carried out on the effective keywords, and the compressed keywords are obtained by reducing the lengths of the effective keywords, so that the matching speed is improved; according to the compressed keywords, searching in a preset Memory, finding out a corresponding matching table entry of the target keywords in the preset Memory, taking the position of the corresponding table entry as an address index RAM (Random Access Memory), further obtaining the offset of the compressed keywords, judging whether the offset is in a preset characteristic interval through a comparator, and obtaining a target transmission resource of which the judgment result is that the offset is in the preset characteristic interval;
the preset Memory is preferably a TCAM (Ternary Content Addressable Memory);
the search processing in the preset memory ensures that the keywords can be transmitted normally and accurately, and meanwhile, the matching is fast and accurate.
In an alternative embodiment of the present invention, step 1221 includes:
12211, screening at least one keyword in the first matching result to obtain a prefix subset;
step 12212, according to the prefix subset, performing parallel operation on the prefix of the at least one keyword through a secondary filter to obtain a matching vector;
and 12213, determining effective keywords according to the matching vectors.
In this embodiment, at least one keyword in the first matching result is judged and screened, and a prefix subset where all keywords are located is screened, where the screening condition is preferably the length of the prefix of the keyword; it should be noted that, because there is at least one keyword, the number of prefixes corresponding to the keyword corresponds to the number of keywords, that is, at least one keyword corresponds to at least one prefix, the prefix subset includes at least one prefix, parallel operation is performed on at least one prefix, the parallel operation is preferably hash calculation, prefixes whose calculation results meet preset conditions are marked, if a matching vector of the prefixes is 0, the keywords corresponding to the prefixes are invalid, and if the matching vector of the prefixes is not 0, the keywords corresponding to the prefixes are valid, and the keywords are determined to be valid keywords.
In an alternative embodiment of the present invention, step 1222 includes:
12221, performing hash compression processing on the effective keywords to obtain a target address;
step 12222, according to the target address, obtaining a storage address of the table entry in a preset storage, and obtaining a compressed keyword.
In the embodiment, the hash compression processing is performed on the effective keyword to obtain the target address of the effective keyword, and then the storage address of the table entry of the effective keyword is found in the preset memory to obtain the compressed keyword, so that the matching speed of the transmission resource is greatly improved; wherein, the HASH compression processing is preferably performed by a HASH-TCAM algorithm; through Hash compression processing, a long mode of the data message is compressed into a shorter mode, so that the throughput rate is improved, wherein the long mode refers to the mode string length being larger than the input width of a preset memory.
In an alternative embodiment of the present invention, step 13 includes:
step 131, performing service control on the target transmission resource to obtain a service control result;
step 132, combining the service control result with the multi-source heterogeneous transmission resource to obtain a combined processing result;
step 133, performing protocol conversion processing on the combining processing result to obtain a resource to be transmitted;
and step 134, sending the resource to be transmitted to a high-speed channel for transmission.
In this embodiment, according to different services, service control is performed on a target transmission resource to obtain a service control result, the service control result and a multi-source heterogeneous transmission resource are combined and subjected to protocol conversion to obtain a resource to be transmitted, where the resource to be transmitted is data to be transmitted through a high-speed channel, and the two-stage filtering matching in step 12 is combined, so that fast and ready matching and accurate and fast transmission can be achieved.
As shown in fig. 2, in a specific embodiment, the chip-level pattern matching engine CPME includes: the system comprises a long and short packet filtering module, an analysis module, a keyword extraction module, a BF (Bloomfilter) engine filtering module, a keyword compression module and a table look-up module;
the method comprises the steps that two groups of multi-source heterogeneous transmission resources are received through an interface adaptation module, the multi-source heterogeneous transmission resources are input into a long and short packet filtering module, and the long and short packet filtering module is used for screening the multi-source heterogeneous transmission resources to obtain a first multi-source heterogeneous transmission resource to be analyzed, wherein the length of the first multi-source heterogeneous transmission resource is larger than a first preset data length and smaller than a second preset data length;
inputting the first multi-source heterogeneous transmission resource into an analysis module, wherein the analysis module is used for analyzing the input first multi-source heterogeneous transmission resource to obtain a multi-source heterogeneous data message;
inputting the multi-source heterogeneous data message into a keyword extraction module, wherein the keyword extraction module is used for extracting data fields of the data message to obtain a first matching result;
inputting the first matching result into a BF engine filtering module, wherein the BF engine filtering module is used for preprocessing at least one keyword in the first matching result through a secondary filter to obtain an effective keyword;
inputting effective keywords into a keyword compression module, wherein the keyword compression module is used for carrying out Hash compression processing on the effective keywords to obtain compressed keywords;
inputting the compressed keywords into a table look-up module, wherein the table look-up module is used for carrying out look-up processing in a preset memory to obtain target transmission resources corresponding to the multi-source heterogeneous transmission resources;
inputting the target transmission resource into a service control module, wherein the service control module is used for carrying out service control on the target transmission resource to obtain a service control result;
combining the service control result and the multi-source heterogeneous transmission resource through a combining module, further performing protocol conversion processing through a corresponding protocol conversion module 1, and finally forwarding the resource to be transmitted obtained by the protocol conversion processing through a high-speed channel transmission module;
the high-speed channel transmission module can input the resource to be transmitted into the protocol conversion module 2, perform the second protocol conversion processing, decapsulate the resource to be transmitted after the second protocol conversion processing through the decapsulation module, and send the decapsulated resource through the interface adaptation module.
In the embodiment of the invention, at least two groups of multi-source heterogeneous transmission resources are received; performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources; and after carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission. The scheme of the invention realizes high-speed and accurate multi-mode matching of transmission resources, and improves the matching precision and reduces the cost while keeping the power consumption at a lower level through two-stage filtering matching.
As shown in fig. 3, the present invention further provides a device 30 for processing transmission resources, where the device 30 includes:
the interface adaptation module 31 is configured to receive at least two sets of multi-source heterogeneous transmission resources;
the filtering module 32 is configured to perform at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain a target transmission resource corresponding to each group of multi-source heterogeneous transmission resources;
and the protocol conversion processing module 33 is configured to perform protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, and then send the target transmission resources and the multi-source heterogeneous transmission resources to a high-speed channel for transmission.
Optionally, the filtering module 32 includes:
the first filtering submodule is used for carrying out first-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain a first matching result;
and the second filtering submodule is used for carrying out second-stage filtering matching on the multi-source heterogeneous transmission resources according to the first matching result to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources.
Optionally, the first filtering submodule includes:
the first filtering unit is used for analyzing a first multi-source heterogeneous transmission resource packaging type which is larger than a first preset data length and smaller than a second preset data length in the multi-source heterogeneous transmission resources to obtain a multi-source heterogeneous data message; the first preset data length is smaller than the second preset data length;
the second filtering unit is used for extracting data fields of the data message to obtain a first matching result; the first matching result comprises at least one keyword.
Optionally, the second filtering submodule includes:
the third filtering unit is used for preprocessing at least one keyword in the first matching result through a secondary filter to obtain an effective keyword;
the fourth filtering unit is used for carrying out Hash compression processing on the effective keywords to obtain compressed keywords;
and the fifth filtering unit is used for searching and processing in a preset memory according to the compressed key words to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources.
Optionally, the third filtering subunit includes:
the first filtering subunit is configured to filter at least one keyword in the first matching result to obtain a prefix subset;
the second filtering subunit is used for performing parallel operation on the prefix of the at least one keyword through a secondary filter according to the prefix subset to obtain a matching vector;
and the third filtering subunit is used for determining effective keywords according to the matching vectors.
Optionally, the fourth filtering unit includes:
the fourth filtering subunit is used for performing hash compression processing on the effective keywords to obtain a target address;
and the fifth filtering subunit is used for obtaining the storage address of the table entry in a preset storage according to the target address to obtain the compressed keyword.
Optionally, the protocol conversion processing module 33 includes:
the first protocol conversion processing submodule is used for carrying out service control on the target transmission resource to obtain a service control result;
the second protocol conversion processing submodule is used for combining the service control result with the multi-source heterogeneous transmission resource to obtain a combined processing result;
a third protocol conversion processing sub-module, configured to perform protocol conversion processing on the combining processing result to obtain a resource to be transmitted;
and the fourth protocol conversion processing submodule is used for sending the resource to be transmitted to a high-speed channel for transmission.
It should be noted that the apparatus is an apparatus corresponding to the method, and all implementation manners in the method embodiments are applicable to the embodiment of the apparatus, and the same technical effects can be achieved.
Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a program or an instruction stored on the memory and executable on the processor, where the program or the instruction, when executed by the processor, implements the steps of the processing method of the transmission resource as described above.
All the implementation manners in the above method embodiments are applicable to the embodiment of the electronic device, and the same technical effect can be achieved.
Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the processing method of transmission resources as described above.
All the implementation manners in the above method embodiments are applicable to the embodiment of the computer-readable storage medium, and the same technical effect can be achieved.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
Furthermore, it is to be noted that in the device and method of the invention, it is obvious that the individual components or steps can be decomposed and/or recombined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention. Also, the steps of performing the series of processes described above may naturally be performed chronologically in the order described, but need not necessarily be performed chronologically, and some steps may be performed in parallel or independently of each other. It will be understood by those skilled in the art that all or any of the steps or elements of the method and apparatus of the present invention may be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or any combination thereof, which can be implemented by those skilled in the art using their basic programming skills after reading the description of the present invention.
Thus, the objects of the invention may also be achieved by running a program or a set of programs on any computing device. The computing device may be a general purpose device as is well known. The object of the invention is thus also achieved solely by providing a program product comprising program code for implementing the method or the apparatus. That is, such a program product also constitutes the present invention, and a storage medium storing such a program product also constitutes the present invention. It is to be understood that the storage medium may be any known storage medium or any storage medium developed in the future. It is further noted that in the apparatus and method of the present invention, it is apparent that each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention. Also, the steps of executing the series of processes described above may naturally be executed chronologically in the order described, but need not necessarily be executed chronologically. Some steps may be performed in parallel or independently of each other.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (8)
1. A method for processing transmission resources, applied to a pattern matching engine, the method comprising:
receiving at least two groups of multi-source heterogeneous transmission resources;
performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources;
after carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission;
the method for filtering and matching each group of multi-source heterogeneous transmission resources at least in two stages to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources comprises the following steps:
performing first-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain a first matching result;
according to the first matching result, performing second-stage filtering matching on the multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources;
wherein, carry out the filtration matching of first level to each group the heterogeneous transmission resource of multisource, obtain first matching result, include:
analyzing a first multi-source heterogeneous transmission resource encapsulation type which is larger than a first preset data length and smaller than a second preset data length in the multi-source heterogeneous transmission resources to obtain a multi-source heterogeneous data message; the first preset data length is smaller than the second preset data length; the packaging types include: point-to-point protocol encapsulation; an internetworking protocol packet encapsulation; non-IP packet encapsulation; layer 1, 2, 3, 4 encapsulation of multi-protocol label switching;
extracting data fields of the data message to obtain a first matching result; the first matching result comprises at least one keyword;
extracting data fields of the data message to obtain a first matching result, wherein the extracting of the data fields of the data message comprises the following steps:
judging the protocol type of the data in the multi-source heterogeneous data message;
extracting at least one keyword in the data message according to the protocol type to obtain a first matching result comprising the at least one keyword; the protocol types include: transport control protocol and user datagram protocol.
2. The method of claim 1, wherein performing a second-stage filtering matching on the multi-source heterogeneous transmission resource according to the first matching result to obtain a target transmission resource corresponding to each group of the multi-source heterogeneous transmission resources comprises:
preprocessing at least one keyword in the first matching result through a secondary filter to obtain an effective keyword;
carrying out Hash compression processing on the effective keywords to obtain compressed keywords;
and searching in a preset memory according to the compressed keyword to obtain a target transmission resource corresponding to each group of multi-source heterogeneous transmission resources.
3. The method of claim 2, wherein preprocessing at least one keyword in the first matching result by a secondary filter to obtain a valid keyword comprises:
screening at least one keyword in the first matching result to obtain a prefix subset;
according to the prefix subset, performing parallel operation on the prefix of the at least one keyword through a secondary filter to obtain a matching vector;
and determining effective keywords according to the matching vectors.
4. The method according to claim 2, wherein performing hash compression on the valid keywords to obtain compressed keywords comprises:
carrying out Hash compression processing on the effective keywords to obtain a target address;
and obtaining the storage address of the table entry in a preset storage according to the target address to obtain the compressed keyword.
5. The method according to claim 1, wherein after performing protocol conversion on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources, sending the target transmission resources to a high-speed channel for transmission, the method includes:
performing service control on the target transmission resource to obtain a service control result;
combining the service control result with the multi-source heterogeneous transmission resource to obtain a combined processing result;
performing protocol conversion processing on the combined processing result to obtain a resource to be transmitted;
and sending the resource to be transmitted to a high-speed channel for transmission.
6. An apparatus for processing transmission resources, comprising:
the interface adaptation module is used for receiving at least two groups of multi-source heterogeneous transmission resources;
the filtering module is used for performing at least two-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources;
the protocol conversion processing module is used for carrying out protocol conversion processing on each group of target transmission resources and the multi-source heterogeneous transmission resources corresponding to the target transmission resources and then sending the target transmission resources and the multi-source heterogeneous transmission resources to a high-speed channel for transmission;
the method for filtering and matching each group of multi-source heterogeneous transmission resources at least at two stages to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources comprises the following steps:
performing first-stage filtering matching on each group of multi-source heterogeneous transmission resources to obtain a first matching result;
according to the first matching result, performing second-stage filtering matching on the multi-source heterogeneous transmission resources to obtain target transmission resources corresponding to each group of multi-source heterogeneous transmission resources;
wherein, carry out the filtration matching of first level to each group the heterogeneous transmission resource of multisource, obtain first matching result, include:
analyzing a first multi-source heterogeneous transmission resource encapsulation type which is larger than a first preset data length and smaller than a second preset data length in the multi-source heterogeneous transmission resources to obtain a multi-source heterogeneous data message; the first preset data length is smaller than the second preset data length;
extracting data fields of the data message to obtain a first matching result; the first matching result comprises at least one keyword;
the packaging types comprise: point-to-point protocol encapsulation; internetworking protocol packet encapsulation; non-IP packet encapsulation; layer 1, 2, 3, 4 encapsulation of multi-protocol label switching;
extracting data fields of the data message to obtain a first matching result, wherein the extracting of the data fields of the data message comprises the following steps:
judging the protocol type of the data in the multi-source heterogeneous data message;
extracting at least one keyword in the data message according to the protocol type to obtain a first matching result comprising the at least one keyword; the protocol types include: transport control protocol and user datagram protocol.
7. An electronic device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the processing method of the transmission resource according to any one of claims 1 to 5.
8. A computer-readable storage medium storing instructions which, when executed on a computer, cause the computer to perform the method of processing the transmission resource of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210500274.1A CN114610958B (en) | 2022-05-10 | 2022-05-10 | Processing method and device of transmission resources and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210500274.1A CN114610958B (en) | 2022-05-10 | 2022-05-10 | Processing method and device of transmission resources and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114610958A CN114610958A (en) | 2022-06-10 |
| CN114610958B true CN114610958B (en) | 2022-08-30 |
Family
ID=81869966
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210500274.1A Active CN114610958B (en) | 2022-05-10 | 2022-05-10 | Processing method and device of transmission resources and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114610958B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105122745A (en) * | 2013-02-27 | 2015-12-02 | 马维尔国际贸易有限公司 | Efficient longest prefix matching techniques for network devices |
| CN105515997A (en) * | 2015-12-07 | 2016-04-20 | 刘航天 | BF_TCAM (Bloom Filter-Ternary Content Addressable Memory)-based high-efficiency range matching method for realizing zero range expansion |
| CN110598129A (en) * | 2019-09-09 | 2019-12-20 | 河南科技大学 | Cross-social network user identity recognition method based on two-stage information entropy |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101848222B (en) * | 2010-05-28 | 2013-05-01 | 武汉烽火网络有限责任公司 | Inspection method and device of Internet deep packet |
| CN103593422B (en) * | 2013-11-01 | 2017-02-15 | 国云科技股份有限公司 | Virtual access management method of heterogeneous database |
| CN104866502B (en) * | 2014-02-25 | 2020-10-13 | 深圳市中兴微电子技术有限公司 | Data matching method and device |
| US10127282B2 (en) * | 2014-04-30 | 2018-11-13 | Hewlett Packard Enterprise Development Lp | Partitionable ternary content addressable memory (TCAM) for use with a bloom filter |
| CN110971601A (en) * | 2019-12-02 | 2020-04-07 | 邑客得（上海）信息技术有限公司 | Efficient network message transmission layer multi-level feature extraction method and system |
| CN113660701A (en) * | 2020-05-12 | 2021-11-16 | 维沃移动通信有限公司 | Information transmission method and device and electronic equipment |
| CN114218371A (en) * | 2021-12-17 | 2022-03-22 | 平安养老保险股份有限公司 | Multilevel directory name retrieval matching method, device, equipment and medium |
-
2022
- 2022-05-10 CN CN202210500274.1A patent/CN114610958B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105122745A (en) * | 2013-02-27 | 2015-12-02 | 马维尔国际贸易有限公司 | Efficient longest prefix matching techniques for network devices |
| CN105515997A (en) * | 2015-12-07 | 2016-04-20 | 刘航天 | BF_TCAM (Bloom Filter-Ternary Content Addressable Memory)-based high-efficiency range matching method for realizing zero range expansion |
| CN110598129A (en) * | 2019-09-09 | 2019-12-20 | 河南科技大学 | Cross-social network user identity recognition method based on two-stage information entropy |
Non-Patent Citations (2)
| Title |
|---|
| "一种基于Bloom-filter表项压缩的TCAM业务识别方法";陈正虎 等;《电子与信息学报》;20110930;第33卷(第9期);参见2213-2217页 * |
| 陈正虎 等."一种基于Bloom-filter表项压缩的TCAM业务识别方法".《电子与信息学报》.2011,第33卷(第9期), * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114610958A (en) | 2022-06-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9762544B2 (en) | Reverse NFA generation and processing | |
| US7949683B2 (en) | Method and apparatus for traversing a compressed deterministic finite automata (DFA) graph | |
| US8180803B2 (en) | Deterministic finite automata (DFA) graph compression | |
| US8572016B2 (en) | Match engine for detection of multi-pattern rules | |
| US20100153420A1 (en) | Dual-stage regular expression pattern matching method and system | |
| US20110016154A1 (en) | Profile-based and dictionary based graph caching | |
| US9589073B2 (en) | Systems and methods for keyword spotting using adaptive management of multiple pattern matching algorithms | |
| WO2011060377A1 (en) | Method and apparatus for real time identification and recording of artifacts | |
| CN109600385B (en) | Access control method and device | |
| CN112532642B (en) | Industrial control system network intrusion detection method based on improved Suricata engine | |
| CN109275045B (en) | DFI-based mobile terminal encrypted video advertisement traffic identification method | |
| CN114610958B (en) | Processing method and device of transmission resources and electronic equipment | |
| CN112054992B (en) | Malicious traffic identification method and device, electronic equipment and storage medium | |
| KR20140040120A (en) | Method and device for extracting data from a data stream travelling around an ip network | |
| CN110602059B (en) | Method for accurately restoring clear text length fingerprint of TLS protocol encrypted transmission data | |
| CN114946167A (en) | Message parsing method and device | |
| CN112187935B (en) | Information identification method and read-only memory | |
| CN112887280B (en) | Network protocol metadata extraction system and method based on automaton | |
| CN113836367A (en) | Character reverse matching method and device | |
| Park et al. | A lightweight software model for signature-based application-level traffic classification system | |
| CN115996203B (en) | Network traffic domain division method, device, equipment and storage medium | |
| CN113489622B (en) | Method, system, equipment and storage medium for extracting network equipment fingerprint | |
| CN114598616A (en) | Efficient mode matching method for solving real-time mass data | |
| CN116320086A (en) | TCAM message processing method, device and medium based on collaborative analysis | |
| CN113839937A (en) | Method and system for detecting unknown Trojan horse by using cross-session technology based on network flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |