CN110224891B

CN110224891B - Intelligent flow scheduling method and system based on DPI and shunt

Info

Publication number: CN110224891B
Application number: CN201910506832.3A
Authority: CN
Inventors: 周思朝
Original assignee: Wuhan Greenet Information Service Co Ltd
Current assignee: Wuhan Greenet Information Service Co Ltd
Priority date: 2019-06-12
Filing date: 2019-06-12
Publication date: 2021-02-19
Anticipated expiration: 2039-06-12
Also published as: CN110224891A

Abstract

The invention relates to the technical field of network flow distribution, in particular to an intelligent flow scheduling method and system based on DPI and a flow divider, wherein the method comprises the following steps: analyzing the broadband flow through the DPI to obtain user attribute information, and further establishing a corresponding relation between the IP information of the user and each piece of non-IP information; when the traffic of a target group needs to be filtered, converting the non-IP information of the target user into IP information based on the non-IP information of the target user and the corresponding relation; generating a filtering rule in real time according to the IP information of the target user, and issuing the filtering rule to the shunt; and the flow divider filters the BRAS flow according to the filtering rule and outputs the obtained target flow to the DPI device for flow analysis. When the IP of the target user is unknown or not fixed, the invention can dynamically generate the filtering rule, and then the required target flow is obtained by filtering, thereby saving the DPI hardware investment of the network operator.

Description

Intelligent flow scheduling method and system based on DPI and shunt

Technical Field

The invention relates to the technical field of network flow distribution, in particular to an intelligent flow scheduling method and system based on DPI and a flow divider.

Background

The wired broadband becomes an important carrier for realizing intelligent family and family informatization, and is a new profit growth point for operators. The average wired broadband flow of an operator in a medium city is about 10Tbps, currently, in the field of broadband network DPI, the mainstream DPI processing capacity in the industry is 100Gbps, if the 10Tbps flow needs to be processed in a full amount, at least 100 DPI devices with high performance of 100Gbps are needed, which brings huge cost investment to the operator, and the operator can only look forward in the face of huge flow treasures. At present, the speed of traffic increase far exceeds the moore law of a CPU, which brings unprecedented challenges to wired broadband service analysis, network quality, and service perception promotion: on one hand, the high increase of broadband flow and the very strict requirements of users on network quality and service experience are met, and how to put limited DPI resources into a specific target group intelligently, in real time and accurately is a problem which needs to be solved urgently in the whole DPI industry at present.

At present, a mainstream flow divider mainly depends on a filtering rule formed by IP quintuple (protocol, sip, sport, dip, dport) of a flow when the flow is divided, and may be a certain element or a combination of a plurality of elements in the IP quintuple, and the flow needs to be configured in advance in a flow divider system to be effective, which determines that the flow to be divided and output can only be the flow of the IP information known in advance. Therefore, the current flow distribution depends on the static filtering rule of the flow divider, the IP address information of the target user needs to be known in advance, and the target flow with unknown IP or unfixed IP cannot be filtered out; therefore, once the IP address of the target user changes, for example, in scenarios such as a DHCP lifetime of the IP address of the user overtime, a terminal device being disconnected, a restart, and a user being offline and acquiring an IP again, the target traffic cannot be accurately filtered. If filtering is not configured or filtering is performed according to a larger IP address field, the target traffic is not accurate enough, which causes excessive load and high cost to the back-end traffic analysis processing system.

In summary, when the current flow splitter splits flow, the flow needs to be split by means of a known IP quintuple and a static filtering rule configured in advance, and when the IP quintuple information is unknown or unfixed, the required flow cannot be effectively filtered, which brings great troubles to intelligent analysis of complex service scenes such as unfixed IP of a target user, multiple services, multiple products sharing an IP pool, and the like.

In view of the above, it is an urgent problem in the art to overcome the above-mentioned drawbacks of the prior art.

Disclosure of Invention

The technical problems to be solved by the invention are as follows:

when the traditional flow is shunted, the shunt needs to rely on the known IP quintuple and a static filtering rule to shunt, and when the IP quintuple information is unknown or unfixed, the needed flow can not be effectively filtered, which brings trouble to the intelligent analysis of complicated service scenes such as unfixed IP of a target user, multi-service, multi-product sharing IP pool and the like.

The invention achieves the above purpose by the following technical scheme:

in a first aspect, the present invention provides an intelligent traffic scheduling method based on DPI and a splitter, including:

analyzing the broadband flow through the DPI to obtain user attribute information in real time, and further establishing the corresponding relation between the IP information of the user and each piece of non-IP information in real time;

when the network traffic of a target group needs to be filtered, converting the non-IP information of the target user into IP information based on the non-IP information of the target user and the corresponding relation;

generating a filtering rule in real time according to the IP information of a target user, and issuing the filtering rule to a rear-end splitter in real time;

the flow divider performs flow filtration according to a filtration rule and outputs a target flow obtained by filtration to a DPI device for flow analysis;

the user attribute information comprises non-IP information and IP information, wherein the non-IP information comprises one or more items of a user broadband account, user MAC information, MAC information of access side equipment and IP information of the access side equipment.

Preferably, the analyzing the broadband traffic through the DPI to obtain the user attribute information in real time, and further establishing a corresponding relationship between the IP information of the user and each piece of non-IP information in real time, specifically includes:

AAA flow or portal flow is forwarded and filtered to a DPI device by utilizing a flow divider;

after the DPI device encodes and decodes the AAA traffic or the portal traffic, acquiring user attribute information in real time;

the DPI device inputs user attribute information into a dynamic database in real time, and then the dynamic database establishes and maintains a corresponding relation between IP information of a user and non-IP information;

and the dynamic database also receives user attribute information input by a human or third-party system.

Preferably, the storing of the user attribute information and the correspondence between the IP information and each piece of non-IP information in a dynamic database, and the converting of the non-IP information of the target user into the IP information based on the non-IP information of the target user and the correspondence specifically include:

inputting non-IP information corresponding to a target user to the dynamic database for query according to a target group to be analyzed;

according to the input non-IP information, IP information corresponding to the target user is inquired from the dynamic database in a correlation mode;

and after CIDR cluster analysis is carried out on the inquired IP information, the IP information is converted into a converged IP address segment so as to be output to a dynamic filtering rule generator at the rear end.

Preferably, the target group is a specific user, a specific device or a specific area;

when the network flow of a specific user needs to be analyzed, the non-IP information input into the dynamic database is specifically a user name, a user broadband account number or an MAC address bit of access side equipment;

when the network flow of a user under specific equipment needs to be analyzed, the non-IP information input into the dynamic database is specifically the OUI bit of the MAC address of a user home gateway manufacturer or user side dialing equipment;

when the network flow of a user in a specific area needs to be analyzed, the non-IP information input into the dynamic database is specifically the position information of the ONU or the OLT equipment; the position information is from a network resource management system of an operator and is imported into the dynamic database through an interface.

Preferably, the generating a filtering rule in real time according to the IP information of the target user, and issuing the filtering rule to the splitter at the back end in real time specifically includes:

inputting the converted IP address field information of the target user into a dynamic filtering rule generator;

the dynamic filtering rule generator generates filtering rules in real time according to the rule expression grammar of the rear-end flow divider and the input IP address field information;

and the dynamic filtering rule generator sends the filtering rules to a rear-end flow divider in real time through an RPC interface.

Preferably, the dynamic filtering rule generator generates the filtering rule in real time according to the rule expression syntax of the back-end splitter and the input IP address field information, specifically:

the dynamic filtering rule generator selects required grammar from grammar selection items of different shunt manufacturers according to the requirement so that the shunt can identify the received filtering rule;

and the dynamic filtering rule generator constructs an expression of the filtering rule based on the selected grammar and the input IP address field information of the target user, and further generates the filtering rule in real time.

Preferably, after the flow diverter performs flow filtration according to a filtration rule and outputs a target flow obtained by filtration to a DPI device for flow analysis, the method further includes:

and after the DPI device analyzes the target flow, feeding back the information obtained by analysis to the dynamic database so as to facilitate data comparison of the dynamic database, and verifying, correcting and expanding the user attribute information and the corresponding relation in the database according to the comparison result.

In a second aspect, the present invention provides an intelligent traffic scheduling system based on DPI and a splitter, which can be used to implement the intelligent traffic scheduling method based on DPI and a splitter in the first aspect, including a splitter, a DPI device, and an intelligent traffic scheduling device, where the intelligent traffic scheduling device includes a dynamic database and a dynamic filtering rule generator;

the dynamic database is used for establishing and maintaining the corresponding relation between the IP information of the user and each piece of non-IP information, is also used for converting the non-IP information of the target user into the IP information, and outputs the IP information to the dynamic filtering rule generator; the dynamic filtering rule generator is used for generating filtering rules in real time according to the IP information of the target user;

the shunt is used for receiving the filtering rule generated by the dynamic filtering rule generator, filtering out target flow according to the filtering rule and outputting the target flow to the DPI device; the DPI device is used for analyzing and processing the flow filtered by the shunt, acquiring user attribute information and generating a user XDR detailed list;

Preferably, the DPI device includes a first DPI analyzer and a second DPI analyzer;

the first DPI analyzer is respectively connected with the splitter and the dynamic database and is used for analyzing and processing AAA (authentication, authorization and accounting) flow or portal flow forwarded and filtered by the splitter so as to obtain user attribute information and inputting the user attribute information into the dynamic database in real time;

and the second DPI analyzer is connected to the rear end of the diverter and is used for analyzing and processing the target flow filtered by the diverter according to the filtering rule and generating a user XDR detailed list.

Preferably, the dynamic filtering rule generator comprises a grammar selection module and an expression construction module; the grammar selection module is connected to the rear end of the dynamic database and is used for selecting required grammars from grammar selection items of different shunt manufacturers and providing a standard criterion for the rule construction of the expression construction module;

the expression building module is connected to the front end of the flow divider and used for building an expression of the filtering rule according to the selected grammar in the grammar selecting module and the IP information of the target user.

The invention has the beneficial effects that:

according to the intelligent flow scheduling method and system provided by the invention, a flow scheduling device and a scheduling link are added between the shunting device and the DPI device, when the IP of a target user is unknown or unfixed, non-IP information of the user can be converted into dynamic IP information, a filtering rule is generated in real time and fed back to the shunt for filtering, and the shunt filters the IP to obtain the required target flow; the method has the characteristics of real-time generation of dynamic configuration rules, dynamic filtering, free shunting and the like, can flexibly define a target analysis group when facing flow treasury, can be used for complex service scenes such as unfixed IP of a target user, multiple services, shared IP pool of multiple products and the like, saves DPI hardware investment of a network operator, enables service perception analysis of a broadband network to be more diversified, lighter and more intelligent, and has positive promotion effect on the healthy development of the broadband network.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

Fig. 1 is a structural diagram of an intelligent traffic scheduling system based on a DPI and a splitter according to an embodiment of the present invention;

fig. 2 is a flowchart of an intelligent traffic scheduling method based on DPI and a splitter according to an embodiment of the present invention;

fig. 3 is a flowchart of a method for acquiring user attribute information and establishing a corresponding relationship according to an embodiment of the present invention;

fig. 4 is a diagram illustrating an example of signaling messages of related fields in an AAA message according to an embodiment of the present invention;

fig. 5 is a diagram illustrating an example of a signaling message of a relevant field in a portal message according to an embodiment of the present invention;

FIG. 6 is a schematic diagram illustrating a working process of a dynamic database according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a detailed table of user information in a dynamic database according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating field filling rules of a dynamic database according to an embodiment of the present invention;

fig. 9 is a flowchart of a method for converting non-IP information of a user into IP information according to an embodiment of the present invention;

FIG. 10 is a schematic diagram illustrating a query and translation workflow within a dynamic database according to an embodiment of the present invention;

fig. 11 is a diagram illustrating an example of signaling messages of related fields in an AAA message when querying and converting in a specific area according to an embodiment of the present invention;

fig. 12 is a flowchart of a method for generating and forwarding a filtering rule according to an embodiment of the present invention;

FIG. 13 is a diagram illustrating operation of a dynamic filter rule generator according to an embodiment of the present invention;

fig. 14 is an exemplary diagram of an RPC roaming communication mode when an HTTP/2 protocol is used to carry an RPC interface according to an embodiment of the present invention;

FIG. 15 is a diagram illustrating an operation process of another dynamic database according to an embodiment of the present invention;

FIG. 16 is a table illustrating a hardware cost comparison of an embodiment of the present invention to a conventional scheme;

fig. 17 is an architecture diagram of an intelligent traffic scheduling apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other. The invention will be described in detail below with reference to the figures and examples.

Example 1:

the embodiment of the invention provides an intelligent flow scheduling method based on DPI and a flow divider, which is suitable for service scenes in which target flow cannot be directly filtered through the flow divider, and effectively meets the flow analysis requirements of complex service scenes such as unfixed target IP, multiple services, multiple products sharing an IP pool and the like. The intelligent traffic scheduling system used in the present invention is shown in fig. 1, and mainly includes a splitter, a DPI device, and an intelligent traffic scheduling device, and the intelligent traffic scheduling device further includes a dynamic database and a dynamic filtering rule generator.

With reference to fig. 1 and fig. 2, an intelligent traffic scheduling method provided in an embodiment of the present invention includes:

and step 10, analyzing the broadband flow through the DPI to obtain user attribute information in real time, and further establishing a corresponding relation between the IP information of the user and each piece of non-IP information in real time.

The AAA flow or portal flow after being forwarded and filtered by the flow divider can be analyzed and processed through the DPI device, and then a plurality of items of user attribute information can be obtained to form one-to-one structured data; the user attribute information comprises non-IP information and IP information of a user, the non-IP information comprises one or more of a user broadband account number, user MAC information, MAC information of access side equipment and IP information of the access side equipment, and the access side equipment mainly refers to equipment which is closest to the user side, such as a home gateway, an optical modem or a wireless router; the IP information mainly refers to the IP address of the user. The DPI device can further input the acquired user attribute information into a dynamic database, and a corresponding relation between the IP information of the user and each piece of non-IP information is established in the dynamic database. The main characteristic of the dynamic database is that the stored data is data which changes instantly along with the development of time, and due to the variability of the IP at the user side, not only the real-time property but also the dynamic property are considered when the data is put into a database; the method is mainly used for storing various user attribute information and dynamically updating.

And 20, when the network traffic of the target group needs to be filtered, converting the non-IP information of the target user into IP information based on the non-IP information of the target user and the corresponding relation.

The step is mainly 'conversion from non-IP information of a user to IP information', and is the basis for realizing intelligent flow scheduling. In the big data analysis work of large-flow service scenes such as a metropolitan area network and the like, management or analysis personnel often only pay attention to the service condition of a part of target groups, at the moment, the non-IP information corresponding to target users can be input into the dynamic database by means of the dynamic database, and then the corresponding IP information can be inquired in a correlation mode according to the corresponding relation.

And step 30, generating a filtering rule in real time according to the IP information of the target user, and issuing the filtering rule to the rear-end splitter in real time.

The step is mainly 'generation of dynamic filtering rules', which is the core for realizing intelligent flow scheduling in the invention, and the premise is the conversion from non-IP information of a user to IP information. This is done here by means of a dynamic filter rule generator: the IP information of the target user is used as the input information of the dynamic filtering rule generator, the dynamic filtering rule generator generates the filtering rule in real time according to the input information, and the filtering rule is issued to the shunt through the negotiated RPC interface. In an alternative embodiment, if the traffic flow of a plurality of target user groups needs to be analyzed simultaneously, the filtering rules can be generated in batch by combining the IP information of each target user.

And step 40, the flow divider performs flow filtration according to the filtration rule, and outputs the filtered target flow to the DPI device for flow analysis.

The step is mainly 'free flow distribution' of the flow divider, and the filtering rule contains characteristic information of target flow and can be used for matching the target flow in the flow divider to form effective filtering. The method comprises the following specific steps: the flow divider can receive and load the filtering rule in real time through the RPC interface, the target flow corresponding to a target user can be obtained after the BRAS full flow is filtered according to the filtering rule, the target flow can be continuously output to the DPI device for flow analysis, and a set of self-produced self-used intelligent flow scheduling system is formed to meet the deployment requirements of various flow monitoring and analysis. When a plurality of target groups need to be analyzed, a plurality of target flows can be correspondingly output after filtering.

According to the intelligent flow scheduling method provided by the invention, the filtering of network flow is not limited by IP quintuple, when the IP of a target user is unknown or unfixed, the non-IP information of the user can be converted into IP information, a filtering rule is generated in real time according to the IP information and fed back to the flow divider to perform filtering, the flow divider filters the IP information to obtain the required target flow, the intelligent analysis of complex service scenes such as unfixed target IP, multi-service and multi-product sharing IP pools is effectively solved, and the DPI construction cost of a large-flow scene is reduced.

The following detailed description of the implementation of the steps of the present invention is made with reference to the accompanying drawings:

as shown in fig. 3, analyzing the broadband traffic through the DPI to obtain the user attribute information in real time, and further establishing the corresponding relationship between the IP information of the user and each piece of non-IP information in real time, that is, step 10, specifically includes:

and step 101, forwarding and filtering AAA traffic or portal traffic to a DPI device by using a flow divider.

Referring to fig. 1, the splitter filters the traffic to the AAA Server or the portal Server from the broadband full traffic; the IP address of the AAA server and the IP address of the portal server in province or city are both fixed IP and will not change within the planning time limit, so the splitter can directly filter through the IP address of the AAA server and the IP address of the portal server. The filtered flow is output to a first DPI analyzer in the DPI device, the flow can only process the upstream flow, the proportion of the upstream flow in the total flow is very small (generally, only about 0.1%), and therefore, the performance consumption of the DPI device is very small.

And step 102, the DPI device encodes and decodes the AAA traffic or the portal traffic, and then acquires user attribute information in real time.

With reference to fig. 1, a first DPI analyzer in the DPI device performs encoding and decoding processing on AAA traffic or portal traffic, so as to obtain multiple items of user attribute information in real time. The user attribute information in the AAA message is as follows: the Accounting-Request message of the AAA message carries information such as a user broadband account, a user side IP address, a user side MAC address and the like. The signaling message of the relevant field can refer to fig. 4, the User-Name is the User Name that needs to be authenticated, and may adopt a format of 'pure User Name @ domain Name' with domain Name, or a format of 'pure User Name' without domain Name; the Framed-IP-Address is the IP Address of the user.

Further, referring to fig. 5, the user attribute information in the portal message may be configured in a domain-divided WIFI manner in the public WIFI scenario, and after accessing the BRAS, the user first enters the domain before authentication and obtains an IP address. At this time, any HTTP message of the user can be jumped to a Portal page by the BRAS to carry out subsequent processes such as user authentication, charging and the like. If the IP address of the Portal server can be confirmed, after the user accesses the IP of the BRAS, the user attribute information can be transmitted to the Portal server in a URL (uniform resource locator) parameter transmission mode; for example, http:// gewifi. online. cq. cn & wlan user MAC = AAAA-BBBB-CCCC-DDDD & bas-IP =111.111.111.111& user-IP-address =222.222.222.222, 222, thereby using MAC + IP in combination with authentication information to locate the target user.

And 103, inputting the user attribute information into a dynamic database in real time by the DPI device, and further establishing and maintaining a corresponding relation between the IP information of the user and each piece of non-IP information by the dynamic database.

With continuing reference to fig. 1, the first DPI analyzer in the DPI device inputs the analyzed and obtained user attribute information into the dynamic database at the back end in real time, and in addition, the dynamic database can also expand to receive data input by a human or third-party system, so that the user attribute information can be enriched. And the dynamic database is updated according to the data input by the front-end DPI analyzer and the data input by the third-party system, so that the real-time performance and the newest performance of the user dynamic database are ensured. When the dynamic database is created, because the data volume of AAA and portal data is small compared with the data volume of service data, relational databases such as MYSQL and PostgreSQL can be used to ensure the efficiency of data storage.

In the embodiment of the invention, the dynamic database is mainly used for storing the user IP information which changes instantly along with the development of time and the corresponding relation between the IP information and each non-IP information; because of the variability of the user IP address, not only the real-time property but also the dynamic property need to be considered when the data is put in storage, namely when the information of a certain user is input, whether the history record of the user exists in the dynamic database needs to be checked, if so, the history record is deleted and the latest information record is inserted instead, so that the condition that only one unique information record exists in one user is ensured. In a specific embodiment, the working process of the dynamic database can refer to fig. 6; after the attribute information of each user is input into the dynamic database, the detailed table of the user information formed in the dynamic database can refer to fig. 7, and the field filling rule in the dynamic database can refer to fig. 8; in the figure, Username represents a User name, Accounts represents a User broadband account, User IP Addr represents a User IP address, and MAC Addr represents an access side device MAC address.

With further reference to fig. 9, the converting the non-IP information of the target user into IP information based on the non-IP information of the target user and the corresponding relationship, that is, step 20, specifically includes:

step 201, inputting non-IP information corresponding to a target user into the dynamic database for query according to a target group to be analyzed.

Step 202, according to the input non-IP information, the IP information corresponding to the target user is related and inquired from the dynamic database.

Step 203, after performing CIDR clustering analysis on the queried IP information, converting the queried IP information into a converged IP address segment so as to output the converged IP address segment to a back-end dynamic filtering rule generator.

In the embodiment of the present invention, if the dynamic database is provided with a corresponding query conversion module, the step 201 and the step 203 are as follows: when the network traffic of a target group is to be filtered, only non-IP information related to a target user needs to be input into a query conversion module, and the query conversion module can send a target IP query instruction to a dynamic database according to the input target cluster information (namely the non-IP information related to the target user); the dynamic database further returns the IP information corresponding to the target cluster to the query conversion module according to the corresponding relation; the query conversion module calculates the received target IP information through CIDR (class Inter-Domain Routing), converts the target IP information into a converged IP address section, and then outputs the converged IP address section to a rear-end dynamic filtering rule generator.

CIDR is a method of creating additional addresses on the Internet, where routes can be collected such that one IP address represents thousands of IP addresses served by a major backbone provider, thereby relieving the burden on Internet routers and IP-based processing systems. The CIDR has the function of carrying out CIDR real-time clustering analysis on the IP of the target user to obtain IP address field information so as to reduce rule items of the filtering rule generator.

With reference to FIG. 10, in an embodiment of the present invention, the query and translation functions are performed inside the dynamic database; the query function supports a user to input or import user non-IP information in batch through a web interface, automatically constructs a query statement by taking the input information as a condition, and outputs target group IP address field information calculated by CIDR to a rear-end filtering generator after executing query.

Further, in different application scenarios, target groups that workers need to analyze may also be different, and sometimes need to pay attention to network traffic required by a specific user, sometimes need to pay attention to network traffic required by a user under a specific device, and sometimes need to pay attention to network traffic required by a user in a specific area, that is, the target groups are specific users, specific devices, or specific areas. According to the difference between the scene and the target group, in step 20, the non-IP information input into the dynamic database during query conversion is different, which is specifically as follows:

first, user-specific queries and conversions. When the network traffic of a specific user needs to be analyzed, the non-IP information input to the dynamic database is specifically a user name, a user broadband account number, or an MAC address bit of the access side device.

Second, device-specific querying and translation. When the network flow of the user under the specific equipment needs to be analyzed, the non-IP information input into the dynamic database is specifically the OUI bit of the MAC address of the home gateway manufacturer or the dialing equipment at the user side of the user.

The second scenario mainly aims at a scenario that target traffic needs to be filtered according to the MAC address; for example, user traffic that needs to be filtered through the network using a home gateway (light cat) of a certain manufacturer is targeted to a target group using an access gateway device of a certain brand manufacturer. Wherein, the first 6 bits of the MAC address represent OUI (organization unique identifier) which represents corresponding manufacturer information, and manufacturers of the main user side gateway devices have beacon communication, hua shi, zhongxing, and the like.

At present, the main technologies of the broadband access network include the following 3 technologies: FTTN (Fiber To Node), FTTB (Fiber To The Building), and FTTH (Fiber To The Home). The FTTN and the FTTB mainly use router dialing, and the MAC address (Calling _ Station _ Id) carried in the AAA message is the MAC address of the wireless router at the user side; FTTH mainly uses home gateway dialing, and the MAC address (Calling _ Station _ Id) carried in AAA message is the MAC address of the home gateway at user side. Either the MAC of the home gateway or the MAC of the wireless router characterizes a smaller range of target individuals.

In summary, in this scenario, when the network traffic of the target group is to be filtered, the query translation module can send a target IP query instruction to the dynamic database according to the input information by only inputting the MAC address OUI bit of the user home gateway manufacturer (such as beacon communication) or the user side dialing device to the query translation module; the dynamic database returns the inquired IP information to the inquiry conversion module, and the inquiry conversion module converts the received target IP information into a converged IP address section through CIDR calculation and outputs the converged IP address section to the dynamic filtering rule generator. The specific query and translation workflow can be referred to in fig. 10.

Third, query and translation of specific areas (locations). When the network flow of a user in a specific area needs to be analyzed, the non-IP information input into the dynamic database is specifically the position information of the ONU or the OLT equipment; the position information is originated from a network resource management system of an operator and can be imported into the dynamic database through an interface.

The third scenario is to query a coverage area of an ONU (Optical Network Unit) or an OLT (Optical Line Terminal) in a broadband access Network, and is mainly directed to a scenario that a target traffic needs to be filtered according to a specific area, for example, a user traffic that needs to filter an industrial park, a residential district, and a large venue in the whole Network is a target group of a specific area (physical location). At this time, the location information of the specific area can be determined through the Access-Request in the AAA message, the radius. The NAS indicates a Network Access Server (Network Access Server).

In connection with the exemplary diagram of the signaling message of the relevant fields in fig. 11, the radius. The NAS _ IP _ Address + port information may determine a coverage area of one OLT device; NAS _ IP _ Address + port information + outer layer VLAN, can confirm the coverage area of a slot position of a OLT; NAS _ IP _ Address + port information + outer VLAN + inner VLAN, the coverage area of one ONU can be determined.

In summary, in this scenario, when the network traffic of the target group is to be filtered, only the location information of the ONU or the OLT device needs to be input to the query conversion module, and the query conversion module can send a target IP query instruction to the dynamic database according to the input information; the dynamic database returns the inquired IP information to the inquiry conversion module, and the inquiry conversion module converts the received target IP information into a converged IP address section through CIDR calculation and outputs the converged IP address section to the dynamic filtering rule generator. The specific query and translation workflow can be referred to in fig. 10.

Continuing with fig. 12, the generating a filtering rule in real time according to the IP information of the target user and issuing the filtering rule to the splitter at the back end in real time, that is, step 30, specifically includes:

step 301, inputting the converted IP address field information of the target user into a dynamic filtering rule generator. Or, the dynamic filtering rule generator calls the IP address field information of the target user from the dynamic database in real time.

And step 302, the dynamic filtering rule generator generates a filtering rule in real time according to the rule expression grammar of the rear-end splitter and the input IP address field information.

With reference to fig. 13, the dynamic filtering rule generator is mainly composed of a syntax selection module and an expression construction module; in the grammar selection module, grammar selection items of different splitter manufacturers can be set according to different grammars, so that the expression format of the filtering rule is selected, a standard criterion is provided for the rule construction of the expression construction module, and the fact that the rear-end splitter can recognize the received filtering rule is guaranteed. The generation process of the filtering rule is specifically as follows:

firstly, the dynamic filtering rule generator selects required grammar from grammar options of different shunt manufacturers according to requirements so that the shunt can identify the received filtering rule; the process is mainly implemented by the grammar selection module. Then, the dynamic filtering rule generator constructs an expression of the filtering rule based on the selected grammar and the input IP address field information of the target user, and further generates the filtering rule in real time; the process is mainly realized by the expression building module.

The filter rule expression format is exemplified as follows:

rule_id{[protocol={tcp|udp|icmp|protocol_number}][srcip=srcip/mask][dstip=dstip/mask] [srcport=srcport/mask] [dstport=dstport/mask] }

the meaning of each keyword in the above rules is as follows: protocol is protocol type, srcport is source protocol port number, dstport is destination protocol port number, srcip is source IP address (supporting IPv4 or IPv6), dstip is destination IP address (supporting IPv4 or IPv 6). If some keys are not defined in the rule, this is ignored and different protocols, port numbers are separated by "|". Further, the meaning and format of each parameter are as follows: the rule _ id is a rule number, and the serial number is 1, 2 and 3 … 10000 in sequence; protocol _ number is a protocol number, such as 0-255; srcport/mask and dstport/mask respectively represent the list of the port numbers of the source and the destination protocol, and can be single port number of the protocol or a section of continuous port number represented by a mask mode; srcip/mask and dstip/mask indicate source and destination IP addresses (supporting IPv4 or IPv6), and a mask is supported, if mask system default is 255.255.255.255, such as sip = 10.10.10.1/255.0.0.0.

For example, when generating an IPv6 rule for an IPv6 address user, the rule format is rule _ id { [ srcip = srcip/mask ] [ dstip = dstip/mask ] }. Wherein, rule _ id is a rule number; the srcip/mask is a source IPv6 address and supports a mask; the dstip/mask is a destination IPv6 address, supports a mask, is not provided with the mask when the IPv6 rule is added, and the default mask of the IPv6 is ffff, and the like. IPv6 filter rule example: rule 1 dstip =2001:0000:3A2F:0000:0000:0100:11A0: ADDF.

When a scene that target traffic needs to be filtered according to a MAC address, if user traffic of a TP-LINK optical cat used in the whole network needs to be filtered, namely a group using access equipment of a certain brand manufacturer needs to be filtered, a MAC rule needs to be generated, and the corresponding rule format is rule _ id { [ src = src/mask ] [ dstmac = dstmac/mask ] }. Wherein, rule _ id is a rule number, and the serial number is 1, 2 and 3 … 10000 in sequence; the SRcmac/mask is used as a source MAC address, a mask is supported, and both the SRcmac and the mask are expressed by a 16-system; dstmac/mask is used as a destination MAC address, a mask is supported, and both dstmac and mask are expressed in a 16-system mode; the MAC rule is added without a mask, the default mask is full FF, and the MAC address format must be completely filled in. Wherein, the MAC format is exemplified by AA, BB, CC, DD, EE, FF; an example MAC filtering rule is as follows: rule 1 src mac =00:12:34:56:78:90/ff: ff: ff: ff:00:00 dstmac =00:12:34:56:78:90/ff: ff: ff:00: 00.

And 303, the dynamic filtering rule generator sends the filtering rule to a rear-end flow divider in real time through an RPC interface.

After the filtering rules are generated, the filtering rules are called by an RPC interface negotiated between the dynamic filtering rule generator and the shunt equipment. In the embodiment of the present invention, a Protobuf API or a Restful API may be used as an interface protocol of the RPC. If Restful API is chosen as the interface protocol for inter-device RPC, then HTTP/2.0 protocol can be used for bearer, with the following advantages: the characteristics of bidirectional flow, message header compression, multiplexing of single TCP, server side pushing and the like are supported, HTTP connection does not need to be closed when a certain RPC is called overtime, and only corresponding Stream flow needs to be closed, so that frequent HTTP connection reconstruction in time-out can be avoided, the problems encountered by the traditional HTTP/1.X protocol are effectively solved, and the effect is close to that of the TCP private protocol of the RPC. In this case, FIG. 14 is referred to as an RPC roaming communication mode using HTTP/2.

If Protobuf is selected as the interface protocol of RPC between devices, Protobuf is a protocol of an extensible, light and efficient serialized data structure, and can be used for network communication and data storage. The Protobuf has the advantages of cross-platform, cross-language, good expandability, small size, high serialization speed, high transmission speed and the like, and after serialization, the data size can be reduced by about 3 times, and the Protobuf is smaller and faster than XML and JSON and is more convenient to use and maintain. Specifically, which interface protocol is used for the negotiation agreement between the visual service scene and the size of the rule file and the manufacturer of the back-end splitter is not specifically limited herein.

In combination with the embodiment of the present invention, there is also a preferred implementation scheme, and in order to increase the accuracy and stability of the filtering rules and form the virtuous circle of the whole system, the embodiment of the present invention further increases the feedback compensation control link of the user information. Feedback compensation control (Feedback compensation control) is a control method in which output information of a system is returned to an input terminal, compared with input information, and controlled by using a deviation between the output information and the input information, and the controlled output is returned to the input terminal as a control input in a fixed manner, thereby exerting a control influence on the input terminal. In this embodiment, the method is mainly used for verifying, correcting, and expanding information in a dynamic database, and after the flow splitter performs flow filtering according to a filtering rule, and outputs a target flow obtained by filtering to a DPI device for flow analysis, that is, after step 40, the method further includes:

and after analyzing the target flow through the DPI device, feeding back the analyzed key information to the dynamic database so as to compare the data of the dynamic database, and verifying, correcting and expanding the user attribute information and the corresponding relation in the database according to the comparison result. With reference to fig. 15, after the first DPI analyzer analyzes and processes the AAA traffic or the portal traffic, the obtained user attribute information is input to the dynamic database; and after the second DPI analyzer analyzes and processes the target flow, the obtained user attribute information is also fed back to the dynamic database, and the dynamic database can compare the original information in the database according to the information fed back by the second DPI analyzer, and then verify, correct and expand the user attribute information according to the comparison result to complete the feedback compensation of the user information. Through feedback compensation of information, a dynamic database can be enriched continuously, and more choices and possibilities are provided for flow filtering modes; the accuracy and stability of the filtering rules can be increased, and virtuous circle of the whole system is formed.

The intelligent flow scheduling method provided by the embodiment of the invention can generate direct economic benefits and social benefits for flow monitoring analysis of various large-flow service scenes:

from the economic benefit, aiming at the DPI system for processing the target flow with the flow scale of 10Tbps, the invention can save the hardware cost by about 1000 RMB, can save the energy consumption cost by about 210 RMB in one year, and has positive influence on the income and profit of network operators. Specifically, the traffic of a cable broadband metropolitan area network of a medium city of an operator is about 10Tbps, and when analyzing a specific target traffic in a conventional scheme, for example, a service perception of a target group needs to be analyzed: the indexes of the uplink and downlink speed, the uplink and downlink time delay and the like need to perform uplink and downlink full processing on the metropolitan area network traffic, find the XDR record of the target group in the XDR ticket generated after the processing, and further analyze the service perception index. This requires that DPI, ETL, and database clusters all have full-flow throughput capability, and the entire system is particularly bulky, heavy, and costly.

Referring to fig. 16, from the viewpoint of hardware cost, the hardware cost of the conventional scheme for processing 10Tbps traffic of a metropolitan area network with only a DPI part is up to 1030 ten thousand yuan, which does not consider the cost of ETL (data distribution, computation) and database cluster at the back end; in the invention, the target flow can be directly filtered out, and the hardware cost can be saved by 1000 ten thousand yuan. From the energy consumption cost, the number of DPI hardware equipment can be reduced by 100, the power of 1 DPI device with 100Gbps processing capacity is 2400W per hour, the annual power consumption of one year is 2102400 kW.h, the power can be saved by 2102400 kW.h according to the electricity price of 1 yuan per degree, and the electricity charge can be saved by 210.24 ten thousand yuan.

In terms of social benefits, the DPI system is located at the upstream of a big data industrial chain in the communication industry, and as a data producer, an intelligent flow scheduling method appears, so that the production scale of data is determined and controlled to a certain extent, and the data is optimized and reduced. The method has social benefits of reducing investment, saving cost, saving energy and reducing emission for the whole industrial chain ecology. The invention can flexibly define the target analysis group, effectively promote the improvement of network quality, promote the experience of network citizens business, promote more intelligent service users of network operators, promote market competition, and is beneficial to improving the communication service quality and promoting the benign development of national economy.

Example 2:

on the basis of the foregoing embodiment 1, an embodiment of the present invention provides an intelligent traffic scheduling system based on a DPI and a splitter, which can be used to implement the intelligent traffic scheduling method in embodiment 1. As shown in fig. 1, the intelligent traffic scheduling system includes a shunting device, a DPI device and an intelligent traffic scheduling device, which are sequentially connected in series; the DPI device further comprises a first DPI analyzer and a second DPI analyzer, and the intelligent flow scheduling device further comprises a dynamic database and a dynamic filtering rule generator. The first DPI analyzer is respectively connected with the splitter and the dynamic database, and the second DPI analyzer is connected to the rear end of the splitter.

The flow divider is mainly a flow divider, and the main function of the flow divider is to realize free guide output by copying, converging, dividing, filtering, cooperating and the like network flow so as to meet the deployment requirements of various flow monitoring and analysis. In the embodiment of the invention, the forwarding and filtering functions are mainly used, on one hand, AAA or portal traffic is forwarded to a first DPI analyzer for processing; and on the other hand, the filtering rule generated by the intelligent flow scheduling device (specifically a dynamic filtering rule generator) is received in real time through an RPC interface, the target flow of the BRAS full flow is filtered according to the filtering rule, and the target flow meeting the filtering condition is output to a second DPI analyzer for processing. The target traffic refers to the traffic generated by a given target group, which is included in the BRAS full traffic and can only be found and output through a specific filtering rule.

The DPI device mainly has the functions of carrying out deep packet inspection, service identification, coding and decoding on the flow message, generating, associating, backfilling and combining an XDR ticket, and generating structured data to a database system; in an embodiment of the present invention, the DPI device is mainly used to analyze and process the flow filtered by the splitter. The first DPI analyzer is used for analyzing and processing AAA traffic or portal traffic filtered by the flow divider, further acquiring user attribute information and inputting the user attribute information into the dynamic database in real time; and the second DPI analyzer is used for analyzing and processing the target flow filtered by the diverter according to the filtering rule and generating a user XDR detailed list.

The intelligent flow scheduling device comprises a dynamic database and a dynamic filtering rule generator, wherein the dynamic database is mainly used for establishing and maintaining the corresponding relation between the IP information of a user and each piece of non-IP information, receiving the output data of the DPI device in real time and also receiving user attribute information input by a manual or third-party system; and the user IP query request of the staff aiming at the target group can be received and processed to complete the conversion from the non-IP information of the target user to the IP information, and the IP information of the target group is output to the dynamic filtering rule generator. The dynamic filtering rule generator has the main functions of constructing a filtering rule meeting the conditions according to the received IP information of the target user and sending the filtering rule to the flow divider in real time through the negotiated RPC interface.

The user attribute information comprises non-IP information and IP information, the non-IP information comprises one or more of a user broadband account, user MAC information, MAC information of access side equipment and IP information of the access side equipment, and the IP information mainly refers to a user IP address.

In the intelligent flow scheduling system provided by the embodiment of the invention, the intelligent flow scheduling device is positioned between the flow divider and the DPI device, and not only depends on the information input of the DPI device, but also generates the filtering rule to filter the target flow for the flow divider, and forms a set of self-produced and self-used intelligent flow scheduling system by deeply matching with the DPI system, so as to meet the flexible deployment of various flow monitoring and flow analysis requirements.

When the network traffic of a target group is to be filtered, only non-IP information (such as user Username, user broadband account or MAC information of access side equipment) related to a target user needs to be input into the query conversion module; the query conversion module can send a target IP query instruction to the dynamic database according to the input non-IP information, and the dynamic database returns the queried IP information corresponding to the target cluster to the query conversion module; and the query conversion module converts the received target IP information into a converged IP address section through CIDR calculation and outputs the converged IP address section to a rear-end dynamic filtering rule generator.

With further reference to FIG. 13, the dynamic filter rule generator includes a grammar selection module and an expression building module. The grammar selection module is connected to the rear end of the dynamic database, and the expression construction module is connected to the front end of the flow divider. In the grammar selection module, grammar selection items of different splitter manufacturers can be set according to different grammars, so that the expression format of the filtering rule is selected, a standard criterion is provided for the rule construction of the expression construction module, and the fact that the rear-end splitter can recognize the received filtering rule is guaranteed. Therefore, the grammar selection module is mainly used for selecting the required grammar from grammar selection items of different shunt manufacturers and providing a standard criterion for the rule construction of the expression construction module; the expression building module is mainly used for building the expression of the filtering rule according to the selected grammar in the grammar selecting module and the IP information of the target user.

The intelligent flow scheduling device provided by the embodiment of the invention can monitor and analyze the flow of various large-flow service scenes to generate direct economic benefits and social benefits:

in terms of economic benefits, for a DPI system processing target traffic of 10Tbps traffic scale, hardware cost can be saved by about 1000 million yuan after an intelligent traffic scheduling device is used, energy consumption cost can be saved by about 210 million in one year, and positive influence is generated on income and profit of network operators.

In terms of social benefits, the intelligent flow scheduling device determines and controls the production scale of data to a certain extent, plays roles in optimizing and reducing the data, and plays roles in reducing investment, saving cost, saving energy and reducing emission in the whole industrial chain ecology. The intelligent flow scheduling device can flexibly define a target analysis group, effectively promote the improvement of network quality, promote the service experience of network residents, promote more intelligent service users of network operators, promote market competition, and is favorable for improving the communication service quality and promoting the benign development of national economy.

Example 3:

on the basis of the intelligent traffic scheduling method provided in embodiment 1, the present invention further provides an intelligent traffic scheduling apparatus for implementing the method, and as shown in fig. 17, the apparatus is a schematic diagram of an apparatus architecture in an embodiment of the present invention. The intelligent traffic scheduling apparatus of the present embodiment includes one or more processors 21 and a memory 22. In fig. 17, one processor 21 is taken as an example.

The processor 21 and the memory 22 may be connected by a bus or other means, and fig. 17 illustrates the connection by a bus as an example.

The memory 22, which is a non-volatile computer-readable storage medium for the intelligent traffic scheduling method, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the intelligent traffic scheduling method in embodiment 1. The processor 21 executes various functional applications and data processing of the intelligent traffic scheduling apparatus by running the nonvolatile software program, instructions and modules stored in the memory 22, that is, implements the intelligent traffic scheduling method of embodiment 1.

The memory 22 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The program instructions/modules are stored in the memory 22 and, when executed by the one or more processors 21, perform the intelligent traffic scheduling method of embodiment 1 described above, for example, perform the steps shown in fig. 2 and 3 described above.

Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the embodiments may be implemented by associated hardware as instructed by a program, which may be stored on a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims

1. An intelligent flow scheduling method based on DPI and a flow divider is characterized by comprising the following steps:

analyzing the broadband flow through a DPI device to obtain user attribute information in real time, and further establishing a corresponding relation between the IP information of the user and each piece of non-IP information in real time;

the user attribute information comprises non-IP information and IP information, wherein the non-IP information comprises one or more of a user broadband account, user MAC information, MAC information of access side equipment and IP information of the access side equipment; the step of converting the non-IP information of the target user into the IP information based on the non-IP information of the target user and the corresponding relationship among the user attribute information and the IP information and the non-IP information is specifically comprised of:

inputting non-IP information corresponding to a target user to the dynamic database for query according to a target group to be analyzed; according to the input non-IP information, IP information corresponding to the target user is inquired from the dynamic database in a correlation mode; and after CIDR cluster analysis is carried out on the inquired IP information, the IP information is converted into a converged IP address segment so as to be output to a dynamic filtering rule generator at the rear end.

2. The intelligent traffic scheduling method based on DPI and splitter according to claim 1, wherein the analyzing of the broadband traffic by the DPI device obtains the user attribute information in real time, and further establishes the correspondence between the IP information of the user and each non-IP information in real time, specifically comprising:

3. The intelligent traffic scheduling method based on DPI and flow splitter according to claim 1, wherein the target group is a specific user, a specific device or a specific area;

4. The intelligent traffic scheduling method based on DPI and splitter according to claim 1, wherein the generating a filtering rule in real time according to the IP information of a target user and issuing the filtering rule to the splitter at the back end in real time specifically includes:

5. The intelligent traffic scheduling method based on DPI and splitter according to claim 4, wherein the dynamic filtering rule generator generates the filtering rule in real time according to the rule expression syntax of the back-end splitter and the input IP address field information, specifically:

6. The intelligent traffic scheduling method based on DPI and flow divider according to claim 2, wherein after the flow divider performs traffic filtering according to a filtering rule and outputs the filtered target traffic to a DPI device for traffic analysis, the method further comprises:

7. An intelligent flow scheduling system based on DPI and shunt, which is used for realizing the intelligent flow scheduling method based on DPI and shunt of any one of claims 1-6, and comprises the shunt, a DPI device and an intelligent flow scheduling device, wherein the intelligent flow scheduling device comprises a dynamic database and a dynamic filtering rule generator;

the shunt is used for receiving the filtering rule generated by the dynamic filtering rule generator, filtering out target flow according to the filtering rule and outputting the target flow to the DPI device; the DPI device is used for analyzing and processing the flow filtered by the shunt, generating a user XDR detailed list and acquiring user attribute information;

8. The DPI and splitter based intelligent traffic scheduling system of claim 7 wherein the DPI device comprises a first DPI analyzer and a second DPI analyzer;

9. The DPI and splitter based intelligent traffic scheduling system of claim 7 wherein the dynamic filter rules generator comprises a grammar selection module and an expression building module;

the grammar selection module is connected to the rear end of the dynamic database and is used for selecting required grammars from grammar selection items of different shunt manufacturers and providing a standard criterion for the rule construction of the expression construction module;