WO2012151843A1

WO2012151843A1 - Ulr filtering system, method and gateway

Info

Publication number: WO2012151843A1
Application number: PCT/CN2011/080608
Authority: WO
Inventors: 王永光; 沈蓓洁; 卢勤元; 李冰
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-05-11
Filing date: 2011-10-10
Publication date: 2012-11-15
Also published as: CN102780681A

Abstract

The present invention relates to a ULR filtering system, method and gateway. The system includes an identification unit, a memory unit, a rule unit, a scanning unit and a matching unit. The method includes: generating a ULR rule file which can be identified by a system according to a user-defined URL list, and loading the URL rule file into the memory; when receiving a message, the system scanning and judging whether the message is an HTTP message, and when it is determined that it is an HTTP message, scanning the ULR information in the HTTP message, matching the same with the URL information in the URL rule file in the memory, and allowing or not allowing the pass of the HTTP message according to the match result. The present invention need not distinguish the type of the URL, accelerating the URL processing speed.

Description

URL filtering system and method for filtering URL, gateway

The present invention relates to the field of communications, and in particular, to a Uniform I Universal Resource Locator (URL) filtering system, a method for filtering a URL, and a gateway. Background technique

A URL, also known as a web page address, is the address of a standard resource on the Internet, an identification method used to fully describe the addresses of web pages and other resources on the Internet. Every web page on the Internet has a unique URL address name identifier, usually called a URL address. This address can be a local disk, or a computer on a local area network, and more is a site on the Internet. . Simply put, a URL is a web address, commonly known as a "URL."

With the popularity of the Internet, information on the Internet has provided more and more convenience for people's lives and work. The number of teenagers who have access to the Internet is increasing, but the information on the Internet is mixed, especially there are still many. Websites that promote pornography, violence, and superstitions, in order to present a healthy and up-to-date website for teenagers, need to filter the URLs they visit to block out unhealthy and illegal websites to ensure the health of teenagers. growing up.

Currently, there are three main URL filtering methods:

First, the hash table is used to store the URL information; the method is applicable to URL searches with different domain names, and when the domain name is the same, it takes a long time to find;

Second, use a string matching algorithm; this method is suitable for keyword lookup, but the search speed is slower;

Third, use a regular matching algorithm; this method is suitable for indeterminate URL lookups, but the search speed is slower.

The speed of the existing method search will decrease significantly as the URL record in the URL list increases. In this way, URL management in today's high-throughput networks cannot be met. Summary of the invention

An object of the present invention is to provide a URL filtering system and a method and a gateway for filtering URLs, so as to solve the problem of improving the speed of searching for URLs in the prior art.

The present invention provides a method for filtering a URL, the method comprising:

Generating a URL rule file recognizable by the URL filtering system according to the user-defined URL list, and loading the URL rule file into the memory;

When the system receives the packet, it scans and determines whether the packet is a Hyper Text Transfer Protocol (HTTP) packet, and when it is determined to be an HTTP packet, scans the URL in the HTTP packet. Information, and matching with the URL information in the URL rule file in the memory;

The HTTP message is allowed or not allowed to pass according to the matching result.

In the above solution, after the URL rule file is loaded into the memory, the method further includes: determining whether the user-defined URL list has changed, and determining the change, according to the changed user-defined URL list, Regenerate the system-recognized URL rule file and load the newly generated URL rule file into memory;

After the loading is successful, the system uses the new URL rule file in the memory to match the URL information and delete the old URL rule file in the memory.

In the above solution, the method further includes: when the system determines that the received packet is not an HTTP packet, the system directly allows the packet to pass.

In the foregoing solution, the user-defined URL list is a blacklist or a whitelist. In the above solution, the enabling or disallowing of the HTTP packet according to the matching result includes:

When the URL list of the user-defined URL is blacklisted, if the URL information in the received HTTP packet matches the URL information in the URL rule file in the memory, the permission is not allowed. The HTTP packet is passed; if the URL information in the received HTTP packet fails to match the URL information in the URL rule file in the memory, the HTTP packet is allowed to pass;

When the user-defined URL list is a whitelist, if the URL information in the received HTTP packet matches the URL information in the URL rule file in the memory, the HTTP packet is allowed to pass; If the URL information in the HTTP packet is not matched with the URL information in the URL rule file in the memory, the HTTP packet is not allowed to pass.

The present invention also provides a URL filtering system, including: an identification unit and a memory unit, the system further comprising a rule unit, a scanning unit, and a matching unit;

The identifying unit is configured to identify whether the received packet is an HTTP packet, and send the identification result to the scanning unit;

The rule unit is configured to generate a URL rule file recognizable by the system according to the user-defined URL list, and load the URL rule file into the memory unit;

The scanning unit is configured to scan the received message and send it to the message identification unit, and scan when the identification result returned by the identification unit is determined to be that the received message is an HTTP message. The URL information in the HTTP packet, and the URL information is sent to the matching unit; and the HTTP packet is allowed or not allowed to pass according to the matching result returned by the matching unit; the matching unit is set to Matching the URL information in the HTTP packet with the URL information in the URL rule file in the memory unit, and transmitting the matching result to the scanning unit.

In the above solution, the rule unit is further configured to determine whether the user-defined URL list has changed, and when determining that the user-defined URL has changed, according to the changed user-defined URL list, Regenerating the system-recognizable URL rule file, loading the newly generated URL rule file into the memory unit, and after the loading is successful, notifying the matching unit to use the new URL rule file for URL information matching.

In the above solution, the matching unit is further configured to receive the notification of the rule unit, The URL information is matched using the new URL rule file, and the old URL rule file in the memory unit is deleted.

In the above solution, the scanning unit is further configured to directly allow the message to pass when receiving the identification result returned by the identifying unit to determine that the received message is not an HTTP message.

The invention also provides a gateway comprising the above URL filtering system.

The invention converts the user-defined URL list into a URL rule file identifiable by the URL system hardware and loads it into the memory. When receiving the message, the system can quickly match the HTTP message with the URL rule file in the memory. And the matching result is given, the scanning matching speed can reach at least 2 Gbps, and the type of the URL is not required to be distinguished, the complicated and cumbersome URL classification and searching in the existing method are omitted, and the URL processing speed is accelerated; the present invention supports big data. The URL filtering is applicable to network devices such as Integrated Service Gateway (ISG), Wireless Application Protocol (WW) gateway, and WEB gateway. DRAWINGS

The drawings are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flow chart of a method for filtering a URL according to the present invention;

2 is a schematic block diagram of a URL filtering system according to the present invention;

3 is a schematic block diagram of a gateway of the present invention. detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments in order to make the present invention. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. FIG. 1 is a flowchart of a method for filtering a URL according to the present invention. In this embodiment, a method for filtering a URL is as follows. As shown in FIG. 1 , the method for filtering a URL includes the following steps:

Step S001: Generate a URL rule file recognizable by the URL filtering system according to the user-defined blacklist;

Step S002: Loading the generated URL rule file into the memory;

Step S003: The system receives the packet;

Step S004: Scan the said ^^ text;

Step S005: determining whether the packet is an HTTP packet, if yes, executing step S006, otherwise, performing step S010;

Step S006: Scan URL information in the packet.

Step S007: Matching the URL information in the URL rule file in the memory; Step S008: determining whether the matching is successful, if yes, executing step S009; otherwise, executing step S010;

Step S009: Filtering the information;

Here, the filtering the packet means that the packet is not allowed to pass.

Step S010: Release the message.

Here, the releasing the message means allowing the message to pass, and the message in this step includes an HTTP packet and a non-HTTP packet.

In other embodiments, when the user-defined URL list is a whitelist, if the URL information in the received HTTP message matches the URL information in the in-memory URL rule file, the HTTP message is released. If the URL information in the received HTTP packet fails to match the URL information in the URL rule file in the memory, the HTTP packet is filtered.

In the present invention, the system may further determine whether the user-defined URL list has changed, and if so, according to the changed user-defined URL list. Regenerate the URL rule file recognizable by the system, and load the newly generated URL rule file into the memory. After the loading is completed, use the new URL rule file to match the URL information, and delete the old URL rule file, which can make this The invention implements real-time update of the URL rule file without interrupting the scan matching service. In a specific embodiment, the memory A and the memory B can be reserved. If the old URL rule file is stored in the memory A, the newly generated URL rule file is loaded after the user-defined URL list is changed. In the memory B, after the loading is completed, the system uses the URL rule file in the memory B to match the URL information, and at the same time, deletes the URL rule file in the memory A, and when the user-defined URL list changes again, the newly generated The URL rule file is loaded into memory A, and so on. That is to say, the system performs two tasks at the same time, one is to process the received message, and the other is to detect whether the user-defined URL list has changed.

The hardware-based filtering method of the present invention improves the speed of processing HTTP messages compared with existing software-based methods.

2 is a schematic block diagram of a URL filtering system according to the present invention. As shown in FIG. 2, the system includes: a scanning unit 01, an identification unit 02, a rule unit 03, a matching unit 04, and a memory unit 05;

The scanning unit 01 is configured to scan the received message and send it to the message identification unit 02, or scan the URL information in the HTTP message, and send the URL information to the matching unit 04; and according to the identification unit 02 The returned recognition result and the matching result returned by the matching unit 04, release and/or filter the received message;

The identifying unit 02 is configured to identify whether the received packet is an HTTP packet, and send the identification result to the scanning unit 01;

The rule unit 03 is configured to generate a system-recognizable URL rule file according to the user-defined URL list, and load the generated URL rule file into the memory unit 05; and determine whether the user-defined URL list changes, and The user-defined URL has changed During the process, according to the changed user-defined URL list, the system-recognized URL rule file is regenerated, the newly generated URL rule file is loaded into the memory unit 05, and after the loading is completed, the matching unit 04 is notified to use the new URL rule file for URL information matching;

The matching unit 04 is configured to match the received URL information with the URL information in the URL rule file in the memory unit 05, and send the matching result to the scanning unit 01, or when receiving the notification of the rule unit 03, The URL information matching is performed using the newly loaded URL rule file in the memory unit 05, and the old URL rule file in the memory unit 05 is deleted.

When the recognition result returned by the identification unit 02 is that the received message is not an HTTP message, the scanning unit 01 scans the URL information in the HTTP message, and sends the URL information to the matching unit 04. And releasing or filtering the HTTP packet according to the matching result returned by the matching unit;

When the recognition result returned by the identification unit 02 is that the received message is not an HTTP message, the scanning unit 01 directly releases the message.

3 is a schematic block diagram of a gateway according to the present invention. As shown in FIG. 3, the URL filtering system shown in FIG. 2 is included. The URL filtering system includes a scanning unit 01, an identification unit 02, a rule unit 03, a matching unit 04, and a memory unit 05. For the function of each unit, refer to the description of Figure 2 above, and it will not be repeated here.

The above description shows and describes a preferred embodiment of the present invention, but as described above, it should be understood that the invention is not limited to the form disclosed herein, and should not be construed as Other combinations, modifications, and environments are possible and can be modified by the teachings of the above teachings or related art within the scope of the inventive concept described herein. All changes and modifications made by those skilled in the art are intended to be within the scope of the appended claims.

Claims

Claim

A method for filtering a uniform resource locator URL, wherein the method comprises: generating a URL rule file recognizable by a URL filtering system according to a user-defined URL list, and loading the URL rule file into a memory;

When the system receives the message, it scans and determines whether the message is a Hypertext Transfer Protocol (HTTP) message, and determines that the HTTP message is in the HTTP message, and Match the URL information in the in-memory URL rule file;

2. The method according to claim 1, wherein, after the URL rule file is loaded into the memory, the method further includes:

Determining whether there is a change in the user-defined URL list, and determining that there is a change, regenerating the system-recognizable URL rule file according to the changed user-defined URL list, and loading the newly generated URL rule file into the memory Medium

3. The method according to claim 1, wherein the method further comprises: when the system determines that the received message is not an HTTP message, directly allowing the message to pass.

4. The method according to claim 2, wherein the user-defined URL list is a blacklist or a whitelist.

The method according to any one of claims 1 to 4, wherein the allowing or disallowing the HTTP ^ 艮 text according to the matching result includes:

If the URL list of the user-defined URL is a blacklist, if the URL information in the received HTTP packet matches the URL information in the URL rule file in the memory, the HTTP packet is not allowed to pass. If the URL information in the received HTTP packet fails to match the URL information in the URL rule file in the memory, the HTTP packet is allowed to pass. When the user-defined URL list is a whitelist, if the URL information in the received HTTP packet matches the URL information in the URL rule file in the memory, the HTTP packet is allowed to pass; If the URL information in the HTTP packet is not matched with the URL information in the URL rule file in the memory, the HTTP packet is not allowed to pass.

A URL filtering system, the system includes: an identification unit and a memory unit, wherein the system further includes: a rule unit, a scanning unit, and a matching unit;

The scanning unit is configured to scan the received message and send it to the message identification unit, and scan when the identification result returned by the identification unit is determined to be that the received message is an HTTP message. Transmitting the URL information in the HTTP packet, and sending the URL information to the matching unit; and allowing or disallowing the HTTP packet to pass according to the matching result returned by the matching unit;

The matching unit is configured to match the URL information in the HTTP packet with the URL information in the URL rule file in the memory unit, and send the matching result to the scanning unit.

7. The system according to claim 6, wherein

The rule unit is further configured to determine whether the user-defined URL list has changed, and when determining that the user-defined URL has changed, according to the changed user-defined URL list, the system may be regenerated. The identified URL rule file loads the newly generated URL rule file into the memory unit, and after the loading is successful, notifies the matching unit to use the new URL rule file to perform URL information matching.

8. The system according to claim 7, wherein The matching unit is further configured to: after receiving the notification of the rule unit, use a new URL rule file to perform URL information matching, and delete the old URL rule file in the memory unit.

The system according to any one of claims 6 to 8, wherein the scanning unit is further configured to: when receiving the recognition result returned by the identification unit, determining that the received message is not an HTTP message, The message is allowed to pass directly.

A gateway, wherein the gateway comprises the URL filtering system of any one of claims 6 to 9.