US10735501B2 - System and method for limiting access request - Google Patents

System and method for limiting access request Download PDF

Info

Publication number
US10735501B2
US10735501B2 US15/542,086 US201615542086A US10735501B2 US 10735501 B2 US10735501 B2 US 10735501B2 US 201615542086 A US201615542086 A US 201615542086A US 10735501 B2 US10735501 B2 US 10735501B2
Authority
US
United States
Prior art keywords
blacklist
load balancer
statistical data
predefined
request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/542,086
Other languages
English (en)
Other versions
US20180278678A1 (en
Inventor
Zhi WENG
Sixing XIAO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Publication of US20180278678A1 publication Critical patent/US20180278678A1/en
Assigned to BEIJING JINGDONG CENTURY TRADING CO., LTD., BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD. reassignment BEIJING JINGDONG CENTURY TRADING CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: WENG, Zhi, XIAO, Sixing
Application granted granted Critical
Publication of US10735501B2 publication Critical patent/US10735501B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H04L67/1002—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227—Filtering policies
    • H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
    • H04L63/101—Access control lists [ACL]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004—Server selection for load balancing
    • H04L67/1023—Server selection for load balancing based on a hash applied to IP addresses or costs
    • H04L67/2814—
    • H04L67/2819—
    • H04L67/2828—
    • H04L67/2833—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/563—Data redirection of data network streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/564—Enhancement of application control based on intercepted application data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/565—Conversion or adaptation of application format or content
    • H04L67/5651—Reducing the amount or size of exchanged application data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/566—Grouping or aggregating service requests, e.g. for unified processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/164—Adaptation or special uses of UDP protocol
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1441—Countermeasures against malicious traffic
    • H04L63/1458—Denial of Service

Definitions

  • the invention relates to a system for limiting access requests from a network and its corresponding method.
  • a legacy load balancer for example, uses a configuration of HaProxy in linux to read cookies or URL descriptions contained in each HTTP request from a network, and rewrite a header based on these pieces of information and send the HTTP request to a backend server cluster, so that a balanced state of traffic and resource consumption is achieved for each server in the backend server cluster. But the legacy load balancer would not automatically perform a filtering check on traffics from the network, and cannot perform throttling or discarding on traffics considered as cyber attack.
  • One conception is to analyze an access to URL, and limit access requests according to the number of access requests per unit time such as QPS.
  • the analysis on the access to the URL of a large scale website usually consumes a lot of memory.
  • it is required to record a timestamp of each access associated with any combination of data fields such as an IP address, a user identification (USERID) and a uniform resource locator (URL).
  • TORID user identification
  • URL uniform resource locator
  • respective time points are filtered out or sorted, which consumes time as well as consumes a memory space.
  • the respective time points are filtered out or sorted, which consumes the time as well as consumes the memory space.
  • the object of the invention is to provide a system and method that at least partially solve the above problem.
  • the invention is particularly suitable for solutions of the problem of DOS attacks (denial of service attacks) and DDOS attacks (distributed denial of service attacks) in a network. This technique is particularly suitable for prevention against HTTP flooding.
  • the invention recognizes attack traffics that follow pattern matching and performs throttling or discarding on the attack traffics by accumulating and analyzing user traffics to thereby protect the backend servers.
  • a system for limiting access requests comprises a load balancer, an aggregator and a summarizer.
  • the load balancer is configured to receive each access request from a network, parse the received access request into a UDP message, and transmit the UDP message to the aggregator;
  • the aggregator is configured to accumulate the plurality of UDP messages received from the load balancer according to a predefined combination of data fields and a predefined duration so as to produce a statistical data, and send the accumulated statistical data to the summarizer in response to a request from the summarizer;
  • the summarizer is configured to request the statistical data from the aggregator according to a predefined request time interval, receive the statistical data from the aggregator, generate a blacklist for access requests according to the received statistical data and a predefined rule, and send the blacklist to the load balancer in response to a request from the load balancer, wherein the blacklist specifies a processing action to be made on each of one or more specific access requests.
  • the load balancer is
  • a computer-implemented method for limiting access requests comprises: a load balancer receiving each access request from a network, parsing the received access request into a UDP message, and transmitting the UDP message to the aggregator; and the load balancer determining a processing action on the received access request according to a current blacklist, wherein the blacklist specifies the processing action to be made on each of one or more specific access requests; the aggregator receiving the UDP message sent by the load balancer, accumulating the received plurality of UDP messages according to a predefined combination of data fields and a predefined duration so as to produce a statistical data, and sending the accumulated statistical data to a summarizer in response to a request from the summarizer; and the summarizer requesting the statistical data from the aggregator according to a predefined request time interval, receiving the statistical data from the aggregator, generating an a blacklist for access requests according to the received statistical data and a predefined rule, and sending the blacklist to the
  • the statistical data includes the number of the UDP messages containing the predefined combination of data fields accumulated within the predefined duration.
  • the network is the Internet
  • the access request is an HTTP request from the Internet.
  • the aggregator comprises a plurality of aggregation units operating independently of each other
  • the load balancer transmits each UDP message to one of the plurality of aggregation units, and each of the plurality of aggregation units performs the accumulation, respectively.
  • the summarizer receives the statistical data from the plurality of aggregation units, respectively, and produces the summarized statistical data.
  • the blacklist is generated based on the summarized statistical data and the predefined rule.
  • the load balancer further comprises a predefined whitelist which includes one or more records relating to the access requests to be passed. And the load balancer determines for each of the received access requests the processing action on the access request as follows: when the access request matches with one of the records in the whitelist or does not match with any one of the records in the blacklist, the access request is passed; and when the access request does not match with any one of the records in the whitelist and matches with one of the records in the blacklist, the access request is processed according to the processing action specified by the matched record in the blacklist.
  • the predefined whitelist in the load balancer can be obtained from the summarizer along with the blacklist.
  • the processing action specified in the blacklist includes one of the following: an interception, a redirection and a delay.
  • FIG. 1 is a structural diagram of a system for limiting access requests according to the invention
  • FIG. 2 is a flow diagram of processing the received access requests by a load balancer according to the invention.
  • FIG. 3 is a block diagram of an exemplary composition of a summarizer according to the invention.
  • the invention provides a system and method for limiting access requests from a network using a statistical data.
  • An access request such as an HTTP request at least includes the following data fields: a destination domain name (host) to be accessed by the user who sends a query; a universal resource identifier (uri) requested by the user; a user identification (uid); a client IP address (cip) and the like
  • the invention provides a system for limiting access requests from a network, the system comprising a load balancer, an aggregator and a summarizer.
  • the load balancer receives the access request from the network, parses each of the received access requests into a UDP message, and transmits the UDP message to the aggregator.
  • the aggregator accumulates the received plurality of UDP messages according to a predefined combination of data fields and a predefined duration so as to produce a statistical data, and sends the statistical data to the summarizer in response to a request from the summarizer.
  • the statistical data includes the number of the UDP messages containing the predefined combination of data fields accumulated within the current predefined duration.
  • the summarizer requests the statistical data relating to the access requests from the aggregator according to a predefined request time interval, receives the statistical data from the aggregator, generates a blacklist for access requests according to the received statistical data and a predefined rule, and sends the blacklist to the load balancer according to a request of the load balancer, wherein the blacklist specifies a processing action on each of one or more specific access requests.
  • the load balancer determines the processing action on each of the received access requests according to the current blacklist.
  • FIG. 1 shows a system 100 according to an embodiment of the invention, and the system 100 comprises the following three modules:
  • the load balancer 101 receives access requests from a network, e.g., an HTTP request from the Internet, parses each of the received access requests into a UDP message, and transmits the UDP message to the aggregator 102 .
  • a network e.g., an HTTP request from the Internet
  • UDP protocol Since the processing for the UDP messages consumes less resources, and has a fast processing speed, overhead of the load balancer can be greatly reduced.
  • a UDP protocol is not a connection-based protocol, so there may be some cases of a message loss. But such loss is within an acceptable error range, and its impact on robustness of the system is negligible.
  • the aggregator 102 may comprise a plurality of aggregation units.
  • the plurality of aggregation units can be scaled up independently, that is, each of the aggregation units is deployed and operated independently, and correspondingly, the aggregation units do not affect and depend on each other. If the processing capacity of the current aggregator cannot satisfy requirements, it is convenient to add server resources so that new aggregation units are deployed. This can better satisfy requirements for a large traffic flow.
  • the load balancer 101 transmits each UDP message to at least one selected aggregation unit. And the respective aggregation units accumulate the received UDP messages, respectively, and produce the statistical data.
  • the load balancer 101 sends the UDP messages to the plurality of aggregation units by turns. For example, the first UDP message is sent to the first aggregation unit, the second UDP message is sent to the second aggregation unit, and the produced UDP messages are sequentially sent to the respective aggregation units. After a UDP message is sent to the last aggregation unit, a new UDP message is sent to the first aggregation unit again.
  • Each aggregation unit analyzes and accumulates the received UDP messages according to a predefined rule or condition such as a predefined combination of data fields and a predefined time interval.
  • Each UDP message contains information of the HTTP access request from the user such as the destination domain name (host) accessed by the user, the universal resource identifier (uri) requested by the user, the user identification (uid) and the client IP address (cip).
  • the aggregation unit counts the number of the access requests having the same destination domain name (same host) and the same source (same uid, same cip) to thereby obtain the number of the access requests from the source.
  • the number of the access requests per unit time from the source can be derived according to the number of the received access requests from the source within a defined period (e.g., the last 60 seconds or the last 5 minutes). For example, a value of a query per second (QPS) relating to the source can be obtained by dividing the total number of the received access requests from the source within the defined period by the duration of the period.
  • QPS query per second
  • the aggregation unit sends the latest statistical data relating to the access requests produced at the aggregation unit to the summarizer 103 according to a request from the summarizer 103 .
  • the aggregator 103 can request the “statistical data” from the aggregator or each of the aggregation units every predefined time, e.g., every 10 seconds, and such request, for example, may be in the form of HTTP.
  • each aggregation unit of the aggregator After each aggregation unit of the aggregator receives the request from the summarizer 103 , it constructs the latest statistical data relating to the access produced thereby into a response message such as a response message in the form of HTTP, and sends the response message containing the latest statistical data to the summarizer 103 .
  • the summarizer 103 processes the statistical data contained in the messages according to a specific classification after receiving the response messages from the respective aggregation units, so as to produce the summarized statistical data and generate the blacklist according to the predefined rule.
  • a blacklist record in the blacklist may contain the following four parameters:
  • the records in the blacklist can be understood as follows: for a request sent to the destination domain name (host), if the user identification and the client IP contained in the request are equivalents to the corresponding values (source user identification, source client IP) in the record, the processing action specified in the record is performed.
  • the processing action may be, for example, an interception, a redirection, or a delay for the request.
  • the access request is considered to match with the blacklist.
  • a threshold value for the number of the requests per unit time and the corresponding processing action in the case that the threshold value is exceeded are set in a configuration file 1036 of the summarizer 103 .
  • the summarizer 103 makes a judgment according to the summarized statistical data for the access requests having the same combination of data fields, when a value of the number of the access requests per unit time (QPS) for such access requests exceeds the threshold value for the number of the access requests per unit time corresponding to the destination domain name thereof set in the configuration file 1036 , a corresponding blacklist record is produced, and the record includes the destination domain name, the user identification, and the client IP of the request, as well as the corresponding processing action.
  • the produced blacklist record is added to the blacklist to thereby produce the current blacklist.
  • a whitelist may be also maintained in the summarizer 103 .
  • the whitelist may have the same structure as the blacklist, but the whitelist can implement a processing policy with priority that directly passes the access request that matches with the whitelist (i.e., in the case that the access request matches with one whitelist record in the whitelist), that is, the access request is guided to its destination backend server (host), and the backend server directly makes a response to the access request.
  • the summarizer 103 can asynchronously load the current blacklist and the predefined whitelist to the load balancer 101 .
  • a listening thread of the summarizer 103 can provide the blacklist to the load balancer 101 .
  • the listening thread acquires the current blacklist at the summarizer 103 , and sends the blacklist to the load balancer 101 along with the whitelist.
  • the load balancer 101 is an actual executor of the “processing action” specified by each record in the blacklist.
  • the whitelist is not necessarily maintained in the summarizer 103 , and can be also maintained, for example, in the load balancer 101 .
  • the load balancer 101 will pass the access request regardless of whether the access request matches with the blacklist. If the access request does not match with the whitelist but matches with the blacklist, the load balancer 101 performs the corresponding processing action according to the specific record matched in the blacklist. If the access request neither matches with the whitelist nor matches with the blacklist, the access request is passed.
  • the load balancer 101 performs the following processing for each access request from the network according to the blacklist and the whitelist:
  • the access request when the access request matches with the whitelist or does not match with the blacklist, the access request is passed, and the corresponding UDP message is constructed and sent to the aggregator (because only such passed traffic is indeed directed to the corresponding backend server, the aggregator needs to continue to calculate the “statistical data” of such access requests);
  • the access request when the access request does not match with the whitelist but matches with the blacklist, the access request is processed according to the processing action specified by the matched record in the blacklist.
  • FIG. 2 shows a method for limiting access requests from a network by a load balancer according to the invention.
  • step S 11 the load balancer receives an access request from the network and parses the received access request into a UDP message.
  • step S 12 the UDP message is transmitted to the aggregator.
  • the load balancer determines the processing action on the received access request according to the current whitelist and the current blacklist at the load balancer, wherein the blacklist specifies the processing actions on certain access requests.
  • step S 13 the load balancer makes a judgment whether the access request matches with the whitelist. If so, step S 14 is turned to, that is, the access request is passed and sent to a corresponding backend server for further process. Otherwise, the process proceeds to step S 15 .
  • step S 15 the load balancer makes a judgment whether the access request matches with the blacklist. If not, step S 14 is turned to, that is, the access request is passed and sent to the corresponding backend server for further process. Otherwise, the process proceeds to step S 16 .
  • step S 16 the load balancer processes the access request according to the processing action specified by the matched record in the blacklist.
  • the aggregator receives the UDP message sent by the load balancer, accumulates the received UDP messages according to the predefined combination of data fields and the predefined duration so as to produce a statistical data, and sends the statistical data to the summarizer in response to the request from the summarizer.
  • the statistical data includes the number of the UDP messages containing the predefined combination of data fields accumulated within the current period with a length of the predefined duration.
  • the summarizer requests the statistical data from the aggregator according to the predefined request time interval, receives the statistical data from the aggregator, generates a blacklist for access requests according to the received statistical data and the predefined rule, and sends the current blacklist at the summarizer to the load balancer according to a request from the load balancer.
  • the aggregator may comprise a plurality of aggregation units.
  • the process of performing the accumulation by each aggregation unit to produce the statistical data is described in detail below.
  • the aggregation unit produces the statistical data with respect to a combination of specific data fields according to the UDP messages received from the load balancer.
  • the aggregation unit comprises:
  • a receiving component for receiving the UDP message sent by the load balancer, the UDP message being produced by the load balancer parsing the access request from the network;
  • a calculating component that accumulates the received UDP messages according to the predefined combination of data fields and the predefined duration so as to produce a statistical data
  • a transmission component that sends the current statistical data to the summarizer according to a request from the summarizer.
  • Each HTTP request from the network it can be only parsed by the load balancer 101 into a UDP message and transmitted to one aggregation unit in a cluster of the aggregation units of the aggregator 102 .
  • Each aggregation unit accumulates the numbers of the access requests according to the predefined condition, for example, according to the combination of specific data fields such as the user identification (userid), the IP address, and the universal resource identifier (uri) (that is, the combination of the specific values contained in these specific data fields) in the UDP message, and the specified duration.
  • specific data fields such as the user identification (userid), the IP address, and the universal resource identifier (uri)
  • the aggregation unit For the received UDP message, the aggregation unit extracts the data fields such as userid, ip and uri from the UDP message, a specified combinations of these data fields (there may be a plurality of combinations) is used as a combination of data fields for accumulation, and the numbers of the requests are accumulated according to the predefined duration (which may include a short period and a long period).
  • the combination of data fields may also be a combination including other data fields.
  • the aggregation unit After the completion of the accumulation, the aggregation unit generates a statistical data, for example, in the form of a web page in a JSONS data format. When the request from the summarizer 103 is received, the aggregation unit transmits the statistical data to the summarizer 103 .
  • the current time when “the first UDP message” is received is used as a valid calculation start time
  • the number of the accesses requests is set to 1
  • the start time is set to the current time
  • the length of time lasting is set to 0:
  • lasting_length is the length of time lasting from the calculation start time (start_time) to the current time (current_time).
  • the start_time (valid calculation start time) will be updated, wherein the reception time of the message is used as an end of a new period with a length being equal to the predefined duration, and the start time of the new period (i.e., a time obtained by subtracting the predefined duration from the reception time of the message) is marked as the valid calculation start time; then the parameter total_count (i.e., the number of the access requests) is updated by firstly calculating the QPS for the first period, multiplying the QPS by a duration of the new period overlapping the first period, and then incrementing the result by 1; and the length of time lasting is updated to the predefined duration; and
  • the current reception time is set to the valid calculation start time, the number of the access requests is reset to 1, the length of time lasting is 0, that is, such case is regarded as an initialization, and then, the traffic will be recalculated.
  • the predefined duration can be set to both of the short period and the long period.
  • the predefined duration is set to the short period of time in order to prevent fast DOS attacks, that is, a large number of connection requests attack a server within a short time, so that all available resources are exhausted, and finally a computer cannot process requests from authenticated users any longer.
  • the predefined duration is set to the long period in order to prevent slow DOS attacks.
  • the summarizer 103 reads the statistical data relating to the number of the requests from all the aggregation units according to the specific classification, summarizes the statistical data, generates a blacklist according to a predefined policy, and asynchronously loads the blacklist to the load balancer 101 .
  • the summarizer 103 comprises a receiving component 1031 , a generating component 1032 , a listening component 1033 , a configuration file 1036 , and a database 1037 .
  • the receiving component 1031 can create a receiving thread for each aggregation unit, and the receiving thread reads the statistical data for the numbers of the requests with respect to different combinations of data fields from a Web page output by each aggregation unit every a period (e.g., one minute).
  • the statistical data are, for example, data in a JSONSG format.
  • the generating component 1032 of the summarizer 103 summarizes the statistical data received from the respective aggregation units according to predefined combinations of data fields.
  • These predefined combinations of data fields may be userid+ip+uri, ip+uri, userid+uri and the like.
  • Hashmap is an implementation of a Map interface based on a hash table, and is used for functions of storage, search and the like in computer programming.
  • blacklist threshold value relating to the combination of data fields predefined in the configuration file, for example 0.5 QPS (query per second)
  • userid or ip is added to the blacklist so as to produce the current blacklist.
  • the summarized data in the hashmap is stored into the database 1037 by a storing thread in the generating component 1032 .
  • a mechanism of double-hashmap is used in order not to lock the shared hashmap between the storing thread and the receiving thread.
  • a monitoring terminal 200 of the system can directly acquire the data from the database 1037 through HTTP protocol, that is, the data can be directly exported out in the JSONS format without through the summarizer 103 .
  • the listening component 1033 of the summarizer 103 is used to provide the blacklist to the load balancer 101 .
  • the listening component 1033 acquires the current blacklist, and sends the current blacklist to the load balancer 101 along with the whitelist.
  • the summarizer 103 may work in a mode in which two servers work together, wherein the master server may act as a normal operating machine, while the slave server may act as a data backup machine.
  • the salve server can automatically take over all of the tasks of the master server, thereby a high reliability of the system is ensured.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Computer And Data Communications (AREA)
US15/542,086 2015-01-09 2016-01-08 System and method for limiting access request Active 2037-05-12 US10735501B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201510011598 2015-01-09
CN201510011598.9A CN104580216B (zh) 2015-01-09 2015-01-09 一种对访问请求进行限制的系统和方法
CN201510011598.9 2015-01-09
PCT/CN2016/070522 WO2016110273A1 (zh) 2015-01-09 2016-01-08 一种对访问请求进行限制的系统和方法

Publications (2)

Publication Number Publication Date
US20180278678A1 US20180278678A1 (en) 2018-09-27
US10735501B2 true US10735501B2 (en) 2020-08-04

Family

ID=53095397

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/542,086 Active 2037-05-12 US10735501B2 (en) 2015-01-09 2016-01-08 System and method for limiting access request

Country Status (6)

Country Link
US (1) US10735501B2 (ja)
JP (2) JP2018508166A (ja)
CN (1) CN104580216B (ja)
HK (1) HK1204726A1 (ja)
RU (1) RU2666289C1 (ja)
WO (1) WO2016110273A1 (ja)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104580216B (zh) * 2015-01-09 2017-10-03 北京京东尚科信息技术有限公司 一种对访问请求进行限制的系统和方法
CN104580228A (zh) * 2015-01-16 2015-04-29 北京京东尚科信息技术有限公司 对来自网络的访问请求产生黑名单的系统和方法
CN105939320A (zh) * 2015-12-02 2016-09-14 杭州迪普科技有限公司 处理报文的方法及装置
CN107454120A (zh) * 2016-05-30 2017-12-08 北京京东尚科信息技术有限公司 网络攻击防御系统和防御网络攻击的方法
CN108683631B (zh) * 2018-03-30 2019-12-20 厦门白山耘科技有限公司 一种防止扫描权限文件的方法和系统
CN109241458A (zh) * 2018-07-11 2019-01-18 上海斐讯数据通信技术有限公司 一种基于路由器的广告拦截方法和路由器
CN109617932B (zh) * 2019-02-21 2021-07-06 北京百度网讯科技有限公司 用于处理数据的方法和装置
CN112953985B (zh) * 2019-12-10 2023-04-07 贵州白山云科技股份有限公司 请求数据处理方法、装置、介质及系统
CN113179317B (zh) * 2021-04-27 2023-02-07 杭州迪普科技股份有限公司 内容重写设备的测试系统及方法
CN113904839A (zh) * 2021-09-30 2022-01-07 杭州数梦工场科技有限公司 访问请求管理方法及装置
CN115174249B (zh) * 2022-07-18 2024-09-24 湖北天融信网络安全技术有限公司 安全日志的处理方法及电子设备、存储介质
CN115396376A (zh) * 2022-08-22 2022-11-25 平安科技（深圳）有限公司 负载均衡方法、装置、设备及存储介质

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7463590B2 (en) * 2003-07-25 2008-12-09 Reflex Security, Inc. System and method for threat detection and response
CN101437030A (zh) 2008-11-29 2009-05-20 成都市华为赛门铁克科技有限公司 一种防止服务器被攻击的方法、检测装置及监控设备
CN103746982A (zh) 2013-12-30 2014-04-23 中国科学院计算技术研究所 一种http网络特征码自动生成方法及其系统
US20140259147A1 (en) * 2011-09-29 2014-09-11 Israel L'Heureux Smart router
CN104104669A (zh) 2014-06-17 2014-10-15 上海地面通信息网络有限公司 适用于因特网数据中心领域的抗DDoS攻击防护系统
CN104580216A (zh) 2015-01-09 2015-04-29 北京京东尚科信息技术有限公司 一种对访问请求进行限制的系统和方法
CN104580228A (zh) 2015-01-16 2015-04-29 北京京东尚科信息技术有限公司 对来自网络的访问请求产生黑名单的系统和方法
CN104579841A (zh) 2015-01-09 2015-04-29 北京京东尚科信息技术有限公司 根据接收的udp报文产生对特定统计数据项的统计结果的系统
US9794272B2 (en) * 2006-01-03 2017-10-17 Alcatel Lucent Method and apparatus for monitoring malicious traffic in communication networks

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020133603A1 (en) * 2001-03-13 2002-09-19 Fujitsu Limited Method of and apparatus for filtering access, and computer product
FR2872983A1 (fr) * 2004-07-09 2006-01-13 Thomson Licensing Sa Systeme de pare-feu protegeant une communaute d'appareils, appareil participant au systeme et methode de mise a jour des regles de pare-feu au sein du systeme
US7478429B2 (en) * 2004-10-01 2009-01-13 Prolexic Technologies, Inc. Network overload detection and mitigation system and method
US8089871B2 (en) * 2005-03-25 2012-01-03 At&T Intellectual Property Ii, L.P. Method and apparatus for traffic control of dynamic denial of service attacks within a communications network
JP4602158B2 (ja) * 2005-05-25 2010-12-22 三菱電機株式会社 サーバ装置保護システム
JP2008135871A (ja) * 2006-11-27 2008-06-12 Oki Electric Ind Co Ltd ネットワーク監視システム、ネットワーク監視方法及びネットワーク監視プログラム
JP4900119B2 (ja) * 2007-08-01 2012-03-21 ヤマハ株式会社 ネットワーク機器
WO2009075007A1 (ja) * 2007-12-12 2009-06-18 Duaxes Corporation 通信制御装置及び通信制御方法
JP5142956B2 (ja) * 2008-11-20 2013-02-13 日本電信電話株式会社 トラフィック情報管理サーバ及びトラフィック情報管理方法
JP2011049794A (ja) * 2009-08-27 2011-03-10 Alaxala Networks Corp パケットフロー統計値取得システム及びパケットフロー統計値取得方法
US20110083179A1 (en) * 2009-10-07 2011-04-07 Jeffrey Lawson System and method for mitigating a denial of service attack using cloud computing
CN103491053A (zh) * 2012-06-08 2014-01-01 北京百度网讯科技有限公司 Udp负载均衡方法、系统及装置
US9553809B2 (en) * 2013-04-16 2017-01-24 Amazon Technologies, Inc. Asymmetric packet flow in a distributed load balancer
RU133954U1 (ru) * 2013-04-29 2013-10-27 Федеральное государственное образовательное бюджетное учреждение высшего профессионального образования "Санкт-Петербургский государственный университет телекоммуникаций им. проф. М.А. Бонч-Бруевича" (СПбГУТ) Устройство защиты сети
US9055095B2 (en) * 2013-06-14 2015-06-09 Microsoft Technology Licensing, Llc DOS detection and mitigation in a load balancer

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7463590B2 (en) * 2003-07-25 2008-12-09 Reflex Security, Inc. System and method for threat detection and response
US9794272B2 (en) * 2006-01-03 2017-10-17 Alcatel Lucent Method and apparatus for monitoring malicious traffic in communication networks
CN101437030A (zh) 2008-11-29 2009-05-20 成都市华为赛门铁克科技有限公司 一种防止服务器被攻击的方法、检测装置及监控设备
US20140259147A1 (en) * 2011-09-29 2014-09-11 Israel L'Heureux Smart router
CN103746982A (zh) 2013-12-30 2014-04-23 中国科学院计算技术研究所 一种http网络特征码自动生成方法及其系统
CN104104669A (zh) 2014-06-17 2014-10-15 上海地面通信息网络有限公司 适用于因特网数据中心领域的抗DDoS攻击防护系统
CN104580216A (zh) 2015-01-09 2015-04-29 北京京东尚科信息技术有限公司 一种对访问请求进行限制的系统和方法
CN104579841A (zh) 2015-01-09 2015-04-29 北京京东尚科信息技术有限公司 根据接收的udp报文产生对特定统计数据项的统计结果的系统
CN104580228A (zh) 2015-01-16 2015-04-29 北京京东尚科信息技术有限公司 对来自网络的访问请求产生黑名单的系统和方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
International Search Report with English translation and Written Opinion for Application No. PCT/CN2016/070522 dated Apr. 13, 2016 (11 pages).

Also Published As

Publication number Publication date
US20180278678A1 (en) 2018-09-27
RU2666289C1 (ru) 2018-09-06
CN104580216A (zh) 2015-04-29
HK1204726A1 (en) 2015-11-27
CN104580216B (zh) 2017-10-03
JP6726331B2 (ja) 2020-07-22
WO2016110273A1 (zh) 2016-07-14
JP2018508166A (ja) 2018-03-22
JP2019134484A (ja) 2019-08-08

Similar Documents

Publication Publication Date Title
US10735501B2 (en) System and method for limiting access request
US11641343B2 (en) Methods and systems for API proxy based adaptive security
US10257224B2 (en) Method and apparatus for providing forensic visibility into systems and networks
US7623466B2 (en) Symmetric connection detection
US10498618B2 (en) Attributing network address translation device processed traffic to individual hosts
US10218733B1 (en) System and method for detecting a malicious activity in a computing environment
US10091198B2 (en) Rule-based fingerprint generation methods and apparatus
US20050278779A1 (en) System and method for identifying the source of a denial-of-service attack
KR20110089179A (ko) 네트워크 침입 방지
CN112311722B (zh) 一种访问控制方法、装置、设备及计算机可读存储介质
US20220174072A1 (en) Data Processing Method and Device
CN104580228A (zh) 对来自网络的访问请求产生黑名单的系统和方法
Sen A robust mechanism for defending distributed denial of service attacks on web servers
Salim et al. Preventing ARP spoofing attacks through gratuitous decision packet
KR20100046524A (ko) 유해 사이트 차단 장치 및 방법
CN104579841B (zh) 根据接收的udp报文产生对特定统计数据项的统计结果的系统
Bellaïche et al. SYN flooding attack detection by TCP handshake anomalies
CN111431942A (zh) 一种cc攻击的检测方法、装置及网络设备
US11757929B2 (en) Traffic-shaping HTTP proxy for denial-of-service protection
CN117395023A (zh) 面向加密网关的网络设备识别方法和装置
Limmer Efficient Network Monitoring for Attack Detection

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD., CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WENG, ZHI;XIAO, SIXING;REEL/FRAME:047267/0576

Effective date: 20170808

Owner name: BEIJING JINGDONG CENTURY TRADING CO., LTD., CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WENG, ZHI;XIAO, SIXING;REEL/FRAME:047267/0576

Effective date: 20170808

Owner name: BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WENG, ZHI;XIAO, SIXING;REEL/FRAME:047267/0576

Effective date: 20170808

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4