CN106911582B - real-time flow control method and system based on Nginx server - Google Patents
real-time flow control method and system based on Nginx server Download PDFInfo
- Publication number
- CN106911582B CN106911582B CN201710130843.7A CN201710130843A CN106911582B CN 106911582 B CN106911582 B CN 106911582B CN 201710130843 A CN201710130843 A CN 201710130843A CN 106911582 B CN106911582 B CN 106911582B
- Authority
- CN
- China
- Prior art keywords
- rate
- preset threshold
- service
- access
- accessing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0894—Packet rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/29—Flow control; Congestion control using a combination of thresholds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses real-time flow control methods based on Nginx servers, which comprise the steps of obtaining the total speed of a news access client, judging whether the obtained total speed of the news access client is greater than a preset threshold of the total speed of the news access client, generating a th control signal when the obtained total speed of the news access client is greater than the preset threshold of the total speed of the news access client, and limiting the total speed of the news access client based on a th control signal.
Description
Technical Field
The invention relates to the technical field of flow monitoring, in particular to real-time flow control methods and systems based on a Nginx server.
Background
Nginx is high-performance HTTP (Hypertext Transfer Protocol) and a reverse proxy server, and is widely applied in in the industry, the traffic limitation is good choices in Nginx, and an instruction issued by a traffic control module provided by the government of Nginx can control the traffic to a certain extent .
Disclosure of Invention
The invention provides real-time flow control methods based on Nginx servers, which can effectively monitor the flow in real time.
The invention provides real-time flow control methods based on Nginx servers, which are characterized by comprising the following steps:
acquiring the total rate of accessing the news client;
judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
limiting a total rate of access to news clients based on the th control signal.
Preferably, the method further comprises:
obtaining the access rate of the IP address;
judging whether the access rate of the acquired IP address is greater than a preset threshold of the access rate of the IP address;
when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal;
limiting an access rate of an IP address based on the second control signal.
Preferably, the method further comprises:
acquiring the rate of each service; judging whether the acquired rate of each service is greater than a preset threshold of the rate of each service;
when the acquired rate of each service is greater than a preset threshold value of the rate of each service, generating a third control signal;
limiting a rate of each traffic based on the third control signal.
Preferably, the method further comprises:
acquiring the rate of each user accessing the service;
judging whether the obtained rate of each user for accessing the service is greater than a preset threshold of the rate of each user for accessing the service;
when the obtained rate of each user accessing the service is greater than a preset threshold value of the rate of each user accessing the service, generating a fourth control signal;
limiting a rate at which each user accesses traffic based on the fourth control signal.
Preferably, the method further comprises:
every n minutes, respectively acquiring an IP address and historical data of each user access service from the Nginx server;
for the time interval [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]If theta is smaller than thetaminStopping if not:
computing
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of if
And if the access rate is greater than the preset threshold of the access rate of each network protocol or the preset threshold of the rate of each user for accessing the service, the preset threshold of the access rate of the IP address or the preset threshold of the rate of each user for accessing the service is set by sending a hypertext transfer protocol request through the webpage platform based on the Nginx server and the script language.
A real-time flow control system based on Nginx server, comprising:
the real-time traffic monitoring module based on the Nginx server is used for acquiring the total rate of accessing the news client;
the dynamic real-time flow control module is used for judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
limiting a total rate of access to news clients based on the th control signal.
Preferably, the real-time traffic monitoring module based on the Nginx server is further configured to obtain an access rate of an IP address;
the dynamic real-time flow control module is also used for judging whether the access rate of the acquired IP address is greater than a preset threshold value of the access rate of the IP address;
when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal;
limiting an access rate of an IP address based on the second control signal.
Preferably, the real-time traffic monitoring module based on the Nginx server is further configured to obtain a rate of each service;
the dynamic real-time flow control module is also used for judging whether the acquired rate of each service is greater than a preset threshold value of the rate of each service;
when the acquired rate of each service is greater than a preset threshold value of the rate of each service, generating a third control signal;
limiting a rate of each traffic based on the third control signal.
Preferably, the real-time traffic monitoring module based on the Nginx server is further configured to obtain a rate at which each user accesses a service;
the dynamic real-time flow control module is also used for judging whether the obtained rate of each user for accessing the service is greater than a preset threshold value of the rate of each user for accessing the service;
when the obtained rate of each user accessing the service is greater than a preset threshold value of the rate of each user accessing the service, generating a fourth control signal;
limiting a rate at which each user accesses traffic based on the fourth control signal.
Preferably, the system further comprises:
the real-time traffic monitoring module based on the Nginx server is further used for respectively acquiring the IP address and historical data of each user access service from the Nginx server every n minutes;
the dynamic real-time flow control module is also used for controlling time intervals [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]The slope θ of (a); if theta is greater than or equal to thetaminTime, calculate
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of ifAnd if the access rate of each network protocol is greater than the preset threshold of the access rate of each network protocol or the preset threshold of the access rate of each user to the service, sending a hypertext transfer protocol request to set the preset threshold of the access rate of each network protocol or the preset threshold of the access rate of each user to the service by a web platform based on the Nginx server and the scripting language.
According to the scheme, when the flow needs to be monitored in real time, the real-time flow control method based on the Nginx server firstly obtains the total rate of accessing the news client, then judges whether the obtained total rate of accessing the news client is greater than the preset threshold of the total rate of accessing the news client, generates a control signal when the obtained total rate of accessing the news client is greater than the preset threshold of the total rate of accessing the news client, and finally limits the total rate of accessing the news client based on a control signal, so that the flow can be monitored in real time more effectively compared with the prior art.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a method in embodiment 1 of real-time traffic control methods based on a Nginx server according to the present disclosure;
FIG. 2 is a flow chart of a method of embodiment 2 of Nginx server-based real-time traffic control methods disclosed in the present invention;
FIG. 3 is a flow chart of the method of embodiment 3 of the Nginx server-based real-time traffic control method disclosed in the present invention;
FIG. 4 is a flow chart of a method of embodiment 4 of Nginx server-based real-time traffic control methods disclosed in the present invention;
fig. 5 is a flowchart of methods for setting a preset threshold of access rate per network protocol or a preset threshold of access rate per user for services according to the present invention;
fig. 6 is a schematic structural diagram of an embodiment 1 of real-time traffic control systems based on a Nginx server disclosed in the present invention;
fig. 7 is a schematic structural diagram of embodiment 2 of real-time traffic control systems based on a Nginx server according to the present invention;
fig. 8 is a schematic structural diagram of embodiment 3 of real-time traffic control systems based on the Nginx server disclosed in the present invention;
fig. 9 is a schematic structural diagram of embodiment 4 of real-time traffic control systems based on a Nginx server according to the present invention;
fig. 10 is a schematic structural diagram of preset thresholds for setting the access rate of each network protocol or the rate of each user accessing the service, which are disclosed by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only partial embodiments of of the present invention, rather than all embodiments.
For example, a module or cell may be implemented in a hardware circuit comprising custom VLSI circuits or arrays, such as logic chips, transistors, or other components.
For example, a module of executable code may include or more physical or logical blocks of computer instructions which may be formed, for example, as objects, programs, or functions, however, the executables of an identified module or unit need not be physically located at , but may comprise disparate instructions stored in different locations which, when logically combined at , form the module or unit and achieve the desired purpose for the module or unit.
Similarly, operational data may be identified and displayed within such modules or units, and may be embodied in any suitable form and organized within any suitable data structure.
Thus, appearances of the phrases "in an embodiment ," "in an embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.
Furthermore, the following description provides many specific details, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc., to provide a thorough understanding of embodiments of the invention.
As shown in fig. 1, the method is a flowchart of embodiment 1 of real-time traffic control methods based on an Nginx server, and the method includes:
s101, acquiring the total rate of accessing a news client;
s102, judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
S103, when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
if the requested rate exceeds a preset threshold, total _ rate, a th control signal is generated to limit the request.
And S104, limiting the total rate of accessing the news client based on the th control signal.
The total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
In summary, in the above embodiment, when the traffic needs to be monitored in real time, first, the total rate of accessing the news client is obtained, then, whether the obtained total rate of accessing the news client is greater than the total rate preset threshold of accessing the news client is determined, when it is determined that the obtained total rate of accessing the news client is greater than the total rate preset threshold of accessing the news client, the -th control signal is generated, and finally, the total rate of accessing the news client is limited based on the -th control signal, which can more effectively monitor the traffic in real time compared with the prior art.
As shown in fig. 2, a flowchart of a method in embodiment 2 of real-time traffic control methods based on an Nginx server disclosed in the present invention is shown, where the method includes:
s201, acquiring the total rate of accessing a news client;
s202, judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
S203, when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
if the requested rate exceeds a preset threshold, total _ rate, a th control signal is generated to limit the request.
S204, limiting the total rate of accessing the news client based on the th control signal;
the total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
S205, obtaining the access rate of the IP address;
s206, judging whether the access rate of the acquired IP address is greater than a preset threshold of the access rate of the IP address;
in order to prevent malicious users from simulating unique identifications of multiple users to access a certain service (API), the obtained service traffic information further includes the access rate of the IP address, and whether the obtained access rate of the IP address is greater than the preset threshold IP _ rate of the access rate of the IP address is judged.
S207, when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal;
and when the obtained access rate of the IP address is judged to be greater than the preset threshold value IP _ rate of the access rate of the IP address, generating a second control signal.
And S207, limiting the access rate of the IP address based on the second control signal.
And limiting the access rate of the IP address based on the generated second control signal.
In summary, in this embodiment, step is based on the above embodiments to implement limitation on the access rate of the IP address.
As shown in fig. 3, a flowchart of a method in embodiment 3 of real-time traffic control methods based on a Nginx server disclosed in the present invention is shown, where the method includes:
s301, acquiring the total rate of accessing the news client;
s302, judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
S303, when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
if the requested rate exceeds a preset threshold, total _ rate, a th control signal is generated to limit the request.
S304, limiting the total rate of accessing the news client based on the th control signal;
the total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
S305, obtaining the access rate of the IP address;
s306, judging whether the access rate of the acquired IP address is greater than a preset threshold of the access rate of the IP address;
in order to prevent malicious users from simulating unique identifications of multiple users to access a certain service (API), the obtained service traffic information further includes the access rate of the IP address, and whether the obtained access rate of the IP address is greater than the preset threshold IP _ rate of the access rate of the IP address is judged.
S307, when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal;
and when the obtained access rate of the IP address is judged to be greater than the preset threshold value IP _ rate of the access rate of the IP address, generating a second control signal.
S308, limiting the access rate of the IP address based on the second control signal;
and limiting the access rate of the IP address based on the generated second control signal.
S309, acquiring the rate of each service;
s310, judging whether the acquired rate of each service is greater than a preset threshold of the rate of each service;
the traffic of the whole news client is limited in the above embodiment, and it cannot be guaranteed that every services will normally operate, because the pressure born by each service is not , even if the request rate of the whole news client does not exceed the total _ rate × N, the threshold API _ total of some services may be exceeded, which may cause the influence on the service.
S311, when the rate of each acquired service is greater than a preset threshold of the rate of each service, generating a third control signal;
and S312, limiting the rate of each service based on the third control signal.
Therefore, in this embodiment, step is further performed to determine whether the obtained rate of each service is greater than a preset threshold API _ total of the rate of each service, and on the Nginx server, the API is accessed times, the count of the accessed API is added to , and the API is stored in the database Redis.
As shown in fig. 4, a flowchart of a method in embodiment 4 of real-time traffic control methods based on an Nginx server disclosed in the present invention is shown, where the method includes:
s401, acquiring the total rate of accessing a news client;
s402, judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
S403, when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
if the requested rate exceeds a preset threshold, total _ rate, a th control signal is generated to limit the request.
S404, limiting the total rate of accessing the news client based on the th control signal;
the total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
S405, obtaining the access rate of the IP address;
s406, judging whether the access rate of the acquired IP address is greater than a preset threshold of the access rate of the IP address;
in order to prevent malicious users from simulating unique identifications of multiple users to access a certain service (API), the obtained service traffic information further includes the access rate of the IP address, and whether the obtained access rate of the IP address is greater than the preset threshold IP _ rate of the access rate of the IP address is judged.
S407, when the access rate of the acquired IP address is greater than the preset threshold of the access rate of the IP address, generating a second control signal;
and when the obtained access rate of the IP address is judged to be greater than the preset threshold value IP _ rate of the access rate of the IP address, generating a second control signal.
S408, limiting the access rate of the IP address based on the second control signal;
and limiting the access rate of the IP address based on the generated second control signal.
S409, acquiring the rate of each service;
the traffic of the whole news client is limited in the above embodiment, and it cannot be guaranteed that every services will normally operate, because the pressure born by each service is not , even if the request rate of the whole news client does not exceed the total _ rate × N, the threshold API _ total of some services may be exceeded, which may cause the influence on the service.
Therefore, in this embodiment, step is further performed to determine whether the obtained rate of each service is greater than a preset threshold API _ total of the rate of each service, and on the Nginx server, the API is accessed times, the count of the accessed API is added to , and the API is stored in the database Redis.
S410, judging whether the acquired rate of each service is greater than a preset threshold of the rate of each service;
the traffic of the whole news client is limited in the above embodiment, and it cannot be guaranteed that every services will normally operate, because the pressure born by each service is not , even if the request rate of the whole news client does not exceed the total _ rate × N, the threshold API _ total of some services may be exceeded, which may cause the influence on the service.
S411, when the rate of each acquired service is greater than a preset threshold of the rate of each service, generating a third control signal;
s412, limiting the rate of each service based on the third control signal;
s413, acquiring the rate of each user for accessing the service;
s414, judging whether the acquired rate of each user accessing the service is greater than a preset threshold of the rate of each user accessing the service;
step is performed to determine whether the obtained rate of each user accessing the service is greater than a preset threshold CID _ API _ rate of the rate of each user accessing the service.
S415, when the obtained rate of each user for accessing the service is greater than a preset threshold value of the rate of each user for accessing the service, generating a fourth control signal;
and S416, limiting the rate of the service access of each user based on the fourth control signal.
In summary, in the above-described embodiments, a limitation on the total rate of accessing the news client, a limitation on the access rate per IP, a rate limitation per service, and a limitation on the rate of accessing the API per user can be achieved.
The invention can also realize the initialization of dynamic setting of the rate parameter, the TOTAL _ rate and the API _ rate, and the two values are not changed as usual according to the result of the pressure test of the whole system and a single service, and if the system or the service is upgraded, the system and the service can be modified through the web platform, so that the system and the service can be provided reasonably.
The initialization of the parameters IP _ rate and CID _ API _ rate is set according to empirical values, the initialized value is not , and needs to be dynamically adjusted according to the situation that each IP and user access the API, otherwise part of the users will be lost, the step of dynamically setting the parameters IP _ rate and CID _ API _ rate is shown in fig. 5, and includes the following steps:
s501, acquiring historical data of each network protocol and each user access service from the Nginx server at intervals of n minutes;
the Web platform obtains the historical data of the IP and CID _ API from the Redis server every n minutes, respectively.
S502, for time interval [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]If theta is smaller than thetaminIf not, the process goes to S503:
for the time interval [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]If theta is smaller than thetaminThen stop, otherwise go to step S503.
S503, calculating
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of ifThe preset threshold value of the access rate of each network protocol or the preset threshold value of the access rate of each user to the service is larger than the preset threshold value of the access rate of each network protocol, and the preset threshold value is based on the Nginx server and the script languageThe web page platform sends a hypertext transfer protocol request to set a preset threshold value of the access rate of each network protocol or a preset threshold value of the access rate of each user to the service.
Computing
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of if
If the current parameter is larger than the IP _ rate, the value of the IP _ rate is set by sending an http request through the web platform, and the setting of the parameters can be effective in real time without restarting or reloading the Nginx server cluster for the CID _ API _ rate.
As shown in fig. 6, which is a schematic structural diagram of embodiment 1 of real-time traffic control systems based on a Nginx server disclosed in the present invention, the system includes:
the real-time traffic monitoring module 601 based on the Nginx server is used for acquiring the total rate of accessing the news client;
the dynamic real-time flow control module 602 is configured to determine whether the obtained total rate of accessing the news client is greater than a preset total rate threshold of accessing the news client, generate -th control signals when the obtained total rate of accessing the news client is greater than the preset total rate threshold of accessing the news client, and limit the total rate of accessing the news client based on the -th control signal.
The method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
If the requested rate exceeds the preset threshold total _ rate, a th control signal is generated to limit the request
The total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
In summary, in the above embodiment, when the traffic needs to be monitored in real time, first, the real-time traffic monitoring module of the Nginx server obtains the total rate of accessing the news client, then determines whether the obtained total rate of accessing the news client is greater than the total rate preset threshold of accessing the news client, generates the -th control signal when it is determined that the obtained total rate of accessing the news client is greater than the total rate preset threshold of accessing the news client, and finally limits the total rate of accessing the news client based on the -th control signal, which can more effectively monitor the traffic in real time compared with the prior art.
As shown in fig. 7, a schematic structural diagram of embodiment 2 of real-time traffic control systems based on a Nginx server disclosed in the present invention is shown, and the system includes:
the real-time traffic monitoring module 701 based on the Nginx server is used for acquiring the total rate of accessing the news client;
the dynamic real-time flow control module 702 is used for judging whether the total rate of the obtained news access client is greater than a total rate preset threshold value of the news access client or not, generating th control signals when the total rate of the obtained news access client is greater than the total rate preset threshold value of the news access client, and limiting the total rate of the news access client based on th control signals;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
If the requested rate exceeds the preset threshold total _ rate, a th control signal is generated to limit the request
The total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
The service flow information further includes an access rate of each network protocol, and the second determining module 705 is configured to determine whether the obtained access rate of each network protocol is greater than a preset threshold of the access rate of each network protocol;
in order to prevent malicious users from simulating unique identifications of multiple users to access a certain service (API), the obtained service traffic information further includes an access rate of each network protocol, and whether the obtained access rate of each network protocol is greater than an access rate preset threshold IP _ rate of each network protocol is determined.
The real-time traffic monitoring module 701 based on the Nginx server is further configured to obtain an access rate of the IP address;
the dynamic real-time flow control module 702 is further configured to generate a second control signal when the access rate of the obtained IP address is greater than a preset threshold of the access rate of the IP address; the access rate of the IP address is limited based on the second control signal.
And when the obtained access rate of the IP address is judged to be greater than the preset threshold value IP _ rate of the access rate of the IP address, generating a second control signal.
And limiting the access rate of the IP address based on the generated second control signal.
In summary, in this embodiment, step is based on the above embodiments to implement limitation on the access rate of the IP address.
As shown in fig. 8, which is a schematic structural diagram of embodiment 3 of real-time traffic control systems based on a Nginx server disclosed in the present invention, the system includes:
the real-time traffic monitoring module 801 based on the Nginx server is used for acquiring the total rate of accessing the news client;
the dynamic real-time flow control module 802 is used for judging whether the total rate of the obtained news accessing client is greater than a total rate preset threshold of the news accessing client or not, generating th control signals when the total rate of the obtained news accessing client is greater than the total rate preset threshold of the news accessing client, and generating th control signals when the total rate of the obtained news accessing client is greater than the total rate preset threshold of the news accessing client;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
If the requested rate exceeds the preset threshold total _ rate, a th control signal is generated to limit the request
The total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
The service flow information further includes an access rate of each network protocol, and the second determining module 805 is configured to determine whether the obtained access rate of each network protocol is greater than a preset threshold of the access rate of each network protocol;
in order to prevent malicious users from simulating unique identifications of multiple users to access a certain service (API), the obtained service traffic information further includes an access rate of each network protocol, and whether the obtained access rate of each network protocol is greater than an access rate preset threshold IP _ rate of each network protocol is determined.
The real-time traffic monitoring module 801 based on the Nginx server is further configured to obtain an access rate of the IP address;
the dynamic real-time flow control module 802 is further configured to generate a second control signal when the access rate of the obtained IP address is greater than a preset threshold of the access rate of the IP address; limiting an access rate of the IP address based on the second control signal;
and when the obtained access rate of the IP address is judged to be greater than the preset threshold value IP _ rate of the access rate of the IP address, generating a second control signal.
And limiting the access rate of the IP address based on the generated second control signal.
The real-time traffic monitoring module 801 based on the Nginx server is further configured to obtain a rate of each service;
the dynamic real-time traffic control module 802 is further configured to determine whether the obtained rate of each service is greater than a preset threshold of the rate of each service; when the acquired rate of each service is greater than a preset threshold value of the rate of each service, generating a third control signal; limiting a rate of each traffic based on the third control signal.
The traffic of the whole news client is limited in the above embodiment, and it cannot be guaranteed that every services will normally operate, because the pressure born by each service is not , even if the request rate of the whole news client does not exceed the total _ rate × N, the threshold API _ total of some services may be exceeded, which may cause the influence on the service.
Therefore, in this embodiment, step is further performed to determine whether the obtained rate of each service is greater than a preset threshold API _ total of the rate of each service, and on the Nginx server, the API is accessed times, the count of the accessed API is added to , and the API is stored in the database Redis.
As shown in fig. 9, which is a schematic structural diagram of embodiment 4 of real-time traffic control systems based on a Nginx server disclosed in the present invention, the system includes:
a real-time traffic monitoring module 901 based on the Nginx server, configured to obtain a total rate of accessing the news client;
the dynamic real-time flow control module 902 is used for judging whether the total rate of the obtained news accessing client is greater than a preset threshold of the total rate of the news accessing client or not, generating th control signals when the obtained total rate of the news accessing client is greater than the preset threshold of the total rate of the news accessing client, and limiting the total rate of the news accessing client based on th control signals;
the method comprises the steps of forwarding a request of a client to a Nginx server cluster of a news client through load balancing equipment of a company, preventing the traffic of the whole news client from being too large due to overhigh normal traffic or malicious attack, influencing services and causing unnecessary loss, limiting the traffic on every Nginx servers, and judging whether the obtained total rate of accessing the news client is greater than the total rate preset threshold total _ rate of accessing the news client.
If the requested rate exceeds a preset threshold, total _ rate, a th control signal is generated to limit the request.
The total rate of access to news clients is limited by the control signal so that the rate of requests arriving at the backend is kept within 0, total _ rate N, where N is the number of Nginx servers.
The real-time traffic monitoring module 901 based on the Nginx server is further configured to obtain an access rate of the IP address;
the dynamic real-time flow control module 902 is further configured to determine whether the access rate of the obtained IP address is greater than a preset threshold of the access rate of the IP address; when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal; limiting an access rate of the IP address based on the second control signal;
in order to prevent malicious users from simulating unique identifications of multiple users to access a certain service (API), the obtained service traffic information further includes the access rate of the IP address, and whether the obtained access rate of the IP address is greater than the preset threshold IP _ rate of the access rate of the IP address is judged.
And when the obtained access rate of the IP address is judged to be greater than the preset threshold value IP _ rate of the access rate of the IP address, generating a second control signal.
And limiting the access rate of the IP address based on the generated second control signal.
The real-time traffic monitoring module 901 based on the Nginx server is further configured to obtain a rate of each service;
the dynamic real-time flow control module 902 is further configured to determine whether the obtained rate of each service is greater than a preset threshold of the rate of each service; when the acquired rate of each service is greater than a preset threshold value of the rate of each service, generating a third control signal; limiting a rate of each service based on a third control signal;
the traffic of the whole news client is limited in the above embodiment, and it cannot be guaranteed that every services will normally operate, because the pressure born by each service is not , even if the request rate of the whole news client does not exceed the total _ rate × N, the threshold API _ total of some services may be exceeded, which may cause the influence on the service.
Therefore, in this embodiment, step is further performed to determine whether the obtained rate of each service is greater than a preset threshold API _ total of the rate of each service, and on the Nginx server, the API is accessed times, the count of the accessed API is added to , and the API is stored in the database Redis.
The real-time traffic monitoring module 901 based on the Nginx server is further configured to obtain a rate at which each user accesses a service;
the dynamic real-time flow control module 902 is further configured to determine whether the obtained rate at which each user accesses the service is greater than a preset threshold of the rate at which each user accesses the service; when the obtained rate of each user accessing the service is greater than a preset threshold value of the rate of each user accessing the service, generating a fourth control signal; the rate at which each user accesses the service is limited based on the fourth control signal.
Step is performed to determine whether the obtained rate of each user accessing the service is greater than a preset threshold CID _ API _ rate of the rate of each user accessing the service.
In summary, in the above-described embodiments, a limitation on the total rate of accessing the news client, a limitation on the access rate per IP, a rate limitation per service, and a limitation on the rate of accessing the API per user can be achieved.
The invention can also realize the initialization of dynamic setting of the rate parameter, the TOTAL _ rate and the API _ rate, and the two values are not changed as usual according to the result of the pressure test of the whole system and a single service, and if the system or the service is upgraded, the system and the service can be modified through the web platform, so that the system and the service can be provided reasonably.
The initialization of the parameters IP _ rate and CID _ API _ rate is set according to empirical values, the initialized value is not , and needs to be dynamically adjusted according to the situation that each IP and user access the API, otherwise part of the users will be lost, the step of dynamically setting the parameters IP _ rate and CID _ API _ rate is shown in fig. 10, and includes:
the real-time traffic monitoring module 1001 based on the nginnx server is further configured to obtain the IP address and the historical data of the service accessed by each user from the nginnx server at intervals of n minutes;
the Web platform obtains the historical data of the IP and CID _ API from the Redis server every n minutes, respectively.
A dynamic real-time traffic control module 1002, configured to control a time interval [ new _ ts-interval, new _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]If theta is smaller than thetaminStopping if not, otherwise calculating
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of ifSending a hypertext transfer protocol (HTTP) through a web platform based on a Nginx server and a scripting language, wherein the HTTP is larger than a preset threshold of access rate of each network protocol or a preset threshold of access rate of each user to servicesRequesting to set a preset threshold of an access rate of each network protocol or a preset threshold of a rate of each user accessing the service.
For the time interval [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]If theta is smaller than thetaminThen, it stops.
Computing
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of if
If the current parameter is larger than the IP _ rate, the value of the IP _ rate is set by sending an http request through the web platform, and the setting of the parameters can be effective in real time without restarting or reloading the Nginx server cluster for the CID _ API _ rate.
Based on the understanding, a part of the method or a part of the technical solution that contributes to the prior art may be embodied in the form of a software product, which is stored in storage media and includes several instructions for making computing devices (which may be personal computers, servers, mobile computing devices, or network devices) execute all or part of the steps of the method described in each embodiment of the present invention.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention.
Claims (2)
1, A real-time flow control method based on Nginx server, characterized by comprising:
acquiring the total rate of accessing the news client;
judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
limiting a total rate of access to news clients based on the th control signal;
further comprising:
every n minutes, respectively acquiring an IP address and historical data of each user access service from the Nginx server;
for the time interval [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]If theta is smaller than thetaminStopping if not:
computingThe value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of if
The preset threshold value of the access rate of each network protocol or the preset threshold value of the access rate of each user to the service is larger than the preset threshold value of the access rate of each network protocol, and the preset threshold value is obtained by the Nginx-based serverA hypertext transfer protocol request is sent to a webpage platform of a script language to set a preset threshold value of the access rate of an IP address or a preset threshold value of the rate of each user for accessing services; the network protocol is an IP protocol;
the method further comprises the following steps:
obtaining the access rate of the IP address;
judging whether the access rate of the acquired IP address is greater than a preset threshold of the access rate of the IP address;
when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal;
limiting an access rate of an IP address based on the second control signal;
the method further comprises the following steps:
acquiring the rate of each service; judging whether the acquired rate of each service is greater than a preset threshold of the rate of each service;
when the acquired rate of each service is greater than a preset threshold value of the rate of each service, generating a third control signal;
limiting a rate of each traffic based on the third control signal;
the method further comprises the following steps:
acquiring the rate of each user accessing the service;
judging whether the obtained rate of each user for accessing the service is greater than a preset threshold of the rate of each user for accessing the service;
when the obtained rate of each user accessing the service is greater than a preset threshold value of the rate of each user accessing the service, generating a fourth control signal;
limiting a rate at which each user accesses traffic based on the fourth control signal.
The real-time flow control system based on the Nginx server is characterized by comprising the following components in parts by weight:
the real-time traffic monitoring module based on the Nginx server is used for acquiring the total rate of accessing the news client;
the dynamic real-time flow control module is used for judging whether the total rate of the obtained news access client is greater than a preset threshold of the total rate of the news access client;
when the obtained total rate of accessing the news client is greater than a preset threshold value of the total rate of accessing the news client, th control signals are generated;
limiting a total rate of access to news clients based on the th control signal;
the real-time traffic monitoring module is also used for respectively acquiring the IP address and historical data of each user access service from the Nginx server every n minutes;
the dynamic real-time flow control module is also used for controlling time intervals [ now _ ts-interval, now _ ts]Fitting the historical data to obtain a fitting function f (x), and judging whether f (x) is in the interval (now _ ts-interval, now _ ts)]The slope θ of (a); if theta is greater than or equal to thetaminTime, calculate
The value of (1), i.e. the interval [ now _ ts-interval, now _ ts]Median value of if
The access rate preset threshold of each network protocol or the speed of each user accessing the service is larger than the preset threshold of the access rate of each network protocol or the preset threshold of the speed of each user accessing the service, and a hypertext transfer protocol request is sent by a webpage platform based on an Nginx server and a script language to set the preset threshold of the access rate of each network protocol or the preset threshold of the speed of each user accessing the service; the network protocol is an IP protocol;
the real-time traffic monitoring module based on the Nginx server is also used for acquiring the access rate of the IP address;
the dynamic real-time flow control module is also used for judging whether the access rate of the acquired IP address is greater than a preset threshold value of the access rate of the IP address;
when the access rate of the acquired IP address is greater than the preset threshold value of the access rate of the IP address, generating a second control signal;
limiting an access rate of an IP address based on the second control signal;
the real-time traffic monitoring module based on the Nginx server is also used for acquiring the rate of each service;
the dynamic real-time flow control module is also used for judging whether the acquired rate of each service is greater than a preset threshold value of the rate of each service;
when the acquired rate of each service is greater than a preset threshold value of the rate of each service, generating a third control signal;
limiting a rate of each traffic based on the third control signal;
the real-time traffic monitoring module based on the Nginx server is also used for acquiring the rate of each user for accessing the service;
the dynamic real-time flow control module is also used for judging whether the obtained rate of each user for accessing the service is greater than a preset threshold value of the rate of each user for accessing the service;
when the obtained rate of each user accessing the service is greater than a preset threshold value of the rate of each user accessing the service, generating a fourth control signal;
limiting a rate at which each user accesses traffic based on the fourth control signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710130843.7A CN106911582B (en) | 2017-03-07 | 2017-03-07 | real-time flow control method and system based on Nginx server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710130843.7A CN106911582B (en) | 2017-03-07 | 2017-03-07 | real-time flow control method and system based on Nginx server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106911582A CN106911582A (en) | 2017-06-30 |
| CN106911582B true CN106911582B (en) | 2020-01-31 |
Family
ID=59186195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710130843.7A Active CN106911582B (en) | 2017-03-07 | 2017-03-07 | real-time flow control method and system based on Nginx server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106911582B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108234342B (en) * | 2018-01-25 | 2021-08-13 | 北京搜狐新媒体信息技术有限公司 | Nginx dynamic active current limiting method and system based on equipment fingerprint |
| CN108234341B (en) * | 2018-01-25 | 2021-06-11 | 北京搜狐新媒体信息技术有限公司 | Nginx dynamic passive current limiting method and system based on equipment fingerprint |
| CN108156088B (en) * | 2018-02-02 | 2021-04-23 | 北京盛科维科技发展有限公司 | Method and system for controlling flow of message sending screen and computer storage medium |
| CN108418764A (en) * | 2018-02-07 | 2018-08-17 | 深圳壹账通智能科技有限公司 | Current-limiting method, device, computer equipment and storage medium |
| CN110417671B (en) * | 2019-07-31 | 2023-01-06 | 中国工商银行股份有限公司 | Current limiting method and server for data transmission |
| CN110971534B (en) * | 2019-11-13 | 2023-05-05 | 哈尔滨哈工智慧嘉利通科技股份有限公司 | Uplink rate regulation and control method and device for government affair public opinion |
| CN113765813A (en) * | 2020-09-24 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Method and device for dynamically adjusting message receiving rate |
| CN112134811B (en) * | 2020-09-30 | 2022-08-09 | 安徽极玩云科技有限公司 | CDN cloud platform flow scheduling method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127632A (en) * | 2006-08-15 | 2008-02-20 | 腾讯科技（深圳）有限公司 | A method, system and redirector for server traffic control |
| CN102014109A (en) * | 2009-09-08 | 2011-04-13 | 华为技术有限公司 | Flood attack prevention method and device |
| CN103379090A (en) * | 2012-04-12 | 2013-10-30 | 腾讯科技（北京）有限公司 | Open platform access frequency control method and system and frequency server |
| CN103944833A (en) * | 2013-01-21 | 2014-07-23 | 中兴通讯股份有限公司 | Method and device for flow control, and flow shaping system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5850057B2 (en) * | 2011-09-07 | 2016-02-03 | 日本電気株式会社 | Content distribution system, control device, and content distribution method |
| JP5657509B2 (en) * | 2011-12-13 | 2015-01-21 | 日本電信電話株式会社 | Network connection control method and network connection control device |
-
2017
- 2017-03-07 CN CN201710130843.7A patent/CN106911582B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127632A (en) * | 2006-08-15 | 2008-02-20 | 腾讯科技（深圳）有限公司 | A method, system and redirector for server traffic control |
| CN102014109A (en) * | 2009-09-08 | 2011-04-13 | 华为技术有限公司 | Flood attack prevention method and device |
| CN103379090A (en) * | 2012-04-12 | 2013-10-30 | 腾讯科技（北京）有限公司 | Open platform access frequency control method and system and frequency server |
| CN103944833A (en) * | 2013-01-21 | 2014-07-23 | 中兴通讯股份有限公司 | Method and device for flow control, and flow shaping system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106911582A (en) | 2017-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106911582B (en) | real-time flow control method and system based on Nginx server | |
| US10178165B2 (en) | Distributing application traffic to servers based on dynamic service response time | |
| CN101815033B (en) | Method, device and system for load balancing | |
| US11671402B2 (en) | Service resource scheduling method and apparatus | |
| WO2020093500A1 (en) | Intelligent scheduling method, terminal device, edge node cluster and intelligent scheduling system | |
| US9294363B2 (en) | Adjusting quality of service in a cloud environment based on application usage | |
| US8260930B2 (en) | Systems, methods and computer readable media for reporting availability status of resources associated with a network | |
| US20170171304A1 (en) | Service updating method and system for server cluster | |
| CN111049901B (en) | Load balancing method, load balancing system and registration server | |
| US11303704B2 (en) | Methods and systems for connecting devices to applications and desktops that are receiving maintenance | |
| EP3926924A1 (en) | Method and system for providing edge service, and computing device | |
| CN112445857A (en) | Resource quota management method and device based on database | |
| US20120179793A1 (en) | Resource Allocation | |
| KR101686073B1 (en) | Method, management server and computer-readable recording medium for allowing client terminal to be provided with services by converting network topology adaptively according to characteristics of the services | |
| CN112751847A (en) | Interface call request processing method and device, electronic equipment and storage medium | |
| US20070265976A1 (en) | License distribution in a packet data network | |
| CN110661673B (en) | Heartbeat detection method and device | |
| CN112437006A (en) | Request control method and device based on API gateway, electronic equipment and storage medium | |
| CN112910793A (en) | Method for connection multiplexing in seven-layer load balancing and load balancer | |
| CN111556135A (en) | Request scheduling method, system and device and electronic equipment | |
| CN111212087A (en) | Method, device, equipment and storage medium for determining login server | |
| CN114510711A (en) | Method, device, medium and computer equipment for preventing CC attack | |
| CN115580618A (en) | Load balancing method, device, equipment and medium | |
| CN113992685B (en) | Service controller determining method, system and device | |
| CN112165495B (en) | DDoS attack prevention method and device based on super-fusion architecture and super-fusion cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |