CN113347222B

CN113347222B - Data processing method and device and electronic equipment

Info

Publication number: CN113347222B
Application number: CN202010140583.3A
Authority: CN
Inventors: 陈飞; 李毅; 周天凌
Original assignee: Beijing Sogou Technology Development Co Ltd
Current assignee: Beijing Sogou Technology Development Co Ltd
Priority date: 2020-03-03
Filing date: 2020-03-03
Publication date: 2022-08-05
Anticipated expiration: 2040-03-03
Also published as: CN113347222A

Abstract

The embodiment of the application provides a data processing method, a data processing device and electronic equipment, wherein the method comprises the following steps: the pressure control system receives a request of a client; determining a target virtual agent pool according to a target site corresponding to the request; selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node; receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server requesting a corresponding target site; and returning the response message to the client corresponding to the request, and removing the target virtual agent node from the target virtual agent pool. Therefore, the used proxy servers are prevented from accessing the server of the target site again, the frequency of accessing the corresponding site server by each proxy server can be effectively controlled, and the site server is prevented from being abnormal.

Description

Data processing method and device and electronic equipment

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, and an electronic device.

Background

Currently, the client accesses the target server in various ways, one way is that the client directly accesses the target server, and the other way is that the client accesses the target server through an HTTP (HyperText Transfer Protocol) proxy service. Because the target server is accessed through the HTTP proxy service, the browsing speed can be improved, the probability of being attacked can be reduced, the network access limitation can be solved, and the like; making accessing a target server through an HTTP proxy service popular with many users.

Generally, the HTTP proxy service is divided into two layers: the first layer is a cluster formed by a plurality of proxy servers, and the proxy servers provide HTTP proxy services independently; the second layer is a reverse proxy server, which provides a uniform proxy interface for the client. The client side directly sends the HTTP request to a reverse proxy server, and the reverse proxy server forwards the request to each proxy server node of the proxy cluster according to a forwarding strategy; each proxy server node accesses the target server to obtain corresponding data and returns the data to the reverse proxy server, and the reverse proxy server returns the data returned by the proxy server node to the client.

The forwarding strategy of the reverse proxy server comprises a plurality of strategies, such as a polling strategy, a weighting strategy and the like; both of these strategies are to split the overall request to different broker nodes. When the total request quantity of a client to a certain target site is large, the more the request quantity forwarded to a server corresponding to the target site by each proxy node is; and further, a great pressure is applied to the server of the target site, and the server failure of the site or the triggering of an anti-attack strategy of the site is caused.

Disclosure of Invention

The embodiment of the application provides a data processing method, so that the pressure of a site server is controlled, and the site server is prevented from being abnormal.

Correspondingly, the embodiment of the application also provides a data processing device and electronic equipment, which are used for ensuring the realization and application of the method.

In order to solve the above problem, an embodiment of the present application discloses a data processing method, which is applied to a data processing platform, where the data processing platform includes: the method comprises the following steps of: the pressure control system receives a request of the client; determining a target virtual agent pool according to a target site corresponding to the request, wherein virtual agent nodes in the target virtual agent pool correspond to the proxy servers of the proxy server cluster; selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node; receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server of a target site corresponding to the request; and returning the response message to the client corresponding to the request, and removing the target virtual agent node from the target virtual agent pool.

Optionally, the determining a target virtual agent pool according to the target site corresponding to the request includes: analyzing the request, and determining identification information corresponding to a target site, wherein the identification information comprises a domain name or an Internet Protocol (IP) address; searching a corresponding target virtual agent pool from the established virtual agent pool according to the identification information of the target site; and if the target virtual agent pool is not found from the established virtual agent pools, adopting the virtual agent nodes of the agent servers corresponding to the target sites to establish the target virtual agent pools corresponding to the target sites.

Optionally, after the target site corresponding to the request determines the target virtual agent pool, the method further includes: judging whether available virtual agent nodes exist in the target virtual agent pool or not; the selecting a target virtual agent node from the target virtual agent pool includes: if available virtual agent nodes exist in the target virtual agent pool, selecting one virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node; and if the target virtual agent pool does not have available virtual agent nodes, returning a first request failure message to the client corresponding to the request.

Optionally, the method further comprises: and after the response message is returned to the client corresponding to the request for preset time, adding the target virtual agent node into the target virtual agent node pool.

Optionally, when receiving requests of a plurality of clients, after receiving the requests of the clients, the method further includes: for each request, verifying user information in the request; if the verification is successful, adding the request into a request queue of a user corresponding to the request; and if the verification fails, returning a second request failure message to the client corresponding to the request.

Optionally, after the verifying the user information in the request for each request, the method further includes: judging whether a second user exists in the first users with the request queues or not, wherein the second user is a user with the request queues and high-priority concurrent connection authority; if the second user exists in the first users, judging whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections; if the third user exists in the second users, selecting a target user from the third user; and selecting a request from a request queue of a target user, and executing the step of determining a target virtual agent pool according to a target site corresponding to the request.

Optionally, the method further comprises: and if the second user does not exist in the first users or the third user does not exist in the second users, selecting a target user from the first users.

Optionally, after the selecting the target user, the method further includes: judging whether a connection requested by a low priority exists in the established connection corresponding to the target user; if the connection of the low-priority request exists in the established connection corresponding to the target user, updating the low-priority request of one established connection into a high-priority request; and if the connection of the low-priority request does not exist in the established connection corresponding to the target user, executing the step of selecting the request from the request queue of the target user.

Optionally, after selecting a request from the request queue of the target user and before determining the target virtual agent pool according to the target site corresponding to the request, the method further includes: judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system; if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, executing the step of determining a target virtual agent pool according to the target station corresponding to the request; if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, judging whether the established connections requested by low priority exist in the pressure control system; closing a low priority requested connection in the pressure control system if there is an established low priority requested connection in the pressure control system; and if the established connection of the low-priority request does not exist in the pressure control system, after the request is selected from the request queue of the target user, a third request failure message is returned to the client corresponding to the request selected from the request queue of the target user.

Optionally, said closing a low priority requested connection in said pressure control system, comprising: searching the connection with the earliest established low-priority request from the currently established connections with low-priority requests of the pressure control system and determining the connection with the earliest established low-priority request as the connection with the target low-priority request; if the user corresponding to the target low-priority request is a user with high-priority concurrent connection permission, judging whether the connection number of the current high-priority request of the user corresponding to the target low-priority request is smaller than the corresponding concurrent connection limit number; if the number of connections of the target low-priority request corresponding to the current high-priority request of the user is less than the corresponding concurrent connection limit number, updating the target low-priority request into a high-priority request; executing the step of searching the connection requested by the low priority which is established earliest from the connections requested by the low priority which are established currently by the pressure control system and determining the connection requested by the target low priority; and if the user corresponding to the target low-priority request does not have the high-priority concurrent connection permission or the user connected with the target low-priority request is the user with the high-priority concurrent connection permission and the number of the connections of the current high-priority request is equal to the number of the corresponding concurrent connection restrictions, closing the connection of the target low-priority request.

Optionally, after the step of selecting a request from the request queue of the target user, the method further includes: judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system; if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, executing the step of determining a target virtual agent pool according to the target station corresponding to the request; and if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, returning a fourth request failure message to the client corresponding to the request selected from the request queue of the target user.

Optionally, after the step of selecting a request from the request queue of the target user and before the step of determining the target virtual agent pool according to the target site corresponding to the request, the method further includes: and updating the request selected from the request queue of the target user into a high-priority request.

Optionally, after a response message of a high-priority request of the target user is returned to the client and the connection of the high-priority request is closed, the method further includes: and if the target user has the established connection with the low-priority request and no user with the currently established connection number smaller than the corresponding concurrent connection limit number exists in other users with high-priority connection authority of the pressure control system, updating one of the established low-priority requests of the target user into the high-priority request.

The embodiment of the present application further discloses a data processing apparatus, which specifically includes: the method is applied to a data processing platform, and the data processing platform comprises: the device is deployed in the pressure control system and comprises: the receiving module is used for receiving the request of the client; a determining module, configured to determine a target virtual agent pool according to a target site corresponding to the request, where a virtual agent node in the target virtual agent pool corresponds to a proxy server of the proxy server cluster; a forwarding module, configured to select a target virtual agent node from the target virtual agent pool, and forward the request to a target proxy server corresponding to the target virtual agent node; a receiving module, configured to receive a response message returned by the target proxy server, where the response message is obtained by the target proxy server from a server of a target site corresponding to the request; and the return module is used for returning the response message to the client corresponding to the request and removing the target virtual agent node from the target virtual agent pool.

Optionally, the determining module is configured to parse the request, and determine identification information of the corresponding target site, where the identification information includes a domain name or an internet protocol IP address; searching a corresponding target virtual agent pool from the established virtual agent pool according to the identification information of the target site; and if the target virtual agent pool is not found from the established virtual agent pools, adopting the virtual agent nodes of the agent servers corresponding to the target sites to establish the target virtual agent pools corresponding to the target sites.

Optionally, the apparatus further comprises: a node judgment module, configured to judge whether there is an available virtual agent node in a target virtual agent pool after the target virtual agent pool is determined by the target site corresponding to the request; the forwarding module is configured to select a virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node if the available virtual agent nodes exist in the target virtual agent pool; and the first message returning module is used for returning a first request failure message to the client corresponding to the request if no available virtual agent node exists in the target virtual agent pool.

Optionally, the apparatus further comprises: and the adding module is used for adding the target virtual agent node into the target virtual agent node pool after the response message is returned to the client corresponding to the request for preset time.

Optionally, the apparatus further comprises: the system comprises a verification module, a verification module and a verification module, wherein the verification module is used for verifying user information in a request after receiving the requests of a plurality of clients after receiving the requests of the clients; if the verification is successful, adding the request into a request queue of a user corresponding to the request; and if the verification fails, returning a second request failure message to the client corresponding to the request.

Optionally, the apparatus further comprises: the user judging module is used for judging whether a second user exists in the first users with the request queues or not after the user information in the requests is verified aiming at each request, wherein the second user is a user with the request queues and high-priority concurrent connection authority; if the second user exists in the first users, judging whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections; the first user selection module is used for selecting a target user from the third users if the third users exist in the second users; the request selection module is used for selecting a request from a request queue of a target user; and the determining module is used for executing the step of determining the target virtual agent pool according to the target site corresponding to the request after selecting the request from the request queue of the target user.

Optionally, the apparatus further comprises: and the second user selection module is used for selecting a target user from the first user if the second user does not exist in the first user or the third user does not exist in the second user.

Optionally, the apparatus further comprises: the first connection judgment module is used for judging whether a connection with a low priority request exists in the established connection corresponding to the target user after the target user is selected; the first request updating module is used for updating one of the established connection low-priority requests into a high-priority request if the connection of the low-priority request exists in the established connection corresponding to the target user; the request selecting module is configured to execute the step of selecting the request from the request queue of the target user if there is no connection of the low-priority request in the connections established by the target user.

Optionally, the apparatus further comprises: a second connection determining module, configured to determine, after the request is selected from the request queue of the target user and before the target virtual agent pool is determined according to the target site corresponding to the request, whether the number of currently established connections of the pressure control system is smaller than a global connection limit number of the pressure control system; the determining module is configured to execute the step of determining the target virtual agent pool according to the target station corresponding to the request if the number of currently established connections of the pressure control system is less than the global connection limit number of the pressure control system; a third connection judgment module, configured to judge whether an established connection with a low priority request exists in the pressure control system if the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system; a connection closing module for closing a low priority requested connection in the pressure control system if the established low priority requested connection exists in the pressure control system; and the second message returning module is used for returning a third request failure message to the client corresponding to the request selected from the request queue of the target user after the request is selected from the request queue of the target user if the established connection of the low-priority request does not exist in the pressure control system.

Optionally, the connection closing module is configured to search, from currently established low-priority requested connections of the pressure control system, an earliest established low-priority requested connection and determine the connection as a target low-priority requested connection; if the user corresponding to the target low-priority request is a user with high-priority concurrent connection permission, judging whether the connection number of the current high-priority request of the user corresponding to the target low-priority request is smaller than the corresponding concurrent connection limit number; if the number of connections of the target low-priority request corresponding to the current high-priority request of the user is less than the corresponding concurrent connection limit number, updating the target low-priority request into a high-priority request; executing the step of searching the connection requested by the low priority which is established earliest from the connections requested by the low priority which are established currently by the pressure control system and determining the connection requested by the target low priority; and if the user corresponding to the target low-priority request does not have the high-priority concurrent connection permission or the user connected with the target low-priority request is the user with the high-priority concurrent connection permission and the number of the connections of the current high-priority request is equal to the number of the corresponding concurrent connection restrictions, closing the connection of the target low-priority request.

Optionally, the apparatus further comprises: a fourth connection judging module, configured to judge whether the number of currently established connections of the pressure control system is smaller than a global connection limit number of the pressure control system after the step of selecting a request from a request queue of a target user; the determining module is configured to execute the step of determining the target virtual agent pool according to the target station corresponding to the request if the number of currently established connections of the pressure control system is less than the global connection limit number of the pressure control system; and the third message returning module is used for returning a fourth request failure message to the client corresponding to the request selected from the request queue of the target user if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system.

Optionally, the apparatus further comprises: and a second request updating module, configured to update a request selected from the request queue of the target user to a high-priority request after the step of selecting the request from the request queue of the target user and before the target virtual agent pool is determined according to the target site corresponding to the request.

Optionally, the apparatus further comprises: and the third request updating module is used for updating one of the low-priority requests of the target user into the high-priority request if the target user has the established connection of the low-priority request and no user with the currently established connection number smaller than the corresponding concurrent connection limit number exists in other users with high-priority connection authority of the pressure control system after the connection of the high-priority request is closed.

The embodiment of the application also discloses a readable storage medium, and when the instructions in the storage medium are executed by a processor of the electronic device, the electronic device can execute the data processing method according to any one of the embodiments of the application.

An electronic device is also disclosed in an embodiment of the present application, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: receiving a request of the client; determining a target virtual agent pool according to a target site corresponding to the request, wherein virtual agent nodes in the target virtual agent pool correspond to the proxy servers of the proxy server cluster; selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node; receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server of a target site corresponding to the request; and returning the response message to the client corresponding to the request, and removing the target virtual agent node from the target virtual agent pool.

Optionally, after the target site corresponding to the request determines the target virtual agent pool, the method further includes an instruction for: judging whether available virtual agent nodes exist in the target virtual agent pool or not; the selecting a target virtual agent node from the target virtual agent pool includes: if available virtual agent nodes exist in the target virtual agent pool, selecting one virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node; and if the target virtual agent pool does not have available virtual agent nodes, returning a first request failure message to the client corresponding to the request.

Optionally, further comprising instructions for: and after the response message is returned to the client corresponding to the request for preset time, adding the target virtual agent node into the target virtual agent node pool.

Optionally, when receiving requests of a plurality of clients, after receiving the requests of the clients, the method further includes an instruction for: for each request, verifying user information in the request; if the verification is successful, adding the request into a request queue of a user corresponding to the request; and if the verification fails, returning a second request failure message to the client corresponding to the request.

Optionally, after verifying the user information in the request for each request, the method further includes: judging whether a second user exists in the first users with the request queues or not, wherein the second user is a user with the request queues and high-priority concurrent connection authority; if the second user exists in the first users, judging whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections; if the third user exists in the second users, selecting a target user from the third user; and selecting a request from a request queue of a target user, and executing the step of determining a target virtual agent pool according to a target site corresponding to the request.

Optionally, further comprising instructions for: and if the second user does not exist in the first users or the third user does not exist in the second users, selecting a target user from the first users.

Optionally, after the target user is selected, instructions for performing the following operations are further included: judging whether a connection requested by a low priority exists in the established connection corresponding to the target user; if the connection of the low-priority request exists in the established connections corresponding to the target user, updating one of the low-priority requests of the established connections into a high-priority request; and if the connection of the low-priority request does not exist in the established connection corresponding to the target user, executing the step of selecting the request from the request queue of the target user.

Optionally, after selecting a request from the request queue of the target user and before determining the target virtual agent pool according to the target site corresponding to the request, the method further includes an instruction for performing the following operations: judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system; if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, executing the step of determining a target virtual agent pool according to the target station corresponding to the request; if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, judging whether the established connections requested by low priority exist in the pressure control system; closing a low priority requested connection in the pressure control system if there is an established low priority requested connection in the pressure control system; and if the established connection of the low-priority request does not exist in the pressure control system, after the request is selected from the request queue of the target user, a third request failure message is returned to the client corresponding to the request selected from the request queue of the target user.

Optionally, after the step of selecting a request from the request queue of the target user, the method further includes an instruction for: judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system; if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, executing the step of determining a target virtual agent pool according to the target station corresponding to the request; and if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, returning a fourth request failure message to the client corresponding to the request selected from the request queue of the target user.

Optionally, after the step of selecting a request from the request queue of the target user, and before the target virtual agent pool is determined according to the target site corresponding to the request, the method further includes an instruction for: and updating the request selected from the request queue of the target user into a high-priority request.

Optionally, after a response message of a high-priority request of the target user is returned to the client and the connection of the high-priority request is closed, the method further includes an instruction for: and if the target user has the established connection with the low-priority request and no user with the currently established connection number smaller than the corresponding concurrent connection limit number exists in other users with high-priority connection authority of the pressure control system, updating one of the established low-priority requests of the target user into the high-priority request.

The embodiment of the application has the following advantages:

in the embodiment of the application, after receiving the request of the client, the pressure control system may determine a target virtual agent pool according to a target site corresponding to the request, and then select a target virtual agent node from the target virtual agent pool; the virtual proxy node in the target virtual proxy pool corresponds to a proxy server of the proxy server cluster, so that the request can be forwarded to the target proxy server corresponding to the target virtual proxy node, and the target proxy server obtains a response message corresponding to the request from a server of a target site corresponding to the request and returns the response message to the pressure control system; after receiving the response message returned by the target proxy server, the pressure control system can return the response message to the client on one hand, so as to respond to the request of the client; on the other hand, the target virtual agent node can be removed from the virtual agent pool, so that the used proxy servers are prevented from visiting the server of the target site again, the frequency of visiting the corresponding site server by each proxy server can be effectively controlled, and the site server is prevented from being abnormal.

Drawings

FIG. 1a is a schematic diagram of a data interaction relationship of a data processing platform according to an embodiment of the present application;

FIG. 1b is a flow chart of the steps of an embodiment of a data processing method of the present application;

FIG. 2 is a flow chart of the steps of an alternative embodiment of a data processing method of the present application;

FIG. 3 is a flow chart of steps in an alternative embodiment of a data processing method of the present application;

FIG. 4 is a flow chart of steps in an alternative embodiment of a data processing method of the present application;

FIG. 5 is a flow chart of steps in an alternative embodiment of a data processing method of the present application;

FIG. 6 is a flow chart of steps in an alternative embodiment of a data processing method of the present application;

FIG. 7 is a flow chart of steps in an alternative embodiment of a data processing method of the present application;

FIG. 8 is a block diagram of an embodiment of a data processing apparatus of the present application;

FIG. 9 is a block diagram of an alternate embodiment of a data processing apparatus of the present application;

FIG. 10 illustrates a block diagram of an electronic device for data processing in accordance with an exemplary embodiment;

fig. 11 is a schematic structural diagram of an electronic device for data processing according to another exemplary embodiment of the present application.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.

The data processing method provided by the embodiment of the application is applied to a data processing platform, and the data processing platform may include: the system comprises a client, a pressure control system, a proxy server cluster and a server cluster; the pressure control system can execute the data processing method to realize pressure control on each server in the server cluster. Referring to fig. 1a, a schematic diagram of a data interaction relationship of a data processing platform according to an embodiment of the present application is shown.

The proxy server cluster can be composed of N proxy servers, and each proxy server provides HTTP proxy service independently. Each proxy server corresponds to an HTTP proxy address and a port, and occupies a public network egress IP (Internet Protocol) address. N is a positive integer, which may be specifically set as required, and this is not limited in this application.

The pressure control system corresponds to an access address and a port, and receives a request of a client through the access address and the port. The pressure system is configured with access parameters (such as HTTP proxy addresses, ports, regional information corresponding to public network outlet IP addresses, and the like) of each proxy server in the proxy server cluster, and subsequently forwards the request of the client to the corresponding proxy server through the access parameters of each proxy server.

The server cluster can be composed of M servers, and one or more sites can be deployed on each server; wherein M is a positive integer, which may be specifically set as required, and this is not limited in this application.

One of the core concepts of the embodiment of the application is that the pressure control system selects a virtual agent node from a virtual agent pool of a target site corresponding to each request, and then adopts an agent server corresponding to the virtual agent node to respond to the request; after the request response is finished, the used virtual agent nodes in the virtual agent pool are removed, so that the frequency of accessing the corresponding site server by each agent server can be effectively controlled, the pressure of the site server is controlled, and the site server is prevented from being abnormal.

Referring to fig. 1b, a flowchart illustrating steps of an embodiment of a data processing method according to the present application is shown, and specifically, the method may include the following steps:

step 102, the pressure control system receives the request of the client.

In the embodiment of the application, when a user needs to use the proxy service to access the server, the proxy service can be configured in the client in advance. The access address and the port of the pressure control system may be configured, and certainly, other information such as user requirement information may also be configured, which is not limited in this embodiment of the application. After a user executes access operation on a client, the corresponding client can receive an access instruction corresponding to the access operation, and then generates a corresponding request according to the access instruction. And sending the request to a corresponding pressure control system according to the access address and the port configured in the client.

After receiving a request sent by a client, the pressure control system can select a proxy server from a proxy server cluster to provide HTTP proxy service for the request; the method for selecting the proxy server from the proxy server cluster can refer to steps 104-106.

And step 104, determining a target virtual agent pool according to the target site corresponding to the request, wherein virtual agent nodes in the target virtual agent pool correspond to the proxy servers of the proxy server cluster.

In the embodiment of the application, a plurality of virtual agent pools can be arranged in the pressure control system, and one station can correspond to one virtual agent pool; each virtual agent pool comprises at least one virtual agent node, and one virtual agent node can correspond to one agent server which can access the site server corresponding to the virtual agent pool.

In this embodiment, the pressure control system may determine a target virtual agent pool corresponding to the request, and then select a proxy server that can provide the HTTP proxy service for the request according to the target virtual agent pool. The request can be analyzed, and a corresponding target site is determined; and then determining a corresponding target virtual agent pool according to the target site, wherein the target virtual agent pool comprises at least one virtual agent node corresponding to a proxy server capable of accessing the target site.

And 106, selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node.

In the embodiment of the application, after a target virtual agent pool is determined, a target virtual agent node can be selected from the target virtual agent pool; the proxy server corresponding to the target virtual proxy node may then be used to provide HTTP proxy services for the request. The method for selecting the target virtual agent node may include multiple methods: such as random selection; for example, when the request includes user requirement information, the request may be selected according to the user requirement information, which is not limited in this embodiment of the application.

The pressure control system may determine a target proxy server corresponding to the target virtual proxy node; then according to the HTTP proxy address and port of the target proxy server, establishing connection with the target proxy server, and forwarding the request to the target proxy server.

And step 108, receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server of a target site corresponding to the request.

Step 110, returning the response message to the client corresponding to the request, and removing the target virtual agent node from the virtual agent pool.

After receiving the request, the target proxy server can analyze the request and determine a corresponding target site; the request is then sent to the server of the target site. After receiving the request, the server of the target site can generate a response message corresponding to the request and return the response message to the target proxy server; the target proxy server may return the response message to the pressure control system. After the pressure control system receives the response message, the response message can be returned to the client corresponding to the request, so that the response to the request is realized.

In the embodiment of the application, the pressure control system can remove the target virtual proxy node from the virtual proxy pool while returning the response message to the client, so that the used proxy servers can be prevented from visiting the server of the target site again, and the frequency of visiting the server of the corresponding site by each proxy server can be effectively controlled.

In summary, in the embodiment of the present application, after receiving the request from the client, the pressure control system may determine a target virtual agent pool according to a target site corresponding to the request, and then select a target virtual agent node from the target virtual agent pool; the virtual proxy node in the target virtual proxy pool corresponds to a proxy server of the proxy server cluster, so that the request can be forwarded to the target proxy server corresponding to the target virtual proxy node, and the target proxy server acquires a response message corresponding to the request from a server of a target site corresponding to the request and returns the response message to the pressure control system; after receiving the response message returned by the target proxy server, the pressure control system can return the response message to the client on one hand, so as to respond to the request of the client; on the other hand, the target virtual agent node can be removed from the virtual agent pool, so that the used proxy servers are prevented from visiting the server of the target site again, the frequency of visiting the corresponding site server by each proxy server can be effectively controlled, and the site server is prevented from being abnormal.

The following describes how to determine the target virtual agent pool and how to select the target virtual agent node.

Referring to fig. 2, a flowchart illustrating steps of an alternative embodiment of the data processing method of the present application is shown, which may specifically include the following steps:

step 202, the pressure control system receives the request of the client.

In the embodiment of the application, a user can configure the proxy service in a proxy service configuration interface of a client. The configuration interface can comprise an access address configuration item of the pressure control system, a port configuration item of the pressure control system and a user requirement configuration item; the user requirement configuration item may include multiple types, such as a network export region configuration item of the proxy server, a bandwidth configuration item, and the like, which is not limited in this embodiment of the application.

If only the access address configuration item and the port configuration item of the pressure control system are configured, the client can acquire the access address in the access address configuration item and the port in the port configuration item after generating a request; and then the client sends the request to the pressure control system corresponding to the access address and the port.

If the access address configuration item and the port configuration item of the pressure control system are configured and simultaneously part of the user requirement information configuration items are also configured, the client can acquire the user requirement information in the configured user requirement configuration items after receiving the access instruction, and then generate a corresponding request according to the access instruction and the user requirement information. Then the client can obtain the access address in the access address configuration item, obtain the port in the port configuration item, and send the request to the access address and the pressure control system corresponding to the port.

Wherein, the determining the target virtual agent pool according to the target site corresponding to the request in step 104 may include the following steps 204 to 208:

step 204, resolving the request, and determining identification information of the corresponding target site, wherein the identification information includes a domain name or an Internet Protocol (IP) address.

One site may correspond to one IP address, and one IP address may correspond to one or more domain names. In the embodiment of the application, the pressure control system can control the pressure of the site server according to the IP address and can also control the pressure of the site server according to the domain name; the method can be specifically set according to requirements, and the method is not limited by the embodiment of the application.

When the pressure control system performs pressure control on the site server according to the domain name, the request may be analyzed according to the HTTP proxy protocol, and the domain name of the target site corresponding to the request is obtained.

When the pressure control System performs pressure control on the site server according to the IP address, after acquiring the Domain Name of the target site corresponding to the request, the Domain Name System (DNS) resolution may be performed on the Domain Name of the target site to acquire the IP address of the target site.

For convenience of the following description, the domain name and the IP address of the destination station may be collectively referred to as identification information of the destination station, and the identification information is used to uniquely identify one station.

Step 206, according to the identification information of the target site, searching a corresponding target virtual agent pool from the created virtual agent pool.

And 208, if the target virtual agent pool is not found from the created virtual agent pools, creating the target virtual agent pool corresponding to the target site by using the virtual agent nodes of the proxy server corresponding to the target site.

In an implementation manner of the embodiment of the present application, the pressure control system may pre-create a virtual agent pool corresponding to each site. Wherein the pressure control system may predetermine each proxy server accessible site; and then aiming at each site, generating a virtual agent pool corresponding to the site by adopting a virtual agent node of the agent server capable of accessing the site, and establishing association between the virtual agent pool and the identification information of the site.

In another implementation manner of the embodiment of the present application, the pressure control system may also create a virtual agent pool corresponding to a certain site when receiving a request of the site sent by a client for the first time. When a pressure control system receives a request of a certain site sent by a client for the first time, the pressure control system can determine a proxy server capable of accessing the site; and then generating a virtual agent pool corresponding to the site by adopting the virtual agent node of the agent server capable of accessing the site, and establishing association between the virtual agent pool and the identification information of the site.

Based on the above-mentioned manner of creating the virtual agent pool of each site, the virtual agent pool corresponding to the target site may or may not have been created. Therefore, a target virtual agent pool corresponding to the identification information of the target site can be searched from the created virtual agent pool.

If the target virtual agent pool is not found from the created virtual agent pools, corresponding virtual agent pools can be created for the target sites. Wherein the pressure control system may determine a proxy server accessible to the target site; and then generating a target virtual agent pool corresponding to the target site by adopting the virtual agent node of the agent server capable of accessing the target site, and establishing association between the generated target virtual agent pool and the identification information of the target site.

Step 210, judging whether an available virtual agent node exists in the target virtual agent pool.

In the embodiment of the present application, the virtual proxy node in the target virtual proxy pool may be an available virtual proxy node or an unavailable virtual proxy node (e.g., a virtual proxy node being used by a corresponding proxy server, or a virtual proxy node in which a corresponding proxy server fails). In order to ensure that the client's request can be responded to, the request can be served by using an available virtual proxy node corresponding to the proxy server.

In an example of the present application, it may be determined whether there is an available virtual agent node in the target virtual agent pool. If there are virtual agent nodes available in the target virtual agent pool, step 212 may be performed. If there are no virtual agent nodes available in the target virtual agent pool, step 214 may be performed.

Step 212, if there is an available virtual agent node in the target virtual agent pool, selecting a virtual agent node from the available virtual agent nodes in the target virtual agent pool as a target virtual agent node.

In this embodiment of the application, when the user requirement information does not exist in the request, one virtual agent node may be randomly selected from available virtual agent nodes in the target virtual agent pool as the target virtual agent node. When the request has user requirement information, one virtual agent node meeting the user requirement information can be selected from available virtual agent nodes of the target virtual agent pool as a target virtual agent node. For example, the user requirement information is the network export region of the proxy server: beijing, then, a virtual agent node corresponding to the network export region Beijing of the proxy server can be selected from the available virtual agent nodes of the target virtual agent pool as the target virtual agent node. Of course, when there is no virtual agent node that satisfies the user requirement information in the available virtual agent nodes of the target virtual agent pool, one virtual agent node may also be randomly selected as the target virtual agent node.

After step 212 is performed, steps 216-222 may be performed.

Step 214, if there is no available virtual agent node in the target virtual agent pool, returning a first request failure message to the client corresponding to the request.

The target virtual agent pool does not have available virtual agent nodes, which can indicate that the request cannot be responded at the moment, and a first request failure message can be returned to the client; to inform the client of the request failure.

Step 216, forwarding the request to a target proxy server corresponding to the target virtual proxy node.

After the target virtual agent node is selected, the pressure Control server is connected to the target agent server corresponding to the target virtual agent node, for example, a TCP (Transmission Control Protocol) connection. And then forwarding the request to a target proxy server corresponding to the target virtual proxy node through the connection between the target proxy servers corresponding to the target virtual proxy node.

Step 218, receiving a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the request.

Step 220, returning the response message to the client, and removing the target virtual agent node from the virtual agent pool.

After acquiring a response message corresponding to the request from the server of the target site, the target proxy server may return the response message to the pressure control system; and then the pressure control system returns the response message to the client so as to realize the response to the request.

Step 222, after the response message is returned to the client for a preset time period, adding the target virtual agent node to the target virtual agent node pool.

In the embodiment of the application, after the virtual proxy node is removed from the virtual proxy pool, the proxy server corresponding to the virtual proxy node cannot provide the HTTP proxy service; therefore, the adding mechanism for adding the removed virtual agent nodes to the corresponding virtual agent pool again can be set, so that the number of the available virtual agent nodes in the virtual agent pool is ensured, and the condition that the response is overtime due to the fact that the number of the available virtual agent nodes in the virtual agent pool is too small is avoided.

In one example of the present application, the adding mechanism may be: and after the response message is returned to the client for a preset time, adding the target virtual agent node into the target virtual agent node pool. The preset duration can be set according to requirements, and the embodiment of the application is not limited thereto. After the response message is returned to the client, timing can be started, and when the timing duration reaches a preset duration, the target virtual agent node is added to the target virtual agent node pool. Of course, after the response message is returned to the client, a timer may be set according to a preset time length, and when the timer times out, the target virtual agent node is added to the target virtual agent node pool.

Of course, the adding mechanism may also be other mechanisms, for example, when the number of the virtual agent nodes remaining and available in the target virtual agent pool is lower than a set number, the target virtual agent node is added to the target virtual agent node pool; the embodiments of the present application do not limit this.

In summary, in the embodiment of the present application, in the process of determining the target virtual agent pool according to the target site corresponding to the request, the request may be analyzed to determine the identification information of the corresponding target site, where the identification information includes a domain name or an IP address; searching a corresponding target virtual agent pool from the established virtual agent pool according to the identification information of the target site; if the target virtual agent pool is not found from the established virtual agent pools, adopting virtual agent nodes of the agent servers corresponding to the target sites to establish the target virtual agent pools corresponding to the target sites; furthermore, the target virtual agent pool can be determined in multiple ways, so that the pressure of the site server can be controlled in multiple ways subsequently, and the universality of the data processing method in the embodiment of the application is enhanced.

Secondly, in the embodiment of the present application, after the response message is returned to the client corresponding to the request, the target virtual agent node may be removed from the virtual agent pool; and then the used proxy servers are prevented from accessing the server of the target site again, the frequency of accessing the corresponding site server by each proxy server is effectively controlled, and the site server is prevented from being abnormal. Then after the response message is returned to the client corresponding to the request for preset time, adding the target virtual agent node into the target virtual agent node pool; therefore, the number of available virtual agent nodes in the virtual agent pool is ensured, and the condition that response is overtime due to the fact that the number of available virtual agent nodes in the virtual agent pool is too small is avoided.

In the prior art, when the request amount is limited (bandwidth limitation, access frequency limitation), and when a plurality of clients (or a plurality of users) access simultaneously, the probability that a user with a high access frequency preempts a line is higher than that of a user with a low access frequency, so that the proxy use right is not reasonably distributed. The following embodiments of the present application provide a data processing method, which can reasonably allocate an agent use right to different users.

Referring to fig. 3, a flowchart illustrating steps of another alternative embodiment of the data processing method of the present application is shown, which may specifically include the following steps:

step 302, the pressure control system receives a plurality of requests from the clients.

In practical application, a plurality of clients send requests to the pressure control system at the same time; and the pressure control system can simultaneously receive the requests of a plurality of clients.

Step 304, for each request, verifying the user information in the request.

In the embodiment of the application, the pressure control system allows users with authority to use the HTTP proxy service, and prohibits users without authority from using the HTTP proxy service. Therefore, after receiving the request sent by each client, the pressure control system can acquire the user information from the request, then verify the user information, and judge whether the user corresponding to the request has the right to use the HTTP proxy service. Wherein the user information may include a user name and a password.

In one example of the application, the stress control system may add user information with the authority to use the proxy to the profile at startup. One way to verify the user information may be to match the requested user information with the user information in a profile. When the user information matched with the requested user information is found in the configuration file, the verification is successful; at this point, it may be determined that the user corresponding to the request has the right to use the HTTP proxy service. When the user information matched with the requested user information is not found in the configuration file, the verification fails; at this point, it may be determined that the user to which the request corresponds does not have the right to use the HTTP proxy service.

When the pressure control system is started, user information of a new user is added, and the newly added user information may be added to a configuration location in an online Application Programming Interface (API) loading registration manner.

In an example of the application, when there are many users of the pressure control system, a user information database may be created, and then user information of each user is added to the user information database; it may not be necessary to add preset user information with the right to use the proxy to the profile at this time. One way to verify the user information may be to match the requested user information with user information in a user information database. When the user information matched with the requested user information is found in the user information database, the verification is successful; at this point, it may be determined that the user corresponding to the request has the right to use the HTTP proxy service. When the user information matched with the requested user information is not found in the user information database, the verification fails; at this point, it may be determined that the user to which the request corresponds does not have the right to use the HTTP proxy service.

In this embodiment of the present application, if the requested user information is successfully verified, step 306 may be executed; if the requested user information fails to be verified, step 328 may be performed.

Step 306, if the verification is successful, adding the request into a request queue of a user corresponding to the request.

In the embodiment of the application, when the verification of the user information of the request is successful, the request can be added to the tail of the request queue of the user corresponding to the request; the subsequent pressure control system may fetch requests from the head of each user's request queue for processing.

Step 308, judging whether a second user exists in the first users with the request queues, wherein the second user is a user with the request queues and high-priority concurrent connection authority; if the second user exists in the first user, executing step 310; if the second user does not exist in the first users, go to step 314; the first user refers to a user with a request queue in the pressure control system.

In an optional embodiment of the present application, the pressure control system may configure the processing right for a user of the pressure control system as required; the processing authority can be determined according to requirements, and the embodiment of the application does not limit the processing authority. In an example of the present application, the processing permission may be a high-priority concurrent connection permission, and the high-priority concurrent connection permission may refer to a permission for establishing a connection corresponding to multiple high-priority requests in parallel. When configuring high-priority concurrent connection rights for a user, configuring a corresponding concurrent connection limit number for the user; the number of concurrent connections may be set as required, which is not limited in the embodiment of the present application. And when the number of the currently established connections of the user with the high-priority concurrent connection authority is lower than the corresponding concurrent connection limit number, the processing priority of the user with the high-priority concurrent connection authority is higher than that of the user without the high-priority concurrent connection authority.

In the embodiment of the present application, a user having a request queue in the pressure control system may be determined, and for convenience of subsequent description, the user having the request queue in the pressure control system may be referred to as a first user. Then judging whether a user with high-priority concurrent connection authority exists in the first users; and subsequently, the user with the request queue and high-priority concurrent connection authority is called a second user. If the second user exists in the first user, step 310 may be executed; if the second user does not exist in the first users, step 314 may be executed.

Step 310, if the second user exists in the first users, determining whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections; if yes, go to step 312; if not, go to step 314.

In the embodiment of the application, because the number of concurrent connections corresponding to the second user with the high-priority concurrent connection authority is limited, when the number of connections currently established by the second user is equal to the corresponding number of concurrent connection limits, the connection may not be continuously established for the second user, so as to ensure that other users can enjoy the proxy use right. Therefore, it may be determined whether a third user whose currently established connection number is lower than the corresponding concurrent connection limit number exists in the second users, and if yes, step 312 may be executed; if not, step 314 may be performed.

And step 312, if the third user exists in the second users, selecting a target user from the third users.

In the embodiment of the application, if a third user with the currently established connection number lower than the corresponding concurrent connection limit number exists in the second users, one third user is randomly selected from the users as a target user.

And step 314, selecting a target user from the first users if the second user does not exist in the first users or the third user does not exist in the second users.

In this embodiment of the application, if the second user does not exist in the first user, or a third user whose currently established connection number is lower than the corresponding concurrent connection limit number does not exist in the second user, one user may be randomly selected from the first users as a target user.

Step 316, select the request from the request queue of the target user.

In the embodiment of the present application, after selecting the target user, on one hand, a request may be selected from the request queue of the target user according to the currently established number of connections of the pressure control system, and then the following steps 318 to 324 may be performed to establish a connection for the request. The following description will specifically describe how to select a request from the request queue of the target user.

On the other hand, step 308-step 312/step 314 may be executed again to continuously select the target user.

Step 318, determining a target virtual agent pool according to the target site corresponding to the request.

Step 320, selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target proxy server corresponding to the target virtual agent node.

Step 322, receiving a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the request.

Step 324, returning the response message to the client, and removing the target virtual agent node from the virtual agent pool.

Step 326, after returning the response message to the client for a preset time, adding the target virtual agent node to the target virtual agent node pool.

The steps 318 to 326 are similar to the steps 204 to 222, and are not described herein again.

Step 328, if the verification fails, returning a second request failure message to the client corresponding to the request.

In the embodiment of the present application, if the user information in the request fails to be verified, a second request failure message may be returned to the client.

In summary, in the embodiment of the present application, after receiving requests from multiple clients, the pressure control system may verify user information in each request; if the verification is successful, adding the request into a request queue of a user corresponding to the request, and subsequently taking out the request from the queue for processing; if the verification fails, returning a second request failure message to the client corresponding to the request; thereby ensuring that only authorized users are provided with the proxy server.

Secondly, in the embodiment of the present application, after verifying the user information in the request for each request, it may be determined whether a second user (i.e., a user having a request queue and having a high-priority concurrent connection authority) exists in the first users having the request queue; if the first user has a second user, judging whether the second user has a third user with the currently established connection number lower than the corresponding concurrent connection limit number; if the third user exists in the second users, selecting a target user from the third user; if the second user does not exist in the first user or the third user does not exist in the second user, selecting a target user from the first user; and further ensuring the number of concurrent connections of the users with high-priority concurrent authority.

In the embodiment of the application, when the target user is a user selected from the first user and the target user is a user selected from the third user, after a request is selected from a request queue of the target user, whether the mode of determining the corresponding target virtual agent pool according to the requested target site is executed is judged to be different; the following description is made separately.

Hereinafter, how to determine whether to execute the determination of the corresponding target virtual agent pool according to the requested target site after determining that the request is selected from the request queue of the target user when the target user is the user selected from the first user will be described.

Referring to fig. 4, a flowchart illustrating steps of another alternative embodiment of the data processing method of the present application is shown, which may specifically include the following steps:

step 402, the pressure control system receives a plurality of requests from the clients.

Step 404, for each request, verifying the user information in the request.

In this embodiment of the present application, if the requested user information is successfully verified, step 406 may be executed; if the requested user information fails to be verified, step 426 may be performed.

Step 406, if the verification is successful, adding the request to a request queue of a user corresponding to the request.

And 408, if the second user does not exist in the first users or a third user with the currently established connection number lower than the corresponding concurrent connection limit number does not exist in the second users, selecting a target user from the first users.

Step 402-step 408 are similar to step 302-step 310 and step 314, and are not described herein again.

Step 410, select a request from the request queue of the target user.

Step 412, determining whether the number of connections currently established by the pressure control system is less than the global connection limit number of the pressure control system. If so, step 414 may be performed. If so, step 424 may be performed.

In the embodiment of the application, the pressure control system can set the corresponding global connection limit number according to the requirement; when the number of established connections reaches the global connection limit number, no new connection can be established. Therefore, after the target user is determined, whether the number of the currently established connections of the pressure control system is smaller than the global connection limit number of the pressure control system can be judged; if the number of currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, then step 414 may be performed. If the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, step 424 may be performed.

And 414, if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, determining a target virtual agent pool according to the target station corresponding to the request.

Step 416, selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node.

Step 418, receiving a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the request.

Step 420, returning the response message to the client, and removing the target virtual agent node from the virtual agent pool.

Step 422, after returning the response message to the client for a preset time, adding the target virtual agent node to the target virtual agent node pool.

The steps 414 to 422 are similar to the steps 204 to 222, and are not described herein again.

Step 424, if the number of connections currently established by the pressure control system is equal to the global connection limit number of the pressure control system, returning a fourth request failure message to the client corresponding to the request selected from the request queue of the target user.

In the embodiment of the application, because a new connection cannot be established when the number of established connections reaches the global connection limit number, when it is determined that the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, a connection cannot be established for a request selected from a request queue of a target user; at this time, a fourth request failure message may be returned to the client corresponding to the request selected from the request queue of the target user.

Step 426, if the verification fails, returning a second request failure message to the client corresponding to the request.

Step 426 is similar to step 328 described above and will not be described further herein.

In the embodiment of the application, when the target user is a user selected from the third users, a request can be selected from the target user request queue and a connection can be established for the request, and the request of the target user for establishing the connection can also be processed; specifically, the following may be mentioned:

referring to fig. 5, a flowchart illustrating steps of another embodiment of the data processing method of the present application is shown, which may specifically include the following steps:

step 502, the pressure control system receives a plurality of requests from the clients.

Step 504, for each request, the user information in the request is verified.

In this embodiment of the present application, if the requested user information is successfully verified, step 406 may be executed; if the requested user information fails to be verified, step 526 may be performed.

Step 506, if the verification is successful, adding the request into a request queue of a user corresponding to the request.

And step 508, if a second user exists in the first users and a third user with the currently established connection number lower than the corresponding concurrent connection limit number exists in the second users, selecting a target user from the third user.

This step 502-step 508 is similar to the above step 302-step 312, and will not be described herein again.

In the embodiment of the application, after the target user is selected, the number of connections of the high-priority request currently established by the target user may be less than the number of corresponding concurrent connection limits; at this time, the low-priority request currently established by the target user may be set as the high-priority request, so as to ensure the number of concurrent connections of the high-priority request of the target user. The high-priority request is for a user with high-priority concurrent connection permission, and may be a request for establishing connection for the user with high-priority concurrent connection permission when the number of currently established connections of the user with high-priority concurrent connection permission is less than the corresponding number of concurrent connection restrictions. The low-priority request is a request which can be possessed by both a user with high-priority concurrent connection authority and a user without the high-priority concurrent connection authority; for a user with high-priority concurrent connection authority, the corresponding low-priority request may refer to a request for establishing a connection for the user with high-priority concurrent connection authority when the number of currently established connections is equal to the corresponding concurrent connection limit number. For users who do not have high-priority concurrent connection authority, all requests are low-priority requests.

Step 510, judging whether a connection requested by a low priority exists in the established connection corresponding to the target user. If so, step 512 may be performed; if not, step 514 may be performed.

In the embodiment of the application, whether the established connection exists in the target user can be determined, and after the established connection exists in the target user, the priority of the request corresponding to each established connection is determined; and then judging whether the connection requested by the low priority exists in the established connection corresponding to the target user. If there is a connection requested by a low priority in the connections established by the target user, step 512 may be executed; if there is no connection requested by the low priority in the established connections corresponding to the target user, step 514 may be executed.

Step 512, if there is a connection with a low priority request in the established connections corresponding to the target user, updating one of the low priority requests with the established connection to be a high priority request.

In the embodiment of the present application, if there is a connection with a low priority request in the established connections corresponding to the target user, a low priority request may be selected from the low priority requests of the established connections of the target user, and the low priority request may be updated to a high priority request. Wherein, a request with low priority can be randomly selected from low priority requests of which the target user establishes connection; or, according to a preset rule (which may be set as required), selecting a low-priority request for an established connection from low-priority requests for connections already established by a target user, which is not limited in the embodiment of the present application.

And 514, if the connection of the low-priority request does not exist in the established connection corresponding to the target user, selecting the request from the request queue of the target user.

In this embodiment of the application, if there is a connection with a low priority request in the established connections corresponding to the target user, a request may be directly selected from the head of the request queue of the target user, and a connection corresponding to the request may be established, and steps 516 to 522 may be performed.

Step 516, determining a target virtual agent pool according to the target site corresponding to the request.

Step 518, selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node.

Step 520, receiving a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the request.

Step 522, returning the response message to the client, and removing the target virtual agent node from the virtual agent pool.

Step 524, after the response message is returned to the client for a preset time period, adding the target virtual agent node to the target virtual agent node pool.

Step 526, if the verification fails, a second request failure message is returned to the client corresponding to the request.

The steps 516 to 524 are similar to the steps 204 to 222, and the step 526 is similar to the step 328, and thus are not described again.

Hereinafter, how to determine whether to execute the determination of the corresponding target virtual agent pool according to the requested target site after selecting the request from the request queue of the target user when the target user is a user selected from third users, which include the second user and have the currently established connection number lower than the corresponding concurrent connection limit number.

Referring to fig. 6, a flowchart illustrating steps of another alternative embodiment of the data processing method of the present application is shown, which may specifically include the following steps:

step 602, the pressure control system receives a plurality of requests from the clients.

Step 604, for each request, the user information in the request is verified.

In this embodiment of the present application, if the requested user information is successfully verified, step 406 may be executed; if the requested user information fails to be verified, step 630 may be performed.

Step 606, if the verification is successful, adding the request into a request queue of a user corresponding to the request.

And 608, if a second user exists in the first users and a third user with the currently established connection number lower than the corresponding concurrent connection limit number exists in the second users, selecting a target user from the third user.

Step 610, select a request from the request queue of the target user.

This step 602-step 610 is similar to the above step 302-step 312, step 316, and will not be described herein again.

And 612, judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system.

If the number of connections currently established by the pressure control system is equal to the global connection limit number of the pressure control system, go to step 614; if the number of connections currently established by the pressure control system is less than the global connection limit number for the pressure control system, step 620 may be performed.

And 614, if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, judging whether the established connections requested by the low priority exist in the pressure control system.

In this embodiment of the present application, since a new connection cannot be established when the number of established connections reaches the global connection limit number, when it is determined that the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, in order to ensure the number of high-priority concurrent connections of a user with high-priority concurrent authority, a connection requested by a low priority in the pressure control system may be closed. Thus, determining whether there is an established connection of a low priority request in the pressure control system; if there is an established low priority requested connection in the pressure control system, step 616 may be performed. If there is not an established low priority requested connection in the pressure control system, step 618 is performed.

Step 616, closing a low priority requested connection in the pressure control system if there is an established low priority requested connection in the pressure control system.

In this embodiment, if there is an established connection with a low priority request in the pressure control system, the connection with a low priority request in the pressure control system may be closed to ensure a high priority concurrent connection number for a target user. Step 620 is then performed.

In one example of the present application, one manner of closing a low priority requested connection in the pressure control system at step 616 may include the sub-steps of:

substep S2, searching the connection with the earliest established low-priority request from the currently established connections with low-priority requests of the pressure control system and determining the connection with the earliest established low-priority request as the connection with the target low-priority request;

step S4, if the user corresponding to the target low priority request is a user with high priority concurrent connection authority, judging whether the connection number of the current high priority request of the user corresponding to the target low priority request is less than the corresponding concurrent connection limit number;

substep S6, if the number of connections corresponding to the current high-priority request of the user for the connection of the target low-priority request is less than the corresponding concurrent connection limit number, updating the target low-priority request to a high-priority request; sub-step S2 is performed.

And a substep S8, if the user corresponding to the target low-priority request does not have the high-priority concurrent connection permission, or the user of the target low-priority connection is the user with the high-priority concurrent connection permission whose current number of connections requested by the high-priority request is equal to the number of concurrent connection restrictions, closing the connection requested by the target low-priority request.

Step 618, if there is no established connection of the low-priority request in the pressure control system, returning a third request failure message to the client corresponding to the request selected from the request queue of the target user.

In the embodiment of the present application, if there is no established connection with a low priority request in the pressure control system, it indicates that all connections in the pressure control system are connections with a high priority request; and no longer establishes a connection for the request selected from the target user request queue. At this time, a third request failure message may be returned to the client corresponding to the request selected from the request queue of the target user.

And step 620, if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, determining a target virtual agent pool according to the target station corresponding to the request.

Step 622, selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target proxy server corresponding to the target virtual agent node.

Step 624, receiving a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the request.

Step 626, returning the response message to the client, and removing the target virtual agent node from the virtual agent pool.

Step 628, after returning the response message to the client for a preset time, adding the target virtual agent node to the target virtual agent node pool.

Step 630, if the verification fails, a second request failure message is returned to the client corresponding to the request.

The steps 622 to 628 are similar to the steps 204 to 222, and the step 630 is similar to the step 328, and are not described again here.

When the target user is a third user selected from the users whose currently established connection number included in the second user is lower than the corresponding concurrent connection limit number, the above step 502 and step 530 may be combined with the step 602 and step 604 to explain a manner of making a connection for the request of the target user.

Referring to fig. 7, a flowchart illustrating steps of another alternative embodiment of the data processing method of the present application is shown, which may specifically include the following steps:

step 702, the pressure control system receives a plurality of requests from the clients.

Step 704, for each request, the user information in the request is verified.

In this embodiment of the present application, if the requested user information is successfully verified, step 706 may be executed; if the requested user information fails to be verified, step 734 may be performed.

Step 706, if the verification is successful, adding the request to a request queue of a user corresponding to the request.

Step 708, if a second user exists in the first users and a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections, selecting a target user from the third user.

This step 702-step 708 is similar to the above step 302-step 312, and will not be described herein again.

Step 710, judging whether a connection requested by a low priority exists in the established connection corresponding to the target user.

If there is a connection requested by a low priority in the connections established by the target user, step 712 is executed. If there is no connection requested by the low priority in the established connections corresponding to the target user, step 714 is executed.

Step 712, if there is a connection with a low priority request in the established connections corresponding to the target user, updating the low priority request of one of the established connections to a high priority request.

Step 714, if there is no connection of low priority request in the established connection corresponding to the target user, selecting a request from the request queue of the target user.

And 716, judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system.

If the number of connections currently established by the pressure control system is equal to the global connection limit number for the pressure control system, then step 718 is performed. If the number of connections currently established by the pressure control system is less than the global connection limit number for the pressure control system, go to step 724.

Step 718, if the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, determining whether an established connection with a low priority request exists in the pressure control system.

If there is an established low priority requested connection in the pressure control system, step 720 is performed. If there is not an established low priority requested connection in the pressure control system, step 722 is performed.

Step 720, if there is an established connection for a low priority request in the pressure control system, closing a connection for a low priority request in the pressure control system.

Step 722, if the established connection of the low-priority request does not exist in the pressure control system, returning a third request failure message to the client corresponding to the request selected from the request queue of the target user.

Step 724, if the number of connections currently established by the pressure control system is less than the global connection limit number of the pressure control system, determining a target virtual agent pool according to the target station corresponding to the request.

Step 726, select a target virtual agent node from the target virtual agent pool, and forward the request to a target proxy server corresponding to the target virtual agent node.

Step 728, receiving a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the request.

Step 730, returning the response message to the client, and removing the target virtual agent node from the virtual agent pool.

Step 732, after returning the response message to the client for a preset time, adding the target virtual agent node to the target virtual agent node pool.

Step 734, if the verification fails, returning a second request failure message to the client corresponding to the request.

In an optional embodiment of the present application, after a response message of a high priority request of the target user is returned to the client, and after the connection of the high priority request is closed, if the target user has an established connection of a low priority request, and there is no user whose currently established connection number is less than the corresponding concurrent connection limit number among other users having high priority and connection authority of the pressure control system, one of the low priority requests of the target user whose connection has been established is updated to the high priority request. And further ensure the connection number of the high-priority requests of the users with high-priority concurrent connection authority.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the embodiments are not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the embodiments. Further, those skilled in the art will also appreciate that the embodiments described in the specification are presently preferred and that no particular act is required of the embodiments of the application.

The embodiment of the present invention further provides a data processing apparatus, which is applied to a data processing platform, where the data processing platform includes: the device comprises a client, a pressure control system, a proxy server cluster and a server cluster, wherein the device is deployed in the pressure control system.

Referring to fig. 8, a block diagram of a data processing apparatus according to an embodiment of the present application is shown, which may specifically include the following modules:

a receiving module 802, configured to receive a request of the client;

a determining module 804, configured to determine a target virtual agent pool according to a target site corresponding to the request, where a virtual agent node in the target virtual agent pool corresponds to a proxy server of the proxy server cluster;

a forwarding module 806, configured to select a target virtual agent node from the target virtual agent pool, and forward the request to a target proxy server corresponding to the target virtual agent node;

a receiving module 808, configured to receive a response message returned by the target proxy server, where the response message is obtained by the target proxy server from a server of a target site corresponding to the request;

a returning module 810, configured to return the response message to the client corresponding to the request, and remove the target virtual agent node from the target virtual agent pool.

Referring to fig. 9, a block diagram of an alternative embodiment of a data processing apparatus of the present application is shown.

In an optional embodiment of the present application, the determining module 804 is configured to parse the request, and determine identification information of a corresponding target site, where the identification information includes a domain name or an internet protocol IP address; searching a corresponding target virtual agent pool from the established virtual agent pool according to the identification information of the target site; and if the target virtual agent pool is not found from the established virtual agent pools, adopting the virtual agent nodes of the agent servers corresponding to the target sites to establish the target virtual agent pools corresponding to the target sites.

In an optional embodiment of the present application, the apparatus further includes:

a node determining module 812, configured to determine whether there is an available virtual agent node in a target virtual agent pool after the target site corresponding to the request determines the target virtual agent pool;

the forwarding module 806 is configured to select a virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node if the available virtual agent node exists in the target virtual agent pool;

a first message returning module 814, configured to return a first request failure message to the client corresponding to the request if there is no available virtual agent node in the target virtual agent pool.

an adding module 816, configured to add the target virtual agent node to the target virtual agent node pool after returning the response message to the client preset time corresponding to the request.

a verification module 818, configured to, when requests of multiple clients are received, verify user information in the requests for each request after the receiving of the requests of the clients; if the verification is successful, adding the request into a request queue of a user corresponding to the request; and if the verification fails, returning a second request failure message to the client corresponding to the request.

a user determining module 820, configured to determine whether a second user exists in the first users having the request queue after verifying the user information in the request for each request, where the second user is a user having a request queue and having a high-priority concurrent connection right; if the second user exists in the first users, judging whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections;

a first user selecting module 822, configured to select a target user from the third users if the third user exists in the second users;

a request selecting module 824, configured to select a request from a request queue of a target user;

the determining module 804 is configured to execute the step of determining the target virtual agent pool according to the target site corresponding to the request after selecting the request from the request queue of the target user.

a second user selecting module 826, configured to select a target user from the first user if the second user does not exist in the first user or the third user does not exist in the second user.

a first connection determining module 828, configured to determine, after the target user is selected, whether a connection with a low-priority request exists in the established connection corresponding to the target user;

a first request updating module 830, configured to update a low-priority request of an established connection to a high-priority request if there is a connection of the low-priority request in the established connections corresponding to the target user;

the request selecting module 824 is configured to execute the step of selecting the request from the request queue of the target user if there is no connection of the low-priority request in the connections established by the target user.

a second connection determining module 832, configured to determine whether the number of currently established connections of the pressure control system is less than the global connection limit number of the pressure control system after the request is selected from the request queue of the target user and before the target virtual agent pool is determined according to the target site corresponding to the request;

the determining module 804 is configured to execute the step of determining the target virtual agent pool according to the target station corresponding to the request if the number of currently established connections of the pressure control system is less than the global connection limit number of the pressure control system;

a third connection determining module 834, configured to determine whether there is an established connection with a low priority request in the pressure control system if the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system;

a connection closure module 836 to close a low priority requested connection in the pressure control system if there is an established low priority requested connection in the pressure control system;

a second message returning module 838, configured to, if there is no established connection of the low-priority request in the pressure control system, return a third request failure message to the client corresponding to the request selected from the request queue of the target user after the request is selected from the request queue of the target user.

In an alternative embodiment of the present application, the connection closing module 836 is configured to search the low-priority requested connection that is earliest established from the low-priority requested connections that are currently established by the pressure control system and determine the connection as the target low-priority requested connection; if the user corresponding to the target low-priority request is a user with high-priority concurrent connection permission, judging whether the connection number of the current high-priority request of the user corresponding to the target low-priority request is smaller than the corresponding concurrent connection limit number; if the number of connections of the target low-priority request corresponding to the current high-priority request of the user is less than the corresponding concurrent connection limit number, updating the target low-priority request into a high-priority request; executing the step of searching the connection requested by the low priority which is established earliest from the connections requested by the low priority which are established currently by the pressure control system and determining the connection requested by the target low priority; and if the user corresponding to the target low-priority request does not have the high-priority concurrent connection permission or the user connected with the target low-priority request is the user with the high-priority concurrent connection permission and the number of the connections of the current high-priority request is equal to the number of the corresponding concurrent connection restrictions, closing the connection of the target low-priority request.

a fourth connection determining module 840, configured to determine, after the step of selecting a request from the request queue of the target user, whether the number of currently established connections of the pressure control system is smaller than the global connection limit number of the pressure control system;

a third message returning module 842, configured to return a fourth request failure message to the client corresponding to the request selected from the request queue of the target user if the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system.

a second request updating module 844, configured to update, after the step of selecting the request from the request queue of the target user and before the target virtual agent pool is determined according to the target site corresponding to the request, the request selected from the request queue of the target user to a high-priority request.

a third request updating module 846, configured to, after a response message of a high-priority request of the target user is returned to the client and a connection of the high-priority request is closed, update a low-priority request of one of the target users that has established a connection to the high-priority request if the target user has an established connection to a low-priority request and no user whose currently established number of connections is smaller than the corresponding concurrent connection limit number exists among other users having high-priority connection permissions of the pressure control system.

In summary, in the embodiment of the present application, after receiving the request of the client, the pressure control system may determine a target virtual agent pool according to a target site corresponding to the request, and then select a target virtual agent node from the target virtual agent pool; the virtual proxy node in the target virtual proxy pool corresponds to a proxy server of the proxy server cluster, so that the request can be forwarded to the target proxy server corresponding to the target virtual proxy node, and the target proxy server acquires a response message corresponding to the request from a server of a target site corresponding to the request and returns the response message to the pressure control system; after receiving the response message returned by the target proxy server, the pressure control system can return the response message to the client on one hand, so as to respond to the request of the client; on the other hand, the target virtual agent node can be removed from the virtual agent pool, so that the used proxy servers are prevented from visiting the server of the target site again, the frequency of visiting the corresponding site server by each proxy server can be effectively controlled, and the site server is prevented from being abnormal.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

Fig. 10 is a block diagram illustrating the structure of an electronic device 1000 for data processing in accordance with an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 10, electronic device 1000 may include one or more of the following components: processing components 1002, memory 1004, power components 1006, multimedia components 1008, audio components 1010, input/output (I/O) interfaces 1012, sensor components 1014, and communication components 1016.

The processing component 1002 generally controls overall operation of the electronic device 1000, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 1002 may include one or more processors 1020 to execute instructions to perform all or a portion of the steps of the methods described above. Further, processing component 1002 may include one or more modules that facilitate interaction between processing component 1002 and other components. For example, the processing component 1002 can include a multimedia module to facilitate interaction between the multimedia component 1008 and the processing component 1002.

The memory 1004 is configured to store various types of data to support operation at the device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 1004 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power components 1006 provide power to the various components of electronic device 1000. The power components 1006 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 1000.

The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 1010 is configured to output and/or input audio signals. For example, the audio component 1010 may include a Microphone (MIC) configured to receive external audio signals when the electronic device 1000 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

I/O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, sensor assembly 1014 may detect the open/closed status of device 1000, the relative positioning of components, such as a display and keypad of electronic device 1000, the change in position of electronic device 1000 or a component of electronic device 1000, the presence or absence of user contact with electronic device 1000, the orientation or acceleration/deceleration of electronic device 1000, and the change in temperature of electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1014 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communications component 1014 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 1000 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 1004 comprising instructions, executable by the processor 1020 of the electronic device 1000 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

A non-transitory computer readable storage medium in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform a data processing method, the method comprising: the pressure control system receives a request of the client; determining a target virtual agent pool according to a target site corresponding to the request, wherein virtual agent nodes in the target virtual agent pool correspond to the proxy servers of the proxy server cluster; selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node; receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server of a target site corresponding to the request; and returning the response message to the client corresponding to the request, and removing the target virtual agent node from the target virtual agent pool.

Optionally, after the selecting the target user, the method further includes: judging whether a connection requested by a low priority exists in the established connection corresponding to the target user; if the connection of the low-priority request exists in the established connections corresponding to the target user, updating one of the low-priority requests of the established connections into a high-priority request; and if the connection of the low-priority request does not exist in the established connection corresponding to the target user, executing the step of selecting the request from the request queue of the target user.

Fig. 11 is a schematic structural diagram of an electronic device 1100 for data processing according to another exemplary embodiment of the present application. The electronic device 1100 may be a server, which may vary widely due to configuration or performance, and may include one or more Central Processing Units (CPUs) 1122 (e.g., one or more processors) and memory 1132, one or more storage media 1130 (e.g., one or more mass storage devices) storing applications 1142 or data 1144. Memory 1132 and storage media 1130 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 1130 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, the central processor 1122 may be provided in communication with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server.

The server may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1158, one or more keyboards 1156, and/or one or more operating systems 1141, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

An electronic device comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors the one or more programs including instructions for: receiving a request of the client; determining a target virtual agent pool according to a target site corresponding to the request, wherein virtual agent nodes in the target virtual agent pool correspond to the proxy servers of the proxy server cluster; selecting a target virtual agent node from the target virtual agent pool, and forwarding the request to a target agent server corresponding to the target virtual agent node; receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server of a target site corresponding to the request; and returning the response message to the client corresponding to the request, and removing the target virtual agent node from the target virtual agent pool.

Optionally, after the target virtual agent pool is determined according to the target site corresponding to the request, the method further includes an instruction for performing the following operations: judging whether available virtual agent nodes exist in the target virtual agent pool or not; the selecting a target virtual agent node from the target virtual agent pool includes: if available virtual agent nodes exist in the target virtual agent pool, selecting one virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node; and if the target virtual agent pool does not have available virtual agent nodes, returning a first request failure message to the client corresponding to the request.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The data processing method, the data processing apparatus and the electronic device provided by the present application are introduced in detail, and specific examples are applied herein to illustrate the principles and embodiments of the present application, and the descriptions of the above embodiments are only used to help understand the method and the core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A data processing method is applied to a data processing platform, and the data processing platform comprises: the method comprises the following steps of:

the pressure control system receives a request of the client;

verifying the user information in the request, and adding the request to a request queue of a user corresponding to the request under the condition of successful verification;

selecting a target user from users with a request queue, selecting a request from the request queue of the target user, and determining a target virtual agent pool according to a target site corresponding to the selected request, wherein virtual agent nodes in the target virtual agent pool correspond to the agent servers of the agent server cluster; the target user is a user of which the number of the currently established connections is lower than the number of the corresponding concurrent connections;

selecting a target virtual agent node from the target virtual agent pool, and forwarding the selected request to a target agent server corresponding to the target virtual agent node;

receiving a response message returned by the target proxy server, wherein the response message is acquired by the target proxy server from a server of the target site corresponding to the selected request;

and returning the response message to the client corresponding to the selected request, and removing the target virtual agent node from the target virtual agent pool.

2. The method of claim 1, wherein determining the pool of target virtual agents based on the target site corresponding to the selected request comprises:

analyzing the selected request, and determining identification information corresponding to a target site, wherein the identification information comprises a domain name or an Internet Protocol (IP) address;

searching a corresponding target virtual agent pool from the established virtual agent pool according to the identification information of the target site;

and if the target virtual agent pool is not found from the established virtual agent pools, adopting the virtual agent nodes of the agent servers corresponding to the target sites to establish the target virtual agent pools corresponding to the target sites.

3. The method of claim 1, wherein after determining the target virtual agent pool according to the target site corresponding to the selected request, the method further comprises:

judging whether available virtual agent nodes exist in the target virtual agent pool or not;

the selecting a target virtual agent node from the target virtual agent pool includes:

if available virtual agent nodes exist in the target virtual agent pool, selecting one virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node;

and if the target virtual agent pool does not have available virtual agent nodes, returning a first request failure message to the client corresponding to the selected request.

4. The method of claim 1, further comprising:

and after the response message is returned to the client preset time corresponding to the selected request, adding the target virtual agent node into the target virtual agent node pool.

5. The method of claim 1, wherein when receiving requests from a plurality of clients, after the receiving requests from the clients and verifying user information in the requests, the method further comprises:

and if the verification fails, returning a second request failure message to the client corresponding to the request.

6. The method according to claim 1, wherein the user information in the request is verified, and in case of successful verification, after the request is added to the request queue of the user corresponding to the request, the method further comprises:

judging whether a second user exists in the first users with the request queues or not, wherein the second user is a user with the request queues and high-priority concurrent connection authority;

if the second user exists in the first users, judging whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections;

and if the third user exists in the second users, selecting a target user from the third users.

7. The method of claim 6, further comprising:

and if the second user does not exist in the first users or the third user does not exist in the second users, selecting a target user from the first users.

8. The method of claim 1, further comprising:

judging whether a connection requested by a low priority exists in the established connection corresponding to the target user;

if the connection of the low-priority request exists in the established connections corresponding to the target user, updating one of the low-priority requests of the established connections into a high-priority request;

and if the connection of the low-priority request does not exist in the established connection corresponding to the target user, executing the step of selecting the request from the request queue of the target user.

9. The method of claim 1, wherein after selecting a request from the request queue of the target user and before determining the target virtual agent pool according to the target site corresponding to the selected request, the method further comprises:

judging whether the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system;

if the number of the currently established connections of the pressure control system is less than the global connection limit number of the pressure control system, executing the step of determining a target virtual agent pool according to the target station corresponding to the selected request;

if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, judging whether the established connections requested by low priority exist in the pressure control system;

closing a low priority requested connection in the pressure control system if there is an established low priority requested connection in the pressure control system;

and if the established connection of the low-priority request does not exist in the pressure control system, after the request is selected from the request queue of the target user, a third request failure message is returned to the client corresponding to the request selected from the request queue of the target user.

10. The method of claim 9, wherein said closing a low priority requested connection in said pressure control system comprises:

searching the connection with the earliest established low-priority request from the currently established connections with low-priority requests of the pressure control system and determining the connection with the earliest established low-priority request as the connection with the target low-priority request;

if the user corresponding to the target low-priority request is a user with high-priority concurrent connection permission, judging whether the connection number of the current high-priority request of the user corresponding to the target low-priority request is smaller than the corresponding concurrent connection limit number;

if the number of connections of the target low-priority request corresponding to the current high-priority request of the user is less than the corresponding concurrent connection limit number, updating the target low-priority request into a high-priority request;

executing the step of searching the connection requested by the low priority which is established earliest from the connections requested by the low priority which are established currently by the pressure control system and determining the connection requested by the target low priority;

and if the user corresponding to the target low-priority request does not have the high-priority concurrent connection permission or the user connected with the target low-priority request is the user with the high-priority concurrent connection permission and the number of the connections of the current high-priority request is equal to the number of the corresponding concurrent connection restrictions, closing the connection of the target low-priority request.

11. The method of claim 1, wherein after the step of selecting a request from the request queue of the target user, the method further comprises:

and if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system, returning a fourth request failure message to the client corresponding to the request selected from the request queue of the target user.

12. The method of claim 1, wherein after the step of selecting a request from the request queue of the target user and before the step of determining the target virtual agent pool according to the target site corresponding to the selected request, the method further comprises:

and updating the request selected from the request queue of the target user into a high-priority request.

13. The method of claim 10, wherein after a response message of a high priority request from the target user is returned to the client and the connection of the high priority request is closed, the method further comprises:

and if the target user has the established connection with the low-priority request and no user with the currently established connection number smaller than the corresponding concurrent connection limit number exists in other users with high-priority connection authority of the pressure control system, updating one of the established low-priority requests of the target user into the high-priority request.

14. A data processing apparatus, for use in a data processing platform, the data processing platform comprising: the device is deployed in the pressure control system and comprises:

the receiving module is used for receiving the request of the client;

the verification module is used for verifying the user information in the request and adding the request to a request queue of a user corresponding to the request under the condition of successful verification;

the first user selection module is used for selecting a target user from users with request queues;

the request selecting module is used for selecting a request from the request queue of the target user; the target user is a user of which the number of the currently established connections is lower than the number of the corresponding concurrent connections;

the determining module is used for determining a target virtual agent pool according to a target site corresponding to the selected request, and virtual agent nodes in the target virtual agent pool correspond to the proxy servers of the proxy server cluster; the forwarding module is used for selecting a target virtual agent node from the target virtual agent pool and forwarding the selected request to a target agent server corresponding to the target virtual agent node;

a receiving module, configured to receive a response message returned by the target proxy server, where the response message is obtained by the target proxy server from the server of the target site corresponding to the selected request;

and the return module is used for returning the response message to the client corresponding to the selected request and removing the target virtual agent node from the target virtual agent pool.

15. The apparatus of claim 14,

the determining module is configured to analyze the selected request, and determine identification information corresponding to a target site, where the identification information includes a domain name or an internet protocol IP address; searching a corresponding target virtual agent pool from the established virtual agent pool according to the identification information of the target site; and if the target virtual agent pool is not found from the established virtual agent pools, adopting the virtual agent nodes of the agent servers corresponding to the target sites to establish the target virtual agent pools corresponding to the target sites.

16. The apparatus of claim 14, further comprising:

the node judgment module is used for judging whether available virtual agent nodes exist in the target virtual agent pool or not after the target station corresponding to the selected request determines the target virtual agent pool;

the forwarding module is configured to select a virtual agent node from the available virtual agent nodes of the target virtual agent pool as a target virtual agent node if the available virtual agent nodes exist in the target virtual agent pool;

and the first message returning module is used for returning a first request failure message to the client corresponding to the selected request if no available virtual agent node exists in the target virtual agent pool.

17. The apparatus of claim 14, further comprising:

and the adding module is used for adding the target virtual agent node into the target virtual agent node pool after the response message is returned to the client-side preset time length corresponding to the selected request.

18. The apparatus of claim 14, further comprising:

the verification module is configured to, when requests of multiple clients are received, after the requests of the clients are received and user information in the requests is verified, return a second request failure message to the client corresponding to the request if the verification fails.

19. The apparatus of claim 14, further comprising:

the user judging module is used for verifying the user information in the request, and under the condition of successful verification, after the request is added into a request queue of a user corresponding to the request, judging whether a second user exists in a first user with the request queue or not, wherein the second user is a user with the request queue and high-priority concurrent connection authority; if the second user exists in the first users, judging whether a third user exists in the second users, wherein the number of the currently established connections is lower than the number of the corresponding concurrent connections;

the first user selection module is used for selecting a target user from the third users if the third users exist in the second users.

20. The apparatus of claim 19, further comprising:

and the second user selection module is used for selecting a target user from the first user if the second user does not exist in the first user or the third user does not exist in the second user.

21. The apparatus of claim 14, further comprising:

the first connection judging module is used for judging whether a connection requested by a low priority exists in the established connection corresponding to the target user after the target user is selected;

the first request updating module is used for updating one of the established connection low-priority requests into a high-priority request if the connection of the low-priority request exists in the established connection corresponding to the target user;

the request selecting module is configured to execute the step of selecting the request from the request queue of the target user if there is no connection of the low-priority request in the connections established by the target user.

22. The apparatus of claim 14, further comprising:

a second connection judging module, configured to judge whether a currently established connection number of the pressure control system is smaller than a global connection limit number of the pressure control system after a request is selected from the request queue of the target user and before a target virtual agent pool is determined according to a target site corresponding to the selected request;

the determining module is configured to execute the step of determining the target virtual agent pool according to the target site corresponding to the selected request if the number of currently established connections of the pressure control system is less than the global connection limit number of the pressure control system;

a third connection judgment module, configured to judge whether an established connection with a low priority request exists in the pressure control system if the number of currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system;

a connection closing module for closing a low priority requested connection in the pressure control system if the established low priority requested connection exists in the pressure control system;

and the second message returning module is used for returning a third request failure message to the client corresponding to the request selected from the request queue of the target user after the request is selected from the request queue of the target user if the established connection of the low-priority request does not exist in the pressure control system.

23. The apparatus of claim 22,

the connection closing module is used for searching the connection with the earliest established low-priority request from the currently established connections with low-priority requests of the pressure control system and determining the connection with the earliest established low-priority request as the connection with the target low-priority request; if the user corresponding to the target low-priority request is a user with high-priority concurrent connection permission, judging whether the connection number of the current high-priority request of the user corresponding to the target low-priority request is smaller than the corresponding concurrent connection limit number; if the number of connections of the target low-priority request corresponding to the current high-priority request of the user is less than the corresponding concurrent connection limit number, updating the target low-priority request into a high-priority request; executing the step of searching the connection requested by the low priority which is established earliest from the connections requested by the low priority which are established currently by the pressure control system and determining the connection requested by the target low priority; and if the user corresponding to the target low-priority request does not have the high-priority concurrent connection permission or the user connected with the target low-priority request is the user with the high-priority concurrent connection permission and the number of the connections of the current high-priority request is equal to the number of the corresponding concurrent connection restrictions, closing the connection of the target low-priority request.

24. The apparatus of claim 14, further comprising:

a fourth connection judging module, configured to judge whether the number of currently established connections of the pressure control system is smaller than a global connection limit number of the pressure control system after the step of selecting a request from a request queue of a target user;

and the third message returning module is used for returning a fourth request failure message to the client corresponding to the request selected from the request queue of the target user if the number of the currently established connections of the pressure control system is equal to the global connection limit number of the pressure control system.

25. The apparatus of claim 14, further comprising:

and a second request updating module, configured to update the request selected from the request queue of the target user to a high-priority request after the step of selecting the request from the request queue of the target user and before the target virtual agent pool is determined according to the target site corresponding to the selected request.

26. The apparatus of claim 23, further comprising:

and the third request updating module is used for updating one of the low-priority requests of the target user into the high-priority request if the target user has the established connection of the low-priority request and no user with the currently established connection number smaller than the corresponding concurrent connection limit number exists in other users with high-priority connection authority of the pressure control system after the connection of the high-priority request is closed.

27. An electronic device comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory, and wherein the one or more programs configured to be executed by the one or more processors comprise instructions for performing the data processing method of any of method claims 1-13.

28. A readable storage medium, characterized in that instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method according to any of method claims 1-13.