CN106911736B - Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy - Google Patents

Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy Download PDF

Info

Publication number
CN106911736B
CN106911736B CN201510975253.5A CN201510975253A CN106911736B CN 106911736 B CN106911736 B CN 106911736B CN 201510975253 A CN201510975253 A CN 201510975253A CN 106911736 B CN106911736 B CN 106911736B
Authority
CN
China
Prior art keywords
data
client
asynchronous
request message
asynchronously
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201510975253.5A
Other languages
Chinese (zh)
Other versions
CN106911736A (en
Inventor
游九龙
陈卫哨
张华�
刘志祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
3600 Technology Group Co ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Qizhi Software Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd, Qizhi Software Beijing Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201510975253.5A priority Critical patent/CN106911736B/en
Publication of CN106911736A publication Critical patent/CN106911736A/en
Application granted granted Critical
Publication of CN106911736B publication Critical patent/CN106911736B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements, protocols or services for addressing or naming
    • H04L61/45Network directories; Name-to-address mapping
    • H04L61/4505Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols
    • H04L61/4511Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols using domain name system [DNS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/10Network architectures or network communication protocols for network security for controlling access to devices or network resources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/133Protocols for remote procedure calls [RPC]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Computer And Data Communications (AREA)

Abstract

The invention provides a cloud proxy server, a pure asynchronous data transmission method and a pure asynchronous data transmission system of a cloud proxy, wherein the method comprises the following steps: asynchronously receiving connection request messages from a plurality of clients; the connection request messages are processed asynchronously and are connected to corresponding receivers respectively; after the connection is successful, data transmission is carried out asynchronously between each client and the corresponding receiver. By adopting the method, the problem of call thread blocking is avoided, the high performance and high concurrency of the cloud proxy service are ensured, the responsiveness of the cloud proxy service is improved, and further the user experience is improved.

Description

Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy
Technical Field
The invention relates to the technical field of computers, in particular to a cloud proxy server, a pure asynchronous data transmission method and a pure asynchronous data transmission system of a cloud proxy.
Background
Proxy (Proxy), also called network Proxy, is a special network service that allows one network terminal (typically a client) to make an indirect connection with another network terminal (typically a server) through the Proxy's service, thereby providing services to the client. With the development of the cloud computing service field, a cloud Proxy Server (Proxy Server) has slowly become an important component of the cloud computing service, and can be used as a service platform for providing comprehensive business capability for various internet users. In practical application, the cloud proxy server is integrated in a computer system or other types of network terminals capable of providing proxy services, similar to a common proxy means, so as to provide services for clients in a cloud network.
When in implementation, a complete cloud agent request process is as follows:
firstly, a client establishes connection with a cloud proxy server;
secondly, the cloud proxy server receives a connection request message from the client, and further requests to establish connection with a target server according to a proxy protocol used by the cloud proxy server;
and finally, after the connection between the cloud proxy server and the target server is successfully established, acquiring corresponding resources provided by the target server.
With the development of the cloud computing service field, the number of clients in the cloud network is greatly increased, even tens of millions of clients may be under the same cloud proxy server, and the multitasking capability of the cloud proxy server may become a bottleneck. If a plurality of request messages exist at the same time, a higher requirement is put forward for the multitasking function of the cloud proxy server, and if the multitasking cannot be carried out, the cloud proxy server is overloaded and the network is blocked easily. And if a new processing process is restarted for multitasking and even a new cloud proxy server is added, additional burden is brought to the system, and cost is increased.
Disclosure of Invention
In view of the above problems, the present invention is proposed to provide a cloud proxy server, a pure asynchronous data transmission method of a cloud proxy and a system thereof, which overcome the above problems or at least partially solve the above problems.
Based on one aspect of the invention, a pure asynchronous data transmission method of a cloud agent is provided, which is applied to a cloud agent server and comprises the following steps:
asynchronously receiving a plurality of connection request messages from a plurality of clients;
the connection request messages are processed asynchronously and are connected to corresponding receivers respectively;
after the connection is successful, data transmission is carried out asynchronously between each client and the corresponding receiver.
Optionally, the asynchronously processing the plurality of connection request messages includes:
asynchronously reading the request message header of each connection request message until each request message header is completely received;
and determining a receiver corresponding to each connection request message according to the content of the head of each request message.
Optionally, the respectively connecting to the corresponding receivers includes:
when the receiver is a server, asynchronously requesting the DNS server for the address of the corresponding remote server for each client;
and receiving the addresses of the remote servers returned by the DNS, and performing asynchronous connection respectively.
Optionally, the asynchronously transmitting data between each client and its corresponding receiver includes:
asynchronously receiving the request body of each client and forwarding the request body to a corresponding receiver;
and asynchronously receiving the data returned by each receiver and forwarding the data to the corresponding client.
Optionally, the asynchronously receiving the request body of each client and forwarding the request body to the corresponding receiver includes:
asynchronously receiving the body of each client;
asynchronously saving the received body to a first memory;
asynchronously judging whether the body of each client is received completely;
and if so, forwarding the received body to the corresponding receiver.
Optionally, the asynchronously receiving the data returned by each receiver and forwarding the data to the corresponding client includes:
when the receiving party is a server side, asynchronously receiving data returned by each remote server;
asynchronously saving the received data to a second memory;
asynchronously judging whether the data reception of each remote server is finished;
and if so, forwarding the received data to the corresponding client at one time.
Optionally, the first storage and/or the second storage comprise a cache and/or a memory.
Based on another aspect of the present invention, there is also provided a cloud proxy server, including:
an asynchronous receiving module adapted to asynchronously receive a plurality of connection request messages from a plurality of clients;
the asynchronous connection module is suitable for carrying out asynchronous processing on the plurality of connection request messages and respectively connecting the connection request messages to corresponding receivers;
and the asynchronous transmission module is suitable for asynchronously transmitting data between each client and the corresponding receiver after the connection is successful.
Optionally, the asynchronous connection module is further adapted to:
asynchronously reading the request message header of each connection request message until each request message header is completely received;
and determining a receiver corresponding to each connection request message according to the content of the head of each request message.
Optionally, the asynchronous connection module is further adapted to:
when the receiver is a server, asynchronously requesting the DNS server for the address of the corresponding remote server for each client;
and receiving the addresses of the remote servers returned by the DNS, and performing asynchronous connection respectively.
Optionally, the asynchronous transmission module is further adapted to:
asynchronously receiving the request body of each client and forwarding the request body to a corresponding receiver;
and asynchronously receiving the data returned by each receiver and forwarding the data to the corresponding client.
Optionally, the asynchronous receiving module is further adapted to: asynchronously receiving the body of each client;
the cloud proxy server further comprises: a first memory adapted to store data received by the asynchronous reception module;
the asynchronous transfer module is further adapted to: asynchronously judging whether the body of each client is received completely; and if so, forwarding the received body to the corresponding receiver.
Optionally, the asynchronous receiving module is further adapted to: when the receiving party is a server side, asynchronously receiving data returned by each remote server;
the cloud proxy server further comprises: a second memory adapted to store data received by the asynchronous reception module;
the asynchronous transfer module is further adapted to: asynchronously judging whether the data reception of each remote server is finished; and if so, forwarding the received data to the corresponding client at one time.
According to another aspect of the present invention, there is also provided a pure asynchronous data transmission system, including:
a plurality of clients adapted to initiate a connection request message;
the cloud proxy server comprises any one of the cloud proxy servers.
By adopting the method provided by the embodiment of the invention, a plurality of connection requests of a plurality of clients are received, the connection requests are processed and are connected to the corresponding receivers, and data transmission is carried out between the clients and the corresponding receivers, wherein each execution link is carried out asynchronously. Asynchronous execution enables each execution link not to block the current thread to wait for processing completion, but to process in a mode of asynchronous callback of other threads, and when the completion operation of other threads is in an idle state, the callback notifies the idle thread, so that the idle thread can be fully utilized. Therefore, the thread calling is more flexible in an asynchronous execution mode, network blockage and even collapse caused by over-concentration on one or more threads are avoided, high performance and high concurrency of the cloud agent service are guaranteed, the responsiveness of the cloud agent service is improved, and user experience is further improved. In addition, asynchronous execution avoids the problem of calling thread blocking, further avoids the problem of network breakdown caused by certain thread blocking, also avoids the problem that one thread is blocked and other threads are in idle states, and enables network resources to be fully utilized.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 illustrates a process flow diagram of a web address access method of a cloud agent according to one embodiment of the present invention;
FIG. 2 is a flowchart illustrating a process of obtaining a recipient address corresponding to a client request message and comparing the address to a blacklist according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram illustrating a website address accessing apparatus of a cloud agent according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a website address access device of a cloud agent according to an embodiment of the invention;
FIG. 5 illustrates a process flow diagram of a data acquisition method according to one embodiment of the invention;
FIG. 6 illustrates another process flow diagram of a data acquisition method in accordance with a preferred embodiment of the present invention;
FIG. 7 shows a schematic structural diagram of a data acquisition device according to one embodiment of the present invention;
FIG. 8 is a schematic diagram of another configuration of a data acquisition device according to an embodiment of the present invention;
FIG. 9 illustrates a process flow diagram of a pure asynchronous data transfer method of a cloud proxy according to one embodiment of the invention;
FIG. 10 illustrates another process flow diagram of a pure asynchronous data transfer method of a cloud proxy according to one embodiment of the invention;
FIG. 11 shows a schematic structural diagram of a cloud proxy server according to one embodiment of the present invention;
fig. 12 is another schematic structural diagram of a cloud proxy according to an embodiment of the present invention; and
fig. 13 shows a schematic diagram of a pure asynchronous data transmission system according to a preferred embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In the embodiment of the invention, the cloud proxy server needs to establish communication links with the client and the receiver respectively so as to communicate between the cloud proxy server and the client and between the cloud proxy server and the receiver.
After the communication link between the cloud proxy server and the client is successfully established, the cloud proxy server receives a connection request message of the client. In the embodiment of the invention, in order to solve the technical problem that the cloud proxy server in the prior art lacks multi-task processing capacity, the embodiment of the invention improves and sets the cloud proxy server to have asynchronous processing capacity.
Therefore, in the embodiment of the present invention, if the cloud proxy server receives connection request messages initiated by a plurality of clients, the connection request messages from the plurality of clients are respectively received in an asynchronous processing manner. Further, in the processing stage of the message, the connection request message is also processed asynchronously by adopting an asynchronous processing mode. Correspondingly, the subsequent resource request stage and the stage of returning the resource to different clients both adopt asynchronous processing modes.
Of course, the cloud proxy server has a certain access restriction, for example, a part of the websites are preset and the access of the cloud proxy server is prohibited (access prohibited url (Uniform Resource Locator) recorded in the blacklist). If such an unexpected situation occurs, when the cloud proxy server receives the connection request message, it may determine whether the access service corresponding to the connection request message can be provided. Taking the url which is recorded in the blacklist and is forbidden to access as an example, the cloud proxy service analyzes the connection request message, compares a receiver corresponding to the connection request message with the blacklist of the url which is recorded in the cloud proxy server and is forbidden to access, determines whether the subsequent steps are required to be executed according to a comparison result, and directly returns a notification message which cannot be accessed or cannot provide corresponding service to the client if the subsequent steps are not required.
Further, in implementation, if the service requested to be accessed by the client can be provided, in order to accelerate the service providing speed of the cloud proxy server, the cloud proxy server may set a cache (cache) for caching the resource provided by the target server. In this case, when the client requests the same server again, the cloud proxy server can directly read the resources from the cache and directly return the resources to the client, and the service or the resources do not need to be requested from the target server, so that the service can be provided for the client in a short time, and the time is saved; furthermore, the manner of directly reading resources from the cache enables the cloud proxy server to reduce the number of times of interaction with the target server and reduce the network burden. However, the resources in the cache are time-sensitive, and if the time-sensitive state is exceeded, the resources may become invalid data. These are the problems that need to be noticed and solved by the embodiments of the present invention, and the specific solutions are shown in the following specific examples.
It should be noted that the receiver corresponding to the connection request message may be a server, or may also be a terminal, such as a computer, a PDA, a notebook, or the like, or may also be some client providing specific functions, such as various common application programs (APP) at present, which is not limited in this respect.
The following describes a processing flow of the cloud proxy server for the client request message in several specific embodiments.
Example one
In the embodiment of the invention, the cloud proxy server receives the connection request message from the client, and in order to accelerate the processing process, firstly, a receiver corresponding to the client request message is compared with a blacklist of the url which is recorded by the cloud proxy server and is prohibited from being accessed, if the blacklist is hit, the processing is not required to be continued, and if the blacklist is not hit, other operations are continued.
Specifically, a receiver corresponding to the client request message is compared with a blacklist of the url to which access is prohibited, which is recorded by the cloud proxy server, and a network address corresponding to the receiver corresponding to the client request message needs to be acquired. Fig. 1 is a process flow diagram illustrating a website address access method of a cloud agent according to an embodiment of the present invention. Referring to fig. 1, the method includes at least the following steps S102 to S106.
In the embodiment of the present invention, first, step S102 is executed, and the request message from the client is received, and the url corresponding to the request message is obtained through parsing.
After the url corresponding to the request message is obtained, step S104 is executed to determine whether the url corresponding to the request message hits the blacklist.
And finally, executing the step S106, if so, returning the error code to the client, and informing the client that the url corresponding to the request message is set as the access prohibition on the cloud proxy server.
By adopting the method in the embodiment of the invention, after the cloud proxy server receives the request message from the client, the url corresponding to the request message is analyzed and obtained, whether the blacklist is hit is determined according to the url corresponding to the request message, and whether the address corresponding to the request message of the client limits the access of the cloud proxy server is further determined. By adopting the method in the embodiment of the invention, whether the address corresponding to the client request message can be accessed can be judged in time. If the address corresponding to the client request message is determined to limit the access of the cloud proxy server, the cloud proxy server does not initiate connection to the address, and the problem that a large amount of time is continuously consumed to execute the request message of the client because the address corresponding to the client request message cannot be identified to limit the access of the cloud proxy server is solved.
It should be noted that url is a compact representation of the location and access method of the resource available from the internet, and is the address of the standard resource on the internet, and each file on the internet has a unique url.
Specifically, in order to facilitate understanding of the comparison process between the recipient corresponding to the client request message and the blacklist of the prohibited-access url recorded by the cloud proxy server, fig. 2 shows a processing flow diagram of a method for obtaining the address of the recipient corresponding to the client request message and comparing the address with the blacklist according to an embodiment of the present invention, and referring to fig. 2, the method at least includes step S201 to step S206.
After receiving the request messages sent by the multiple clients through the communication link, the cloud proxy server performs asynchronous processing on the request messages of the multiple clients, and first performs step S201 to asynchronously read the request message headers of the request messages. The header of the request message is information informing the server of the request of the client, and consists of key-value pairs, each row is a pair, and the key and the value can be separated by an English colon. The request message header includes at least the following: a browser type of the request, a list of content types that the client can identify, and a hostname of the request are generated.
The embodiment of the invention can determine the host (host) corresponding to each request message according to the content of the header of each request message, so the integrity of the header of each request message ensures the accuracy of determining the host corresponding to each request message. In the process of receiving the request message header, step S202 is further executed to determine whether each request message header is completely received.
If the header of each request message is not completely received, step S201 is continuously executed until the header of each request message is completely received. If the header of each request message is completely received, step S203 is executed to analyze the header of the request message, determine a host corresponding to each request message, and then determine a url corresponding to the request message according to the host.
After the url corresponding to the request message is obtained, step S204 is executed to determine whether the url corresponding to the request message hits the blacklist. If the blacklist is hit, step S205 is executed, and the error code is returned to the client, so as to inform the client that the url corresponding to the request message is set as prohibited access on the cloud proxy server. If the blacklist is not hit, step S206 is executed, the cloud proxy server immediately connects to the receiver corresponding to the client request message, receives the data corresponding to the client request message, and forwards the received data to the client at one time.
In a preferred embodiment of the present invention, the blacklist may be obtained by mining an access log of the cloud proxy server. Specifically, the access log of the cloud proxy server is obtained first, and then useful information is mined through the access log of the proxy server. For example, the access logs are compared to find out the web pages included in the access logs which fail to access.
Further, the web page with access failure in the cloud proxy server needs to be judged, whether the web page is a web page prohibited from being accessed is determined according to the judgment result, and whether the url corresponding to the web page is a web page restricted from being accessed by the cloud proxy server is further judged. In the embodiment of the present invention, a direct connection test mode may be adopted in the specific determination process. The direct connection test refers to the direct connection with a destination address without passing through a cloud proxy server. When a plurality of clients access a webpage and return results are abnormal, direct connection test can be performed through a program to verify whether the contents of the webpage which cannot be accessed can be normally displayed. If the content of the webpage which cannot be accessed can be normally displayed, the number of times of access failure of the webpage by the cloud proxy server is further acquired, and whether the website should be included in a blacklist is determined according to the number of times of access failure.
In implementation, the result of the direct connection test can be judged through the following two calculation modes:
first, the number of access failures corresponding to the web page in the cloud proxy server is compared with a threshold number. If the access failure times of the cloud proxy server exceed the time threshold, for example, the set access failure time threshold is 100 times, and the access failure times of the cloud proxy server are 120 times, determining that the webpage is a webpage prohibited from being accessed, further determining that the url corresponding to the webpage is restricted from being accessed by the cloud proxy server, and adding the url corresponding to the webpage into a blacklist.
And secondly, comparing the proportion of the access failure times corresponding to the webpage in the cloud proxy server in the total access times with a proportion threshold value. If the proportion of the access failure times in the total access times exceeds a proportion threshold, for example, the set access failure proportion threshold is 50%, and the proportion of the cloud proxy access failure times in the total access times is 60%, determining that the webpage is a webpage prohibited from being accessed, further determining that the url corresponding to the webpage is restricted from being accessed by the cloud proxy, and adding the url corresponding to the webpage into a blacklist.
According to the method for generating the blacklist, manual operation is not needed, whether the webpage is the forbidden webpage or not can be judged only by comparing the access failure times of the webpage in the access log with the set threshold value, and whether the webpage is added into the blacklist or not can be further judged.
Based on the same invention concept, the embodiment of the invention also provides a cloud proxy website access device which is applied to the cloud proxy server. Fig. 3 is a schematic structural diagram illustrating a website address accessing apparatus of a cloud agent according to an embodiment of the present invention. Referring to fig. 3, the apparatus comprises at least:
a storage module 310 adapted to set a blacklist recording urls to which access is prohibited;
a receiving module 320 adapted to receive a request message from a client;
the analyzing module 330 is coupled to the receiving module 320 and adapted to analyze the url corresponding to the obtaining request message;
a hit module 340, one end of which is coupled to the parsing module 330 and one end of which is coupled to the storage module 310, and adapted to determine whether the url corresponding to the request message hits the blacklist;
and an error code returning module 350, coupled to the hit module 340, adapted to return an error code to the client if the error code is valid, and notify the client that the url corresponding to the request message is set to be prohibited from being accessed on the cloud proxy server.
In a preferred embodiment of the present invention, referring to fig. 4, the website address accessing apparatus of the cloud agent further includes:
and the mining module 360 is coupled with the parsing module 330 and is suitable for mining the access log of the cloud proxy server to obtain a blacklist.
In a preferred embodiment of the present invention, the excavation module 360 is further adapted to:
obtaining an access log of a cloud proxy server;
comparing the access logs to find out webpages which cannot be accessed;
performing direct connection test on the webpage which cannot be accessed, and verifying whether the content of the webpage which cannot be accessed can be normally displayed;
if the content of the webpage which cannot be accessed can be normally displayed, acquiring the number of times of access failure of the webpage by the cloud proxy server;
and if the access failure times of the webpage by the cloud proxy server exceed the time threshold, or the proportion of the access failure times in the total access times exceeds the proportion threshold, adding the url of the webpage into a blacklist.
In a preferred embodiment of the present invention, the parsing module 330 is further adapted to:
analyzing and reading a request message header of a request message;
when the reading of the request message head is finished, matching a host corresponding to the request message according to the request message head;
and determining the url corresponding to the request message according to the matching result.
In a preferred embodiment of the present invention, referring to fig. 4, the website address accessing apparatus of the cloud agent further includes:
the asynchronous module 370 is coupled to the receiving module 320 and adapted to invoke other modules to perform asynchronous processing on the request messages of the multiple clients if the receiving module 320 receives the request messages sent by the multiple clients.
Example two
In the embodiment of the invention, if the website requested by the client does not hit the blacklist, the cloud proxy server can establish connection with the client, and further respond to the request initiated by the client. In order to accelerate the processing process, for the service that the client has requested before, the corresponding resource may be stored in the cache, and the cache provides the corresponding service, instead of connecting to the receiver corresponding to the request message of the client. That is, the cloud proxy server first searches whether data corresponding to the data request message is stored in a cache of the cloud proxy server. FIG. 5 illustrates a process flow diagram of a data acquisition method according to one embodiment of the invention. Referring to fig. 5, the method includes at least the following steps S502 to S506.
In the embodiment of the present invention, step S502 is first executed to receive a data request message from a client.
After receiving the data request message initiated by the client, step S504 is executed to find whether valid data hit by the data request message is stored in the cache of the cloud proxy server.
And finally, executing the step S506, if yes, reading the valid data from the cache of the cloud proxy server, and returning the valid data to the client.
By adopting the method provided by the embodiment of the invention, after the cloud proxy server receives the data request message from the client, the cloud proxy server does not immediately connect the receiver corresponding to the client request message, but searches whether the cache of the cloud proxy server stores the data corresponding to the data request message or not, searches the required data from the cache, and can effectively utilize the resources in the cache. In addition, if the data corresponding to the client request message is found in the cloud proxy server, the data is directly acquired from the cache, and compared with the data acquired from the target address corresponding to the client request message, the speed of acquiring the data from the cache is higher, and the burden of the system is reduced.
In the embodiment of the invention, after the cloud proxy server receives the data request message from the client, the cloud proxy server does not immediately connect with the corresponding receiver, but searches in the cache of the cloud proxy server according to the data request message of each client. For example, according to the request headers (headers) of each client, it is determined whether valid data hit by the headers of the client is stored in the cache. If the cache of the cloud proxy server stores the effective data hit by the heads, the effective data are returned to the clients, the cloud proxy server is not connected with the corresponding receiver, and the receiver does not need to respond to the message request of the client.
The embodiment of the invention judges whether the data in the cache of the cloud proxy server is valid, firstly judges whether the data corresponding to the headers of the client side is stored in the cache of the cloud proxy server, and if the data is stored in the cache of the cloud proxy server, further judges the data, namely judges whether the storage time of the data corresponding to the headers of the client side exceeds a first storage time threshold t 1. And if the first storage time threshold t1 is not exceeded, determining that the data corresponding to the headers of the client is valid data. If the cache of the proxy server does not store the data corresponding to the headers of the client, or if the cache of the cloud proxy server stores the data corresponding to the headers of the client, but the storage time of the data corresponding to the headers of the client exceeds the first storage time threshold t1, the data are determined to be invalid data in both cases.
According to the embodiment of the invention, whether the data in the cache is valid data is further determined by judging the storage time of the data corresponding to the clients' heads in the cloud proxy cache, so that the timeliness and the accuracy of the clients for acquiring the resources in the cloud proxy cache are ensured.
When the data corresponding to the headers of the client stored in the cache is invalid data, the cloud proxy server analyzes the data request message of the client, and determines a destination according to the content in the data request message. The destination may be a server, or may be a terminal, such as a computer, a PDA, a notebook, or the like, or may be a client storing resources corresponding to the data request message, such as various APPs, which is not limited in this invention.
The cloud proxy server is connected with a corresponding receiver, the receiver responds to the message request of the client and returns the inquired data related to the request to the cloud proxy server. And the cloud proxy server judges that the data of each remote server are received completely, and forwards the received data to the corresponding client at one time.
When the data corresponding to the clients' headers stored in the cache is valid data, the storage time of the data needs to be further judged. And if the storage time of the valid data does not exceed the set second storage time threshold t2, directly returning the valid data in the cloud proxy server to the client. If the storage time of the valid data does not exceed the first storage time threshold t1 but exceeds a set second storage time threshold t2, where t2< t1, the request message for the valid data is further forwarded to the prefetch server. t2 is usually set to be a small interval from t1, which may even be less than the time required for the cloud proxy to read the data to the destination, to ensure the validity of the data. For example, t1 is set to 30 minutes, t2 is set to 28 minutes, when the cloud proxy server reads data from the cache, it is found that the storage time of the data in the cache exceeds 28 minutes, but does not exceed 30 minutes, at this time, the data in the cache is still valid data, and the data in the cache can be returned to the client initiating the data request. However, since the storage time of the data in the cache exceeds 28 minutes, which is closer to the valid time, the real data may change, or may change within the period of time from the cloud proxy server to the destination, so that the prefetching process of the real data may be initiated synchronously.
The prefetch server initiates a prefetch request to a corresponding receiver according to the request message of the valid data, the receiver responds to the prefetch request and returns the current resource corresponding to the prefetch request to the prefetch server, and as the actual data of the destination changes according to the actual situation, the prefetch behavior of the prefetch server may have the following two situations:
first, if the current resource pre-fetched by the pre-fetching server is consistent with the hit data in the cache, the data hit by the data request message in the cache of the cloud proxy server is kept unchanged, and the data hit by the cache of the cloud proxy server is returned to the client.
And secondly, if the current resource pre-fetched by the pre-fetching server is inconsistent with the hit data in the cache, the data hit by the current resource updating data request message corresponding to the pre-fetching request in the cache of the cloud proxy server is returned to the client. The present invention provides two solutions with strong possibility that the reason for the inconsistency between the current resource pre-fetched by the pre-fetch server and the hit data in the cache may be various: in the first scheme, the resource of the destination is updated in a valid period; in the second scheme, the time for prefetching the resources from the server to the destination is long, so that when the current resources are read, the effective time of data storage in the cache is exceeded, and the read data may be new data naturally and is different from the hit data in the cache.
In order to facilitate understanding of the process of obtaining valid data cached at the cloud proxy server and applied to the client request message, the embodiment of the present invention provides a specific embodiment to describe the process. Fig. 6 shows another processing flow diagram of a data acquisition method according to a preferred embodiment of the present invention, referring to fig. 6, the method includes at least steps S601 to S611.
In the embodiment of the present invention, step S601 is first executed to receive a data request message from a client. Then, step S602 is executed to parse the headers of the data request message of the client. The request content of the headers of the client is the golden price of today. After the request content of the clients 'headers is obtained, step S603 is executed to find whether data corresponding to the clients' headers are stored in the cache of the cloud proxy server. If not, step S604 is executed to determine the destination, establish a connection with the destination, obtain the resource corresponding to the golden price today from the destination, and return the obtained resource to the client.
If yes, step S605 is executed to determine whether the storage time of the data corresponding to the clients' headers in the cache exceeds the first storage time threshold t 1.
If the storage time exceeds the first storage time threshold t1, step S604 is executed to determine the destination, establish a connection with the destination, obtain the resource corresponding to the golden price today from the destination, and return the obtained resource to the client. For example, the storage time of the golden price of this day stored in the cache of the cloud proxy server is 10:00 at 12/15/2015, and as the golden price fluctuates every day, the golden price is updated by 00:00 at 16/12/2015, so that the first storage time threshold corresponding to the golden price of this day is 14 hours, and the request time of the headers of the client is 08 at 16/2015: 00, it can be obtained that the time is more than 14 hours, and at this time, the golden price today stored in the cloud proxy server cannot be returned to the client.
If the storage time does not exceed the first storage time threshold t1, step S606 is executed to determine whether the storage time of the data corresponding to the clients' headers in the cache exceeds the second storage time threshold t 2.
If the golden price does not exceed the second storage time threshold t2, step S607 is executed to return the golden price today stored in the cache of the cloud proxy server to the client.
If the second storage time threshold t2 is exceeded, step S608 is executed to forward the data request message of the client to the prefetch server. For example, the storage time of the golden price today stored in the cache of the cloud proxy server is 10:00 at 12/15/2015, the deadline time corresponding to the preset second storage time threshold t2 is 23:50 at 12/15/2015, and then the second storage time threshold t2 is 13 hours and 50 minutes. When the request time of the clients' headers is 2015, 12, 15, 23: 54, the second storage time threshold t2 is exceeded, the client's data request message is forwarded to the prefetch server.
The prefetch server initiates a prefetch request to a corresponding receiver according to the data request message of the client, the receiver responds to the prefetch request and returns the current resource corresponding to the prefetch request to the prefetch server, and step S609 is executed to receive the current resource corresponding to the data request message returned by the prefetch server.
The prefetch server initiates a prefetch request, the receiver responds to the prefetch request, and the current resource corresponding to the prefetch request is returned to the prefetch server. Further, step S610 is executed to determine whether data hit by the data request message in the cache of the cloud proxy server is the same as the current resource corresponding to the data request message.
If the execution results in step S610 are the same, for example, the prefetch server initiates a prefetch request, the receiver responds to the prefetch request, and returns the current resource corresponding to the prefetch request to the prefetch server, then the cloud proxy server receives the current resource corresponding to the data request message returned by the prefetch server, and the current resource corresponding to the data request message returned by the prefetch server is consistent with the golden price of the request message today in the cache of the cloud proxy server, step S611 is executed to return the golden price of the current day stored in the cache of the cloud proxy server to the client. Of course, in the execution process, the sequence of steps S608 to S610 and S611 is not fixed, so that the hit data in the cache is still valid data, and therefore, step S608 may be executed while step S611 is executed to directly return the hit data in the cache to the client.
If the execution result of step S610 is different, for example, the end time of the entire response process is 2015 year, 12 th, 16 th, 00:06, it indicates that the current resource corresponding to the data request message returned by the prefetch server is the golden price of 2015 year, 12 th, 16 th, and is not consistent with the today 'S golden price of the request message in the cache of the cloud proxy server, at this time, the today' S golden price corresponding to the data request message returned by the prefetch server replaces the today 'S golden price in the cache of the cloud proxy server, then step S612 is executed to update the current resource corresponding to the data request message returned by the prefetch server with the today' S golden price in the cache of the cloud proxy server. After the replacement, step S613 is executed to return the updated golden price of today in the cache of the cloud proxy server to the client.
Based on the same inventive concept, the embodiment of the invention also provides a data acquisition device which is applied to the cloud proxy server. Fig. 7 shows a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention. Referring to fig. 7, the apparatus includes at least:
a receiving module 710 adapted to receive a data request message from a client;
the searching module 720, coupled to the receiving module 710, is adapted to search whether valid data hit by the data request message is stored in the cache of the cloud proxy server;
the reading module 730 is coupled with the searching module 720 and is suitable for reading the valid data from the cache of the cloud proxy server if the valid data is read;
and a transmission module 740, coupled to the reading module 730, adapted to return the valid data read by the reading module to the client.
In a preferred embodiment of the present invention, the lookup module 720 is further adapted to:
searching whether data corresponding to the data request message is stored in a cache of the cloud proxy server;
if yes, further judging whether the storage time of the data corresponding to the data request message does not exceed a first storage time threshold t 1;
if so, determining the data in the cache as valid data;
and if any one of the data is not matched with the data in the cache, determining that the data in the cache is invalid data.
In a preferred embodiment of the present invention, referring to fig. 8, the data acquisition apparatus further includes:
a prefetch module 750, coupled to the lookup module 720, adapted to store the data corresponding to the data request message if the storage time does not exceed t1 but exceeds a set second storage time threshold t2, where t2< t1
And further forwarding the data request message to a pre-fetching server, and initiating a pre-fetching request aiming at the data request message by the pre-fetching server so as to obtain the current resource corresponding to the data request message.
In a preferred embodiment of the present invention, the receiving module 710 is further adapted to receive a current resource corresponding to the data request message returned by the prefetch server;
referring to fig. 8, the data acquisition apparatus further includes:
an updating module 760, coupled to the prefetching module 750, and adapted to determine whether data hit by the data request message in the cache of the cloud proxy server is the same as a current resource corresponding to the data request message; if so, keeping the data hit by the data request message in the cache of the cloud proxy server unchanged; and if not, updating the data hit by the data request message in the cache of the cloud proxy server by using the current resource corresponding to the data request message.
In a preferred embodiment of the present invention, referring to fig. 8, the data acquisition apparatus further includes:
an obtaining module 770, coupled to the searching module 720, adapted to search whether valid data hit by the data request message is stored in the cache of the cloud proxy server, and if not, analyze the data request message to determine a destination of the data request message; establishing connection with a destination, and acquiring a resource corresponding to the data request message from the destination; and returning the acquired resources to the client.
In a preferred embodiment of the invention, the destination comprises:
other clients storing resources corresponding to the data request message; or a remote server storing the resource corresponding to the data request message.
EXAMPLE III
The first embodiment and the second embodiment respectively illustrate specific implementation manners in the case that the cloud proxy server has a blacklist and a cache, and generally, for most of requests initiated by the client, the cloud proxy server still follows a manner that the receiver acquires corresponding resources for processing. It is pointed out in the prior art that current cloud proxy servers lack the ability to handle multiple tasks.
In order to solve the technical problem, the embodiment of the invention improves the task processing mode of the cloud proxy server. In the embodiment of the invention, the cloud proxy server receives the connection request messages from the plurality of clients in an asynchronous processing mode, further carries out asynchronous processing on the connection request messages, and then directly initiates connection asynchronously to the receivers corresponding to the client request messages. And after the cloud proxy server is asynchronously connected with the receiving party, the data in the client request message is asynchronously sent to the receiving party, the data sent by the receiving party is asynchronously received, and the data sent by the receiving party is asynchronously forwarded to the client. In other words, in the whole data request process, the embodiment of the invention adopts an asynchronous mode for processing, thereby greatly increasing the parallelism of data processing and enabling the cloud proxy server to have high parallel performance. Fig. 9 shows a process flow diagram of a pure asynchronous data transmission method of a cloud proxy according to an embodiment of the present invention. Referring to fig. 9, the method includes at least the following steps S902 to S906.
In this embodiment of the present invention, first, step S902 is executed to asynchronously receive connection request messages from a plurality of clients.
After the client request is asynchronously received, step S904 is executed to perform asynchronous processing on the multiple connection request messages, and the connection request messages are respectively connected to the corresponding receivers.
Finally, step S906 is executed, and after the connection is successful, data transmission is asynchronously performed between each client and its corresponding receiver.
By adopting the method provided by the embodiment of the invention, a plurality of connection requests of a plurality of clients are received, the connection requests are processed and are connected to the corresponding receivers, and data transmission is carried out between the clients and the corresponding receivers, wherein each execution link is carried out asynchronously. Asynchronous execution enables each execution link not to block the current thread to wait for processing completion, but to process in a mode of asynchronous callback of other threads, and when the completion operation of other threads is in an idle state, the callback notifies the idle thread, so that the idle thread can be fully utilized. Therefore, the thread calling is more flexible in an asynchronous execution mode, network blockage and even collapse caused by over-concentration on one or more threads are avoided, high performance and high concurrency of the cloud agent service are guaranteed, the responsiveness of the cloud agent service is improved, and user experience is further improved. In addition, asynchronous execution avoids the problem of calling thread blocking, further avoids the problem of network breakdown caused by certain thread blocking, also avoids the problem that one thread is blocked and other threads are in idle states, and enables network resources to be fully utilized.
Fig. 10 shows another process flow diagram of a pure asynchronous data transmission method of a cloud proxy according to an embodiment of the present invention. Referring to fig. 10, first, the cloud proxy server reads a client request (proxy _ read _ first), further reads a request header (proxy _ read _ headers) of the client, determines a corresponding receiver (process _ request _ headers) according to content of the client request, establishes a connection (proxy _ connect) with the corresponding receiver, sends the request header of the client to the receiver (proxy _ send _ headers), and then caches data provided by the client in a client side cache (process _ client _ buffer).
After reading the request message (proxy _ read _ first) of the client forwarded by the cloud proxy server, the receiver further reads the request header (proxy _ read _ headers) of the client, responds to the request (process _ response _ headers) of the client according to the content of the request header of the client, subsequently returns the queried data related to the request to the cloud proxy server (proxy _ send _ headers), and caches the queried data related to the request to a receiver side cache (process _ server _ buffer).
It should be added that the cloud proxy server reads the data provided by the client in the client-side cache and reads the data related to the client-side request in the receiver-side cache through the cloud proxy relay connection (proxy _ relay _ connection).
Specifically, the cloud proxy server asynchronously reads the request message headers of each connection request message until each request message header is completely received. In the embodiment of the invention, the receiver corresponding to each connection request message is determined according to the content of the head of each request message, so that the integrity of the head of each request message ensures the accuracy of determining the receiver corresponding to each connection request message.
The header of the request message is information for informing the server about the request of the client, and consists of key words/value pairs, wherein each row is paired, and the key words and the values are separated by English colons. The request message header includes at least the following: a browser type of the request, a list of content types that the client can identify, and a hostname of the request are generated.
According to the content of the header of each request message, determining the receiver corresponding to each connection request message, for example, determining the receiver according to the type of the request message in the header of the received request message, determining the corresponding search server according to the search type connection request, determining the corresponding download server by the download type server, or determining the receiver according to the host name in the header of the received request message.
When the receiving party is a server, the address of the remote server of the receiving party needs to be determined, and specifically, the address of the remote server corresponding to each client is asynchronously requested from a computer Domain Name System (DNS) server. The DNS server holds domain names and corresponding IP addresses for all hosts in the network and is able to convert domain names to IP addresses. The DNS server returns the address of the remote server corresponding to the receiver to the cloud proxy server, and the cloud proxy server is respectively and asynchronously connected with the remote server corresponding to the receiver.
And after the cloud proxy server is successfully asynchronously connected with each remote server, asynchronously receiving the request body of each client and forwarding the request body to the corresponding remote server. The body of the client typically consists of three parts, respectively: request line, message header, request body. The cloud proxy server asynchronously receives the body of each client, namely, the request line, the message header and the request body of the body of each client, and asynchronously stores the received body into the first memory of the cloud proxy server. In addition, the cloud proxy server also needs to asynchronously judge whether the body of the client is completely received, that is, whether the request line, the message header and the request text of the body of each client are completely received, and if it is determined that the body of each client is completely received, the received body is forwarded to the corresponding remote server.
And after receiving the body of each client asynchronously forwarded by the cloud proxy server, each remote server responds to the corresponding request of the client according to the request text of the body of each client, and asynchronously returns the inquired data related to the request to the cloud proxy server. And after the cloud proxy server asynchronously receives the data returned by each remote server, the data are asynchronously stored in the second memory. The data returned by the remote server is usually composed of three parts, namely a status line, a message header and a response body. The cloud proxy server also needs to asynchronously judge whether the data reception of each remote server is finished, namely whether the state line, the message header and the response text in the returned data of each remote server are completely received or not is judged, and if the returned data of each remote server are determined to be completely received, the received data are forwarded to the corresponding client at one time.
The following describes a pure asynchronous data transmission method of a cloud proxy according to an embodiment of the present invention.
It is set that 3 threads, i.e., thread 1, thread 2, and thread 3 are currently provided. The two clients simultaneously initiate a connection request message to the cloud proxy server, that is, the client 1 initiates a connection request message 1, and the client 2 initiates a connection request message 2. After receiving the connection request messages of the two clients, the cloud proxy server calls a thread 1 to process the data connection request message 1, and calls a thread 2 to process a connection request message 2. When the client 3 and the client 4 initiate the connection request message 3 and the connection request message 4 in sequence, the call thread 3 processes the connection request message 3 and the connection request message 4. At this time, the thread 3 is processing the connection request message 3 of the client 3, the thread 2 is in the connection request message 2 of the client 2, the thread 1 has already processed the connection request message 1 of the client 1, and is in an idle state, and the data request message 4 of the client 4 is not in a waiting state, but is transferred to the thread 1 for processing.
After receiving a connection request message of a client, the cloud proxy server asynchronously sends data in the request messages of the client 1, the client 2, the client 3 and the client 4 to corresponding receivers, the receivers respond to the request of the client, then the cloud proxy server asynchronously receives data returned by the receivers and asynchronously forwards the data sent by the corresponding receivers to the four clients, in the whole execution process, a thread set in each execution step does not only process a specified message, and if a certain thread is in an idle state, messages waiting for processing in other threads can be called and processed, so that pure asynchronous data transmission is realized.
Based on the same inventive concept, the embodiment of the invention also provides a cloud proxy server, which is used for supporting the pure asynchronous data transmission method of the cloud proxy provided by the embodiment of the invention. Fig. 11 shows a schematic structural diagram of a cloud proxy server according to an embodiment of the present invention. Referring to fig. 11, the apparatus includes at least:
an asynchronous receiving module 1110 adapted to asynchronously receive a plurality of connection request messages from a plurality of clients;
an asynchronous connection module 1120, coupled to the asynchronous receiving module 1110, adapted to perform asynchronous processing on a plurality of connection request messages, respectively connected to corresponding receivers;
an asynchronous transmission module 1130, coupled to the asynchronous connection module 1120, is adapted to transmit data asynchronously between each client and its corresponding receiver after the connection is successful.
In a preferred embodiment of the present invention, the asynchronous connection module 1120 is further adapted to:
asynchronously reading the request message header of each connection request message until each request message header is completely received;
and determining a receiver corresponding to each connection request message according to the content of the head of each request message.
In a preferred embodiment of the present invention, the asynchronous connection module 1120 is further adapted to:
when the receiver is a server, asynchronously requesting the DNS server for the address of the corresponding remote server for each client;
and receiving the addresses of the remote servers returned by the DNS server, and performing asynchronous connection respectively.
In a preferred embodiment of the present invention, the asynchronous transfer module 1130 is further adapted to:
asynchronously receiving the request body of each client and forwarding the request body to a corresponding receiver;
and asynchronously receiving the data returned by each receiver and forwarding the data to the corresponding client.
In a preferred embodiment of the present invention, the asynchronous receiving module 1110 is further adapted to: asynchronously receiving the body of each client;
referring to fig. 12, the cloud proxy server further includes: a first memory 1140, coupled to the asynchronous receiving module 1110, adapted to store data received by the asynchronous receiving module;
the asynchronous transfer module 1130 is further adapted to: asynchronously judging whether the body of each client is received completely; and if so, forwarding the received body to the corresponding receiver.
In a preferred embodiment of the present invention, the asynchronous receiving module 1110 is further adapted to: when the receiving party is a server side, asynchronously receiving data returned by each remote server;
referring to fig. 12, the cloud proxy server further includes: a second memory 1150, coupled to the asynchronous receiving module 1110, adapted to store data received by the asynchronous receiving module;
the asynchronous transfer module 1130 is further adapted to: asynchronously judging whether the data reception of each remote server is finished; and if so, forwarding the received data to the corresponding client at one time.
Based on the same inventive concept, the embodiment of the present invention further provides a pure asynchronous data transmission system, including:
a plurality of clients adapted to initiate a connection request message; the cloud proxy server of any one of the above is also included.
Any one or a combination of the preferred embodiments provided in the embodiments of the present invention may be applied to an asynchronous data transmission process between a client, a cloud proxy server, and a receiver, and fig. 13 is a schematic diagram illustrating a pure asynchronous data transmission system according to a preferred embodiment of the present invention. Referring to fig. 13, the system includes a client 1310, a cloud proxy 1320. To embody the integrity of the pure asynchronous data transmission system, fig. 13 further includes a terminal 1330 as a receiving side. The number of the clients and the terminals in fig. 13 is only an illustrative value, and may be thousands or hundreds or more in practical application, which is not limited in the present invention.
Example four
In the embodiment of the invention, after receiving a data request message initiated by a client, a cloud proxy server firstly checks whether the request hits a blacklist, and if not, the cloud proxy server processes the data request message in a pure asynchronous mode.
Specifically, before the cloud proxy performs asynchronous processing on the data request message of the client, firstly, a receiver corresponding to the data request message of the client is compared with a blacklist of url which is recorded by the cloud proxy and is prohibited from accessing, and the specific embodiment refers to steps S201 to S204 in fig. 2.
If the blacklist is not hit after the url corresponding to the request message of the client is obtained, a pure asynchronous manner is adopted to perform specific operations on the data request message, and specific steps refer to steps S904 to S906 of fig. 9, which is not described in detail herein.
EXAMPLE five
In the embodiment of the invention, the cloud proxy server processes the request message of the client, and in order to accelerate the processing process, the cloud proxy server firstly searches whether the data corresponding to the data request message is stored in the cache of the cloud proxy server. If yes, the data stored in the cache is directly returned to the client, and if not, the data request message is processed in a pure asynchronous mode.
Specifically, before the cloud proxy server performs asynchronization on the data request message of the client, it is first searched whether data corresponding to the data request message is stored in the cache of the cloud proxy server, in a specific embodiment, refer to steps S502 to S506 in fig. 5, if an execution result of step S504 in fig. 5 is that valid data hit by the data request message is not stored in the cache of the cloud proxy server, refer to steps S904 to S906 in fig. 9, which is not described in detail herein.
Step S604 in fig. 6 is an overview of the steps executed in steps S904 to S906 in fig. 9, and is not described herein again.
EXAMPLE six
In the embodiment of the invention, after receiving a data request message initiated by a client, a cloud proxy server firstly checks whether the request hits a blacklist, if so, the cloud proxy server informs the client that a url corresponding to the request message is set as access prohibition on the cloud proxy server, and if not, the cloud proxy server searches whether data corresponding to the data request message is stored in a cache of the cloud proxy server in order to accelerate a processing process. And if the cache stores the data corresponding to the data request message, directly returning the data stored in the cache to the client, and if the cache does not store the data corresponding to the data request message, processing the data request message in a pure asynchronous mode.
Specifically, the cloud proxy server performs asynchronous processing on a connection request message of a client, and first compares a receiver corresponding to the client request message with a blacklist of urls which are recorded by the cloud proxy server and access of which is prohibited, and in a specific embodiment, refer to steps S201 to S204 in fig. 2, if the blacklist is not hit after the url corresponding to the client request message is obtained, find whether data corresponding to the data request message is stored in a cache of the cloud proxy server, and in the specific embodiment, refer to steps S502 to S506 in fig. 5, if an execution result of step S504 in fig. 5 is that valid data hit by the data request message is not stored in the cache of the cloud proxy server, refer to steps S904 to S906 in fig. 9, which is not described in detail herein.
The embodiments provided in the fourth, fifth and sixth embodiments are all combined, and it should be noted that the combination of the present invention is not limited to the fourth, fifth and sixth embodiments, and other combinations may be formed according to actual situations.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that microprocessors or Digital Signal Processors (DSPs) may be used in practice to implement some or all of the functions of some or all of the components of a pure asynchronous data transmission system of a cloud proxy server and cloud proxy according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.

Claims (10)

1. A pure asynchronous data transmission method of a cloud agent is applied to a cloud agent server and comprises the following steps:
asynchronously receiving a plurality of connection request messages from a plurality of clients;
searching whether effective data hit by the data request message is stored in a cache of the cloud proxy server;
if the cache of the cloud proxy server does not store the effective data hit by the data request message, performing asynchronous processing on the plurality of connection request messages, and respectively connecting the connection request messages to corresponding receivers;
after the connection is successful, data transmission is asynchronously carried out between each client and the corresponding receiver;
wherein the asynchronously processing the plurality of connection request messages comprises:
asynchronously reading the request message header of each connection request message until each request message header is completely received;
determining a receiver corresponding to each connection request message according to the content of the head of each request message;
the asynchronous data transmission between each client and the corresponding receiver comprises the following steps:
asynchronously receiving the request body of each client and forwarding the request body to a corresponding receiver;
asynchronously receiving data returned by each receiver and forwarding the data to corresponding clients;
the asynchronous receiving, asynchronous processing, asynchronous data transmission and asynchronous reading comprise: a thread in an idle state calls and processes messages waiting to be processed in other threads.
2. The method of claim 1, wherein the respectively connecting to the corresponding recipients comprises:
when the receiver is a server, asynchronously requesting the DNS server for the address of the corresponding remote server for each client;
and receiving the addresses of the remote servers returned by the DNS, and performing asynchronous connection respectively.
3. The method of claim 1, wherein the asynchronously receiving the request body of each client and forwarding to the corresponding receiver comprises:
asynchronously receiving the body of each client;
asynchronously saving the received body to a first memory;
asynchronously judging whether the body of each client is received completely;
and if so, forwarding the received body to the corresponding receiver.
4. The method of claim 3, wherein the asynchronously receiving the data returned by each receiver and forwarding the data to the corresponding client comprises:
when the receiving party is a server side, asynchronously receiving data returned by each remote server;
asynchronously saving the received data to a second memory;
asynchronously judging whether the data reception of each remote server is finished;
and if so, forwarding the received data to the corresponding client at one time.
5. The method of claim 4, wherein the first storage and/or the second storage comprise a cache and/or a memory.
6. A cloud proxy server, comprising:
an asynchronous receiving module adapted to asynchronously receive a plurality of connection request messages from a plurality of clients;
the searching module is suitable for searching whether the cache of the cloud proxy server stores the effective data hit by the data request message;
the asynchronous connection module is suitable for performing asynchronous processing on the plurality of connection request messages and respectively connecting the connection request messages to corresponding receivers if the cache of the cloud proxy server does not store valid data hit by the data request messages;
the asynchronous transmission module is suitable for asynchronously transmitting data between each client and the corresponding receiver after the connection is successful;
the asynchronous connection module is also suitable for asynchronously reading the request message headers of the connection request messages until the request message headers are completely received; determining a receiver corresponding to each connection request message according to the content of the head of each request message;
the asynchronous transmission module is also suitable for asynchronously receiving the request body of each client and forwarding the request body to a corresponding receiver; asynchronously receiving data returned by each receiver and forwarding the data to corresponding client
The asynchronous receiving, asynchronous processing, asynchronous data transmission and asynchronous reading comprise: a thread in an idle state calls and processes messages waiting to be processed in other threads.
7. The cloud proxy server of claim 6, wherein the asynchronous connection module is further adapted to:
when the receiver is a server, asynchronously requesting the DNS server for the address of the corresponding remote server for each client;
and receiving the addresses of the remote servers returned by the DNS, and performing asynchronous connection respectively.
8. The cloud proxy server of claim 6,
the asynchronous reception module is further adapted to: asynchronously receiving the body of each client;
the cloud proxy server further comprises: a first memory adapted to store data received by the asynchronous reception module;
the asynchronous transfer module is further adapted to: asynchronously judging whether the body of each client is received completely; and if so, forwarding the received body to the corresponding receiver.
9. The cloud proxy server of claim 8, wherein the asynchronous receive module is further adapted to: when the receiving party is a server side, asynchronously receiving data returned by each remote server;
the cloud proxy server further comprises: a second memory adapted to store data received by the asynchronous reception module;
the asynchronous transfer module is further adapted to: asynchronously judging whether the data reception of each remote server is finished; and if so, forwarding the received data to the corresponding client at one time.
10. A pure asynchronous data transmission system comprising:
a plurality of clients adapted to initiate a connection request message;
further comprising the cloud proxy server of any of claims 6-9.
CN201510975253.5A 2015-12-22 2015-12-22 Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy Expired - Fee Related CN106911736B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510975253.5A CN106911736B (en) 2015-12-22 2015-12-22 Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510975253.5A CN106911736B (en) 2015-12-22 2015-12-22 Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy

Publications (2)

Publication Number Publication Date
CN106911736A CN106911736A (en) 2017-06-30
CN106911736B true CN106911736B (en) 2021-11-26

Family

ID=59200055

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510975253.5A Expired - Fee Related CN106911736B (en) 2015-12-22 2015-12-22 Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy

Country Status (1)

Country Link
CN (1) CN106911736B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109615758B (en) * 2018-10-22 2022-10-04 中国平安人寿保险股份有限公司 Lottery drawing data processing method and device, storage medium and server

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1512707A (en) * 2002-12-27 2004-07-14 �Ҵ���˾ Surrogate server, access control method and access control program
CN101325561A (en) * 2007-06-12 2008-12-17 阿里巴巴集团控股有限公司 Method, apparatus and system for processing electronic mail
CN103197968A (en) * 2013-03-18 2013-07-10 焦点科技股份有限公司 Thread pool processing method and system capable of fusing synchronous and asynchronous features
CN104219284A (en) * 2014-08-11 2014-12-17 华侨大学 Server designing method based on semi-synchronization, semi-synchronization and pipe filter mode
CN104243620A (en) * 2014-10-17 2014-12-24 浪潮电子信息产业股份有限公司 High-performance WEB production environment architecture

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9112808B2 (en) * 2007-02-06 2015-08-18 At&T Intellectual Property Ii, L.P. Devices, systems, and methods for providing data
US8832159B2 (en) * 2012-05-22 2014-09-09 Google Inc. Systems and methods for asynchronous schema changes

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1512707A (en) * 2002-12-27 2004-07-14 �Ҵ���˾ Surrogate server, access control method and access control program
CN101325561A (en) * 2007-06-12 2008-12-17 阿里巴巴集团控股有限公司 Method, apparatus and system for processing electronic mail
CN103197968A (en) * 2013-03-18 2013-07-10 焦点科技股份有限公司 Thread pool processing method and system capable of fusing synchronous and asynchronous features
CN104219284A (en) * 2014-08-11 2014-12-17 华侨大学 Server designing method based on semi-synchronization, semi-synchronization and pipe filter mode
CN104243620A (en) * 2014-10-17 2014-12-24 浪潮电子信息产业股份有限公司 High-performance WEB production environment architecture

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于Servlet3.0的Web异步处理的研究;李林等;《科技风》;20110131;第19-20、22页 *
轻量级高并发Web服务器的研究与实现;杨小娇;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150515;第I139-41页 *

Also Published As

Publication number Publication date
CN106911736A (en) 2017-06-30

Similar Documents

Publication Publication Date Title
CN106911735B (en) Data acquisition method and device
US8285808B1 (en) Loading of web resources
US8495220B2 (en) Managing CDN registration by a storage provider
US9172674B1 (en) Managing request routing information utilizing performance information
US8789198B2 (en) Triggering a private browsing function of a web browser application program
EP2985705A2 (en) Webpage access method and apparatus, and router
US20140280691A1 (en) Updating dynamic content in cached resources
US7987243B2 (en) Method for media discovery
US10735528B1 (en) Geographic relocation of content source in a content delivery network
US8972513B2 (en) Content caching
CN110851680B (en) Web crawler identification method and device
WO2017202255A1 (en) Page display method and apparatus, and client device
US8914542B1 (en) Content caching
CN106911733B (en) Cloud proxy website access method and device
US20210406217A1 (en) Method for processing resource description file, page resource acquisition method, and intermediate server
US9998559B2 (en) Preemptive caching of data
CN106911736B (en) Cloud proxy server, pure asynchronous data transmission method and system of cloud proxy
CN111783005A (en) Method, apparatus and system for displaying web page, computer system and medium
CN110825603A (en) Page first loading time determining method and device, electronic equipment and storage medium
KR102196403B1 (en) Reduced redirection
US11645033B2 (en) Method and apparatus for accessing proprietary resources on a Co-Browse session
US9307052B1 (en) Edge side landing pages
WO2008011314A2 (en) Conditional url for computer devices
CN110224973B (en) Multi-server parallel implementation method based on programming game, electronic equipment and medium
US20080016219A1 (en) Conditional URL For Computer Devices

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220822

Address after: No. 9-3-401, No. 39, Gaoxin 6th Road, Binhai Science and Technology Park, High-tech Zone, Binhai New District, Tianjin 300000

Patentee after: 3600 Technology Group Co.,Ltd.

Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park)

Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd.

Patentee before: Qizhi software (Beijing) Co.,Ltd.

CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20211126