CN102075570A

CN102075570A - Method for implementing HTTP (hyper text transport protocol) message caching mechanism based on keywords

Info

Publication number: CN102075570A
Application number: CN2010106163150A
Authority: CN
Inventors: 张炎; 朱鹏飞
Original assignee: NANJING ZHONGXING SPECIAL SOFTWARE CO Ltd
Current assignee: Nanjing Sinovatio Technology LLC
Priority date: 2010-12-31
Filing date: 2010-12-31
Publication date: 2011-05-25
Anticipated expiration: 2030-12-31
Also published as: CN102075570B

Abstract

The invention discloses a method for implementing an HTTP message caching mechanism based on keywords. The method comprises the following steps: matching an HTTP request message of a client with a corresponding configuration item; and searching the request message of the client according to the keywords in the configuration item, and determining whether to give an HTTP response to the user from a cache or to acquire a complete HTTP response from a server or to acquire a partial HTTP response from the server and modify the cached message according to the configuration so as to compose a complete HTTP response. By using the method provided by the invention, data traffic between a proxy server and a WEB server is effectively reduced; latency for the client to access a WEB page is effectively shortened; the WEB page requested by the client each time can be ensured to be the updated page; the effects of reducing the data traffic and shortening the latency of the WEB page related to the user status and browser type at the client can also be achieved; and the cache of the WEB page can be optimized according to configuration files.

Description

A kind of implementation method of the HTTP message caching mechanism based on keyword

Technical field

The invention belongs to a kind of HTTP network agent caching method, in particular for reverse proxy, the WEB server is netted the method for scenes such as acceleration more, specifically a kind of implementation method of the HTTP message caching mechanism based on keyword.

Background technology

Acting server is the station server between browser and Web server, had after it, browser is not directly to reach back webpage to Web server but send request to acting server, the Request signal can be delivered to acting server earlier, is fetched the needed information of browser and is sent to your browser by acting server.

HTTP Proxy, its function be exactly the agency network user reach the network information; Also being used for WEB class website now quickens.Common HTTP Proxy is divided into no caching mechanism, and caching mechanism arranged.

The HTTP Proxy of no caching mechanism: acting server does not detect the content of user's request, and is own to the server requests related content, returns to the user; There are the following problems for it: the HTTP Proxy flow of no caching mechanism is big, network delay is big; All data all need from client, acting server, and the WEB server transmits, and network traffics do not have and dwindle, and each step Network Transmission all has network delay, and it comprises:

Request: client browser-acting server-WEB server

Response: WEB server-acting server-client browser

Each steps flow chart all exists complete data traffic to produce; The all corresponding generation that produces network delay of each steps flow chart.

The HTTP Proxy that caching mechanism is arranged: acting server has a very big Cache, it has very big memory space, it constantly will newly obtain data storing to the memory of its this machine, if the data that browser is asked have existed on the memory of its this machine and have been up-to-date, it does not just fetch data from Web server again so, and directly the data on the memory are sent to user's browser, so just can significantly improve surfing and efficient.But there are the following problems for it:

What 1, client might get access to is not the up-to-date page.

For example: acting server is at buffer update in the cycle, the WEB server update WEB page.The page that client is at this time asked is the old page before not upgrading.

2, can not the identical URL of buffer memory, the still page relevant with User Status.

For example: after certain WEB page needs the user to land, the user name difference that the fixed position shows on the page, this page can't carry out buffer memory.

3, can not the identical URL of buffer memory, the page relevant with browser type.

For example: certain WEB page uses dynamic sentence, and IE is different with the script that the Firefox browser uses, and this page can't carry out buffer memory.

Summary of the invention

The objective of the invention is at existing the problems referred to above the implementation method of a kind of HTTP message caching mechanism based on keyword of proposition.

Technical scheme of the present invention is:

A kind of implementation method of the HTTP message caching mechanism based on keyword may further comprise the steps:

(a), the user sets up configuration file, this configuration file comprises a plurality of configuration items; Each configuration item includes domain name, the page, Cookie and the user browser type that needs processing, is provided with corresponding processing mode corresponding to each configuration item; Domain name in each configuration item, the page, Cookie and user browser type are equipped with a plurality of keywords;

(b), setting up message handles needed buffer memory, startup processing threads, binding message processing port;

(c), read configuration file, in internal memory, each configuration item in the configuration file is conspired to create the configuration chained list;

(d), when the client access acting server, acting server and client connect, and obtain user's request message;

(e), according to client's request message, the keyword in the retrieval configuration chained list, configuration item under determining is handled according to the method in the configuration item;

(f) if the processing mode in the step (e) need be obtained all from server, perhaps the part message then connects with server, obtains corresponding message from server; Otherwise, directly from the buffering internal memory, obtain corresponding message;

(g), the http response message that step (f) was handled sends to the client user.

Among the present invention: client's HTTP request message needs the corresponding configuration item of coupling earlier; Need be according to the keyword search client requests message in the configuration item, and determine it is from buffer memory, to give to user's http response, and still obtain complete http response from server, still obtain the part http response from server, and according to configuration modification buffer memory message, synthetic complete http response.

Beneficial effect of the present invention:

The present invention can be according to configuration file, and the feasible static page that has nothing to do with all states of client carries out caching process.

The present invention can make whether renewal of the page only detects according to the HTTP head, and this page of unnecessary download compares according to configuration file.

The present invention can solve and client state the problem of the relevant page cache of browser type according to configuration file.

The present invention effectively reduces the data traffic between acting server and the WEB server; Effectively shorten client and obtain the time of the wait time-delay of the WEB page; Can guarantee that the WEB page that client is asked at every turn is the up-to-date page; With client user's state, the WEB page that browser type is relevant also can reach the minimizing data flow, shortens the effect of time-delay; Simultaneously, can optimize the buffer memory of the WEB page according to configuration file.

Embodiment

The present invention is further illustrated below in conjunction with embodiment.

A kind of implementation method of the HTTP message caching mechanism based on keyword, it may further comprise the steps:

During concrete enforcement:

A kind of implementation method of the HTTP message caching mechanism based on keyword, one, need read configuration from configuration file and form the search index chained list; Two, needs and client connect, and obtain the HTTP request message from client; Three, need judge which kind of processing mode each HTTP request uses is classic mode according to keyword; Four, the processing mode of selecting according to previous step forms complete response message; Five, give client with response message.

1, all WEB page graduation are handled:

The WEB page that a, static page often do not change.

B, static page are distinguished the WEB page of client type.

C, the dynamic WEB page with buffer memory value.

The dynamic WEB page that d, no buffer memory are worth.

Illustrate: the meaning of above-mentioned a is that the message content of the WEB page is only relevant with URL, and is irrelevant with other all states.

The meaning of above-mentioned b is that the message content of the WEB page is not only relevant with URL, also with asking interior other fields of HTTP head relation is arranged.

The meaning of above-mentioned c is, some dynamic page is worth buffer memory and handles, and the google homepage after landing such as different user only has the user name difference in the upper right corner, and all the other messages are all the same.

The meaning of above-mentioned d is, some dynamic page, and difference is very big, is unworthy carrying out buffer memory, and such as the WEB mailbox of different user, full page is all inequality except framework, and the buffer memory cost is big, and effect is bad.

2, configuration file item, as follows:

<[206]>

DomainID=21

http_hdr_up={[GET?/loginfaild.srf]}&{[Host:?login.live.com]}|{[HOST:?login.live.com]}

http_hdr_dwn=

CheckAck=0

ChangeUp=0

ChangeDown=1

pktclient=cms_pkt//202.up.cmspkt

pktserver=cms_pkt//206.cmspkt

SSL=1

srch_dwn_val={[Set-Cookie:?MSPOK=]}#{[;]}*{[64]}@{[246]}

srch_dwn_val={[https://login.live.com/ppsecure/post.srf?bk=]}#{["]}*{[10]}@{[4102]}

srch_dwn_val={[input?type="hidden"?name="PPFT"]}&{[value="]}#{[/>]}*{[256]}@{[8595]}

srch_dwn_val={[base?href="]}&{[login.live.com/pp]}#{["]}*{[6]}@{[765]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[1561]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[1899]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[3129]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[3816]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[4947]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[5499]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[6310]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[8023]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[9094]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[9358]}

pktwarn=cms_pkt//202_warn.cmspkt

Illustrate:

A.＜[206]〉expression is configuration sequence number, and each configuration item all has a numbering, does not repeat.

B.DomainID represents to change that configuration item represents is under what domain name.

C.http_hdr_up represents to detect project in user's request message, user search URL, and the user browser type, Cookie is etc. field

D.http_hdr_down represents to detect WEB server downlink message and whether contains some specific fields.

E.CheckAck represents whether whether need to detect the server response message satisfies condition, and is satisfied then revise from buffer memory and give the user, saves Internet resources and time, do not satisfy then to obtain message from the WEB server and send the user to, guarantees message correctly.

Whether f.ChangeUp need represents user's request is made amendment, the same such as message text, the Cookie difference of different user is only revised user's GET request so and is asked for HEAD, it is just passable only need to obtain heading from server, does not need to obtain message body content again.

G.ChangeDown represents to revise server response, and as the example of f the inside, there has been the downlink message body buffer memory the inside, the message HTTP head of WEB server is revised into the message body before, return to client again, be exactly a complete downlink message.

H.pktclient represents up modification message, proxy server caches client uplink message file path.

I.pktserver represents descending modification message, proxy server caches WEB server downlink message file path.

J.SSL represents to satisfy whether the message that changes a http_hdr_up and http_hdr_dwon configuration item is that HTTPS encrypts message content.

K.srch_dwn_val represents concrete operations mode that downlink message is made amendment, the content retrieval sign: { [revise character string before character] } #{[revises character after character string] } * [maximum length] } @{[pointer offset amount] }, can

L.srch_up_val represents concrete mode that uplink message is made amendment, the content retrieval sign: { [revise character string before character] } #{[revises character after character string] } * [maximum length] } @{[pointer offset amount] }

J.pktwarn represents that srch_dwn_val or srch_up_val make a mistake if match under the situation of http_hdr_up and http_hdr_down, produces the alarm behavior.

3. configuration file

The configuration text file that configuration file is made up of a plurality of configuration items

4. handling process

4.1 main thread.

A) initialization system comprises internal memory initialization, global variable initialization, Socket communication initialization.

B) Bind communication port.

C) read configuration file, the initial configuration chained list.

D) start customer end A ccept thread.

E) start Connect and Send thread.

F) start the select processing threads.

G) start the message processing threads.

H) enter the order line wait state.

4.2Accept thread

A. thread initialization.

B. the Accept function is called in unblock

C. after calling successfully the Socket handle put into and wait for the Select array

D. recursive call b.

4.3Connect and Send thread

A. thread initialization.

B. to be sent, connect formation and obtain data.

If c. to be connected, it is overtime that connect is set, and connects the WEB server, and the Socket handle is sent into and waited in the Select array.

If d. to be sent, call send and send to the WEB server.

E. recursive call b

4.4Select processing threads.

A. thread initialization.

B. empty FD

C. traversal is treated Select array, FD SET.

D. call Select and go out pending Socket.

E. put into pending Socket formation.

F. b circulates.

4.5 message processing threads.

A. thread initialization.

B. read out pending Socket from pending Socket formation.

The pending Socket of c.Read

If d. close then close Socket.

E. under the situation that d satisfies, search the WEB server that changes the Sokcet correspondence, the perhaps Socket of client, and close.

F. the message that reads out being configured file detects.

G. testing result is put into the Send formation.

H. b circulates

4.6 message configuration detection processing module

A. HOST field in the detection messages head

B. find the configuration chained list of corresponding Domainid.

If c. client data is according to http_hdr_up detection messages URL, Host, Cookie, User-Agent, fields such as Content-type.

If d. WEB server data and CheckAck=1 need detection messages.

E. under the situation of d of satisfying condition, according to http_hdr_down detection messages response, Location, fields such as Set-Cookie.

If f. need message is made amendment under the situation of client data and ChangeUp=1.

If g. need message is made amendment under the situation of WEB server data and ChangeDown=1.

If h. client data pktclient is not empty, then from internal memory, find the corresponding cache message.Do not need to revise under the situation of message and directly return.

If i. WEB server data pktserver is not empty, then from internal memory, find the corresponding cache message.Do not need to revise under the situation of message and directly return.

If j. client data srch_up_val is not empty, then revise the message of buffer memory in the internal memory according to the mode of regulation in the configuration, all srch_up_val return after revising and finishing.

If k. WEB server data srch_dwn_val is not empty, then revise the message of buffer memory in the internal memory according to the mode of regulation in the configuration, all srch_dwn_val return after revising and finishing.

L. at above-mentioned j, do not meet, when perhaps wrong situation takes place, produce alarm according to pktwarn if run into the modification message in the k process.

The part that the present invention does not relate to prior art that maybe can adopt all same as the prior art is realized.

Claims

1. implementation method based on the HTTP message caching mechanism of keyword is characterized in that it may further comprise the steps:

2. according to the implementation method of a kind of HTTP message caching mechanism based on keyword described in the claim 1, it is characterized in that client's HTTP request message, need the corresponding configuration item of coupling earlier; Need be according to the keyword search client requests message in the configuration item, and determine it is from buffer memory, to give to user's http response, and still obtain complete http response from server, still obtain the part http response from server, and according to configuration modification buffer memory message, synthetic complete http response.