CN102075570B

CN102075570B - Method for implementing HTTP (hyper text transport protocol) message caching mechanism based on keywords

Info

Publication number: CN102075570B
Application number: CN 201010616315
Authority: CN
Inventors: 张炎; 朱鹏飞
Original assignee: NANJING ZHONGXING SPECIAL SOFTWARE CO Ltd
Current assignee: Nanjing Sinovatio Technology LLC
Priority date: 2010-12-31
Filing date: 2010-12-31
Publication date: 2013-01-30
Anticipated expiration: 2030-12-31
Also published as: CN102075570A

Abstract

The invention discloses a method for implementing an HTTP message caching mechanism based on keywords. The method comprises the following steps: matching an HTTP request message of a client with a corresponding configuration item; and searching the request message of the client according to the keywords in the configuration item, and determining whether to give an HTTP response to the user from a cache or to acquire a complete HTTP response from a server or to acquire a partial HTTP response from the server and modify the cached message according to the configuration so as to compose a complete HTTP response. By using the method provided by the invention, data traffic between a proxy server and a WEB server is effectively reduced; latency for the client to access a WEB page is effectively shortened; the WEB page requested by the client each time can be ensured to be the updated page; the effects of reducing the data traffic and shortening the latency of the WEB page related to the user status and browser type at the client can also be achieved; and the cache of the WEB page can be optimized according to configuration files.

Description

A kind of implementation method of the HTTP message caching mechanism based on keyword

Technical field

The invention belongs to a kind of HTTP network agent caching method, in particular for reverse proxy, the WEB server is netted the method for the scenes such as acceleration more, specifically a kind of implementation method of the HTTP message caching mechanism based on keyword.

Background technology

Acting server is the station server between browser and Web server, had after it, browser is not directly to reach back webpage to Web server but send request to acting server, the Request signal can be delivered to first acting server, is fetched the needed information of browser and is sent to your browser by acting server.

HTTP Proxy, its function are exactly that the agency network user reaches to get the network information; Also being used for now WEB class website accelerates.Common HTTP Proxy is divided into without caching mechanism, and caching mechanism arranged.

HTTP Proxy without caching mechanism: acting server does not detect the content of user's request, and is own to the server request related content, returns to the user; There are the following problems for it: the HTTP Proxy flow without caching mechanism is large, network delay is large; All data all need from client, acting server, and the WEB server transmits, and network traffics are without dwindling, and each step Internet Transmission has network delay, and it comprises:

Request: client browser-〉 acting server-〉 WEB server

Response: WEB server-〉 acting server-〉 client browser

Each steps flow chart all exists complete data traffic to produce; Each steps flow chart is the corresponding generation that produces network delay all.

The HTTP Proxy that caching mechanism is arranged: acting server has a very large Cache, it has very large memory space, it constantly will newly obtain data storing to the memory of its machine, if the data that browser is asked have existed on the memory of its machine and have been up-to-date, it does not just fetch data from Web server again so, and the direct browser that data on the memory is sent to the user so just can significantly improve surfing and efficient.But there are the following problems for it:

What 1, client might get access to is not the up-to-date page.

For example: acting server is at buffer update in the cycle, the WEB server update WEB page.The page that client is at this time asked is the old page before not upgrading.

2, can not the identical URL of buffer memory, the still page relevant with User Status.

For example: after certain WEB page needed the user to log in, the user name that the fixed position shows on the page was different, and this page can't carry out buffer memory.

3, can not the identical URL of buffer memory, the page relevant with browser type.

For example: certain WEB page uses dynamic sentence, and IE is different with the script that the Firefox browser uses, and this page can't carry out buffer memory.

Summary of the invention

The objective of the invention is for existing the problems referred to above the implementation method of a kind of HTTP message caching mechanism based on keyword of proposition.

Technical scheme of the present invention is:

A kind of implementation method of the HTTP message caching mechanism based on keyword may further comprise the steps:

(a), the user sets up configuration file, this configuration file comprises a plurality of configuration items; Each configuration item includes needs domain name to be processed, the page, Cookie and user browser type, is provided with corresponding processing mode corresponding to each configuration item; Domain name in each configuration item, the page, Cookie and user browser type are equipped with a plurality of keywords;

(b), setting up message processes needed buffer memory, startup processing threads, binding message processing port;

(c), read configuration file, in internal memory, each configuration item in the configuration file is conspired to create the configuration chained list;

(d), when the client access acting server, acting server and client connect, and obtain user's request message;

(e), according to client's request message, the keyword in the retrieval configuration chained list, configuration item under determining is processed according to the method in the configuration item;

(f) if the processing mode in the step (e) need to be obtained all from server, perhaps the part message then connects with server, obtains corresponding message from server; Otherwise, directly from buffer memory, obtain corresponding message;

(g), the http response message of step (f) being processed sends to the client user.

Among the present invention: client's HTTP request message needs first configuration item corresponding to coupling; Need to be according to the keyword search client requests message in the configuration item, and determine it is from buffer memory, to give to user's http response, or obtain complete http response from server, or obtain the part http response from server, and according to configuration modification buffer memory message, synthetic complete http response.

Beneficial effect of the present invention:

The present invention can be according to configuration file, so that carry out caching process with the irrelevant static page of all states of client.

The present invention can be according to configuration file, so that whether renewal of the page only detects according to the HTTP head, this page of unnecessary download compares.

The present invention can according to configuration file, solve and client state the problem of the relevant page cache of browser type.

The present invention effectively reduces the data traffic between acting server and the WEB server; Effectively shorten client and obtain the time of the wait time-delay of the WEB page; Can guarantee that the each WEB page of asking of client is the up-to-date page; With client user's state, the WEB page that browser type is relevant also can reach the minimizing data flow, shortens the effect of time-delay; Simultaneously, can optimize according to configuration file the buffer memory of the WEB page.

Embodiment

The present invention is further illustrated below in conjunction with embodiment.

A kind of implementation method of the HTTP message caching mechanism based on keyword, it may further comprise the steps:

During implementation:

A kind of implementation method of the HTTP message caching mechanism based on keyword, one, need to read configuration from configuration file and form the search index chained list; Two, needs and client connect, and obtain the HTTP request message from client; Three, need to judge which kind of processing mode each HTTP request uses is classic mode according to keyword; Four, the processing mode of selecting according to previous step forms complete response message; Five, give client with response message.

1, all WEB page graduation are processed:

The WEB page that a, static page often do not change.

B, static page are distinguished the WEB page of client type.

C, the dynamic web pages with buffer memory value.

D, the dynamic web pages that is worth without buffer memory.

Illustrate: the meaning of above-mentioned a is that the message content of the WEB page is only relevant with URL, and is irrelevant with other all states.

The meaning of above-mentioned b is that the message content of the WEB page is not only relevant with URL, also with asking interior other fields of HTTP head relation is arranged.

The meaning of above-mentioned c is, some dynamic page is worth buffer memory and processes, and the google homepage after logging in such as different user only has the user name in the upper right corner different, and all the other messages are all the same.

The meaning of above-mentioned d is, some dynamic page, and difference is very large, is unworthy carrying out buffer memory, and such as the WEB mailbox of different user, full page is all not identical except framework, and the buffer memory cost is large, and effect is bad.

2, configuration file item, as follows:

<[206]>

DomainID=21

http_hdr_up={[GET?/loginfaild.srf]}&{[Host:?login.live.com]}|{[HOST:?login.live.com]}

http_hdr_dwn=

CheckAck=0

ChangeUp=0

ChangeDown=1

pktclient=cms_pkt//202.up.cmspkt

pktserver=cms_pkt//206.cmspkt

SSL=1

srch_dwn_val={[Set-Cookie:?MSPOK=]}#{[;]}*{[64]}@{[246]}

srch_dwn_val={[https://login.live.com/ppsecure/post.srf?bk=]}#{["]}*{[10]}@{[4102]}

srch_dwn_val={[input?type="hidden"?name="PPFT"]}&{[value="]}#{[/>]}*{[256]}@{[8595]}

srch_dwn_val={[base?href="]}&{[login.live.com/pp]}#{["]}*{[6]}@{[765]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[1561]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[1899]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[3129]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[3816]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[4947]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[5499]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[6310]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[8023]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[9094]}

srch_dwn_val={[.css?x=]}#{["]}*{[12]}@{[9358]}

pktwarn=cms_pkt//202_warn.cmspkt

Illustrate:

A.＜[206]〉expression is configuration sequence number, and each configuration item all has a numbering, does not repeat.

B.DomainID represents to change that configuration item represents is under what domain name.

C.http_hdr_up represents to detect project in user's request message, user search URL, and the user browser type, Cookie is etc. field

D.http_hdr_down represents to detect WEB server downlink message and whether contains some specific fields.

E.CheckAck represents whether whether need to detect the server response message satisfies condition, and is satisfied then revise from buffer memory and give the user, saves Internet resources and time, do not satisfy then to obtain message from the WEB server and send the user to, guarantees message correctly.

Whether f.ChangeUp need represents user's request is made amendment, the same such as message text, only the Cookie of different user is different, and the GET request of revising so the user is the HEAD request, only need to obtain heading from server just passable, not need to obtain again message body content.

G.ChangeDown represents to revise the server response, and such as the example of f the inside, there has been the downlink message body buffer memory the inside, and the message HTTP head of WEB server is revised before the message body, returns to client again, has been exactly a complete downlink message.

H.pktclient represents up modification message, proxy server caches client uplink message file path.

I.pktserver represents descending modification message, proxy server caches WEB server downlink message file path.

J.SSL represents to satisfy whether the message that changes a http_hdr_up and http_hdr_dwon configuration item is that HTTPS encrypts message content.

K.srch_dwn_val represents concrete operations mode that downlink message is made amendment, the content retrieval sign: { [revise character string before character] } #{[revises character after character string] } * { [maximum length] }@{ [pointer offset amount] }, can

L.srch_up_val represents concrete mode that uplink message is made amendment, the content retrieval sign: { [revise character string before character] } #{[revises character after character string] } * { [maximum length] }@{ [pointer offset amount] }

J.pktwarn represents that srch_dwn_val or srch_up_val make a mistake if match in the situation of http_hdr_up and http_hdr_down, produces the alarm behavior.

3. configuration file

The configuration text file that configuration file is comprised of a plurality of configuration items

4. handling process

4.1 main thread.

A) initialization system comprises internal memory initialization, global variable initialization, Socket communication initialization.

B) Bind communication port.

C) read configuration file, the initial configuration chained list.

D) start customer end A ccept thread.

E) start Connect and Send thread.

F) start the select processing threads.

G) start the message processing threads.

H) enter the order line wait state.

4.2Accept thread

A. thread initialization.

B. Non-blocking call Accept function

C. call successfully afterwards the Socket handle to be put into and wait for the Select array

D. recursive call b.

4.3Connect and Send thread

A. thread initialization.

B. to be sent, connect formation and obtain data.

If c. to be connected, connect is set overtime, connect the WEB server, the Socket handle is sent into and is waited in the Select array.

If d. to be sent, call send and send to the WEB server.

E. recursive call b

4.4Select processing threads.

A. thread initialization.

B. empty FD

C. traversal is treated Select array, FD SET.

D. call Select and go out pending Socket.

E. put into pending Socket formation.

F. b circulates.

4.5 message processing threads.

A. thread initialization.

B. read out pending Socket from pending Socket formation.

The pending Socket of c.Read

If d. close then close Socket.

E. in the situation that d satisfies, search and change WEB server corresponding to Sokcet, the perhaps Socket of client, and close.

F. the message that reads out being configured file detects.

G. testing result is put into the Send formation.

H. b circulates

4.6 message configuration detection processing module

A. HOST field in the detection messages head

B. find the configuration chained list of corresponding Domainid.

If c. client data is according to http_hdr_up detection messages URL, Host, Cookie, User-Agent, the fields such as Content-type.

If d. WEB server data and CheckAck=1 need detection messages.

E. in the situation of d of satisfying condition, according to http_hdr_down detection messages response, Location, the fields such as Set-Cookie.

If f. need message is made amendment in the situation of client data and ChangeUp=1.

If g. need message is made amendment in the situation of WEB server data and ChangeDown=1.

If h. client data pktclient is not empty, then from internal memory, find corresponding buffer memory message.Do not need to revise in the situation of message and directly return.

If i. WEB server data pktserver is not empty, then from internal memory, find corresponding buffer memory message.Do not need to revise in the situation of message and directly return.

If j. client data srch_up_val is not empty, then revise the message of buffer memory in the internal memory according to the mode of regulation in the configuration, all srch_up_val return after revising and finishing.

If k. WEB server data srch_dwn_val is not empty, then revise the message of buffer memory in the internal memory according to the mode of regulation in the configuration, all srch_dwn_val return after revising and finishing.

L. at above-mentioned j, do not meet if run into the modification message in the k process, when perhaps wrong situation occurs, produce alarm according to pktwarn.

The part that the present invention does not relate to all prior art that maybe can adopt same as the prior art is realized.

Claims

1. implementation method based on the HTTP message caching mechanism of keyword is characterized in that it may further comprise the steps:

2. a kind of implementation method of the HTTP message caching mechanism based on keyword described in according to claim 1 is characterized in that client's HTTP request message needing first configuration item corresponding to coupling; Need to be according to the keyword search client requests message in the configuration item, and determine it is from buffer memory, to give user's http response, or obtain complete http response from server, or obtain the part http response from server, and according to configuration modification buffer memory message, synthetic complete http response.