WO2011110042A1 - Web request scheduling method and system - Google Patents

Web request scheduling method and system Download PDF

Info

Publication number
WO2011110042A1
WO2011110042A1 PCT/CN2010/079740 CN2010079740W WO2011110042A1 WO 2011110042 A1 WO2011110042 A1 WO 2011110042A1 CN 2010079740 W CN2010079740 W CN 2010079740W WO 2011110042 A1 WO2011110042 A1 WO 2011110042A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
cache
data
path information
identification code
Prior art date
Application number
PCT/CN2010/079740
Other languages
French (fr)
Chinese (zh)
Inventor
陈普
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2010/079740 priority Critical patent/WO2011110042A1/en
Publication of WO2011110042A1 publication Critical patent/WO2011110042A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/957Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching

Abstract

The present invention provides a Web request scheduling method and system. The method includes the following steps: analyzing the received Web request to obtain file path information, and transmitting the cache request carrying the file path information; obtaining the file identifier code corresponding to the file path information, obtaining and returning the cache data corresponding to the file identifier code, wherein, the correspondence between the file identifier code and the cache data is one to one. The embodiments of the present invention can improve cache utilization rate and reduce costs.

Description

 Web request scheduling method and system

Technical field

 The present invention relates to network communication technologies, and in particular, to a web request scheduling method and system. Background technique

 In a complex network system, a web service can be provided by multiple servers, and each server may have different functional positioning, such as a cache server, a load balancing server, a web server, a database server, and the like. Through abstraction, the Web request will be forwarded by the forwarding unit on the server side, and finally processed by the processing unit. Cache data may be stored in the forwarding unit and/or processing unit to increase access speed.

 Based on the existing solution, when the cache is based on the Uniform Resource Locator (URL) cache corresponding data, when the same two sets of website systems are deployed in the distributed Web environment, and the access URL is different, this will result in Two identical systems will cache two copies of virtually identical data on the same forwarding unit or processing unit. From a system-wide perspective, for the same URL request, the request of the forwarding unit in the processing algorithm may also cause the same URL (or different URL but the actual content of the request file is consistent) to be forwarded to different forwarding units or On the processing unit,

The same cached data. Summary of the invention

 The embodiment of the present invention provides a web request scheduling method and system, which are used to solve the problem of resource waste caused by multiple identical requests of the same web request in the prior art, thereby improving cache utilization and reducing the web. The cost of the service environment.

 The embodiment of the invention provides a web request scheduling method, which includes:

Parsing the received web request to obtain file path information, and sending a cache request, carrying the text Piece path information;

 Obtaining a file identification code corresponding to the file path information, and obtaining and returning cache data corresponding to the file identification code, where the file identification code and the cache data are corresponding to each other.

 The embodiment of the invention provides a web request scheduling system, including:

 a request processing module, configured to parse the received web request to obtain file path information, and send a cache request, and carry the file path information;

 The local cache module is configured to obtain a file identifier corresponding to the file path information, obtain and return cache data corresponding to the file identifier, where the file identifier and the cache data are corresponding relationships.

 According to the above technical solution, the web request scheduling method and system according to the embodiment of the present invention can make the web request requesting the same content corresponding to the same cache data by using the unique cache data corresponding to each file identification code, thereby improving the utilization of the cache. Rate and reduce the cost of the Web services environment. DRAWINGS

 In order to more clearly illustrate the technical solutions in the embodiments of the present invention, a brief description of the drawings to be used in the description of the embodiments will be briefly made. It is obvious that the drawings in the following description are some embodiments of the present invention. It will be apparent to those skilled in the art that other drawings may be obtained from these drawings without the inventive labor.

 1 is a schematic flow chart of a method according to a first embodiment of the present invention;

 2 is a first structural diagram of a system according to an embodiment of the present invention;

 3 is a schematic structural diagram 1 of a shared cache unit according to an embodiment of the present invention;

 4 is a second structural diagram of a system according to an embodiment of the present invention;

 FIG. 5 is a schematic structural diagram 2 of a shared cache unit according to an embodiment of the present invention;

 6 is a schematic flow chart of a method according to a second embodiment of the present invention;

 7 is a schematic flow chart of a method according to a third embodiment of the present invention;

FIG. 8 is a schematic flowchart of a method according to a fourth embodiment of the present invention; FIG. FIG. 9 is a schematic diagram of an internal flow 1 of a shared cache unit according to an embodiment of the present invention; FIG. 10 is a schematic diagram of an internal flow 2 of a shared cache unit according to an embodiment of the present invention;

 FIG. 11 is a schematic diagram of an internal process 3 of a shared cache unit according to an embodiment of the present invention;

 FIG. 12 is a schematic diagram of an internal process 4 of a shared cache unit according to an embodiment of the present invention;

 13 is a schematic diagram of an internal process 5 of a shared cache unit according to an embodiment of the present invention;

 14 is a schematic diagram of an internal process 6 of a shared cache unit according to an embodiment of the present invention;

 FIG. 15 is a schematic diagram of correspondence between cache data, file path information, and file identification code according to an embodiment of the present invention. detailed description

 The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

 FIG. 1 is a schematic flowchart of a method according to a first embodiment of the present invention, including:

 Step 11: The web request scheduling system parses the received web request to obtain file path information, and sends a cache request to carry the file path information.

 Step 12: The web request scheduling system acquires a file identification code corresponding to the file path information, and obtains and returns cache data corresponding to the file identification code, where the file identification code and the cache data are in a corresponding relationship.

 In this embodiment, each file identifier code corresponding to the unique cache data can make the request for the same web content corresponding to the same cache data, which can improve the cache utilization rate and reduce the cost of the Web service environment.

2 is a system architecture diagram corresponding to an embodiment of the present invention, including a forwarding unit/processing unit 21, a file metadata storage unit 22, a shared cache unit 23, and a file system 24, and a forwarding unit/processing list. The element 21 includes a request processing module 211 and a local cache module 212. The interface between the request processing module 211 and the local cache module 212 is an H2 interface, the interface between the local cache module 212 and the file metadata storage unit 22 is an F1 interface, and the interface between the local cache module 212 and the shared cache unit 23 is On the T1 interface, the interface between the shared cache unit 23 and the file system 24 is an F2 interface, and the interface between the file metadata storage unit 22 and the shared cache unit 23 is an F1 interface.

 The request processing module 211 is mainly responsible for parsing the web request, for example, parsing the URL, requesting the header, and the like. After parsing, the H2 interface is used to request whether the web request corresponds to the cached data, and the cached data may be the corresponding file, the bytecode of the script file, or the memory data parsed by the script file. Among them, a script is a program written in a scripting language, such as a program written in a programming language such as PHP or Ruby.

 The local cache module 212 is an engine for caching data search. The local cache module 212 stores the correspondence between the file path information and the cache data, for example, the correspondence between the URL and the cache data, and the corresponding relationship and the URL may be used. The query obtains the corresponding cached data. In the embodiment of the present invention, the local cache module 212 also needs to save the correspondence between the file identification code and the cached data, and the file identification code can uniquely represent the file content, such as the md5 value of the file. Moreover, in the embodiment of the present invention, each file identification code only corresponds to one cache data.

 The request processing module 211 and the local cache module 212 can communicate with each other through the internal interface H2. For example, after receiving the web request, the request processing module 211 sends a cache request to the local cache module 212, carrying the file path information and the cache type, and the cache type. If the cached data of the request is a file cache or a script cache, if there is corresponding cache data in the local cache module 212, it can be returned to the request processing module 211 through the H2 interface; if there is no corresponding cache data in the local cache module 212, The cache data can be obtained from the shared cache unit 23 through the T1 interface and returned to the request processing module 211, which can be a remote communication interface.

The file metadata storage unit 22 refers to a storage unit that stores metadata of all the web application files. The metadata in the metadata storage unit needs to include information of the file, and the information of the file includes but is not limited to the file identification code and the file path information. The file identification code can uniquely identify a file, such as a file Md5 value. The file metadata storage unit 22 can open the metadata query interface to the outside through the F1 interface. Using the F1 interface, the file identification code can be queried by the file path information and the correspondence between the file path information and the file metadata stored in the file metadata storage unit 22.

 The shared cache unit 23 provides shared cache storage independently of the forwarding unit/processing unit 21, and opens the cache query through the T1 interface. When receiving the cache query request, if there is no cache data on the shared cache unit 23, the shared cache unit 23 can read data from the file system through the F2 interface; for the script cache, the script file can be obtained in the file system and then compiled. Get the script cache. The shared cache unit 23 provides a way to query the cached data by means of file path information or a file identification code, and can provide a file cache, a bytecode cache, or a cached memory structure cache.

 FIG. 3 is a schematic structural diagram 1 of a shared cache unit according to an embodiment of the present invention, including a file cache module 31, a script cache module 32, and a script engine module 33. In the shared cache unit, the T1 interface shown in Figure 2 can be decomposed into two interfaces: T1.1 and T1.2. The T1.1 interface corresponds to the file cache read request, and the T 1.2 interface corresponds to the script cache read request. Of course, the T 1.1 interface and the T 1.2 interface can also be combined, and the type of the cache is obtained by input parameters, and then processed by the internal scheduling module.

 The file cache module 31 is responsible for the file type cache operation. When the file cache is received, the file cache module searches for the file path information or the file identifier code. The file cache module queries whether the corresponding data has been cached, and if it exists, returns the existing data. The data is judged as follows: If the cache request through the file path cannot be queried through the file path information, the file identification code is queried through the F1 interface, and the data corresponding to the file identification code is queried by the file identification code, and the file path information is also obtained. And the corresponding relationship between the file identifier and the cached data is also cached. If the cached data cannot be obtained through the file path information and the file identifier, the file cache module 31 obtains data from the file system 24 through the F2 interface. The data acquired by system 24 can be cached by file caching module 31 for the next cached query.

The script cache read request is processed by the script cache module 32 and the script engine module 33. After receiving the script cache read request, the script cache module 32 will query whether there is a script cache. Similar to the file cache, the script cache read request may carry the file path information or the file identification code of the script file. When querying the cache, it can also be similar to the file cache. The cached data can be queried through the file path information or the file identification code, and the correspondence between the file path information, the file identification code, and the cached data is established. If there is any cached data to be returned, if not, the script cache module 32 will read and compile the script file through the N1 request script engine module 33. After receiving the N1 interface call, the script engine module 33 reads the script file through the F2 interface, compiles the script file after reading, and returns the compiled script data to the script cache module 32, and the compiled script data can be scripted. The cache module 32 caches for the next cached query.

 FIG. 4 is a second structural diagram of a system according to an embodiment of the present invention. FIG. 5 is a second schematic structural diagram of a shared cache unit according to an embodiment of the present invention. Different from FIG. 2-3, the file identification code calculation module 41, 51 is used instead of the file metadata storage unit, and the file identification code calculation module is used to calculate the file identification code file identification code of the cached data, for example, the MD5 of the cached data. value. Taking the file cache as an example, after the local cache module cannot query the corresponding file cache data according to the file path information, the file data is obtained from the file system, and after the file data is obtained, the file identifier code calculation module calculates the identification code of the file content. The file identification code and the cache data and the correspondence between the file path information and the cache data are established by the file cache module. If the subsequent request finds that the calculated file identifier already exists in the cache module, the relationship between the file path and the existing cached data is directly established, and it is not necessary to cache the same data. The functions of the remaining modules are the same as above, and will not be described again.

 The embodiment of the present invention will be described below by taking the structural diagram shown in Fig. 2 or Fig. 3 as an example. The structure shown in Fig. 4-5 can be referred to for execution. The specific processes of the embodiments of the present invention may be specifically referred to the following embodiments.

 FIG. 6 is a schematic flowchart of a method according to a second embodiment of the present invention. In this embodiment, file cache data corresponding to file path information locally exists as an example. Referring to Figure 6, this embodiment includes:

 Step 601: The request processing module receives the web request.

Step 602: The request processing module parses the web request, obtains corresponding file path information, and sends a cache request to the local cache module to carry the file path information. Step 603: The local cache module queries, by using the file path information, whether the file cache data exists locally.

 Step 604: If there is corresponding file cache data, the local cache module returns file cache data to the request processing module.

 The mapping between the file path information and the file cache data is saved in the local cache module. The corresponding relationship can be used to query whether the file path information carried in the cache request has corresponding file cache data. Further, if there is corresponding file cache data, the file cache data of the query can be returned.

 FIG. 7 is a schematic flowchart of a method according to a third embodiment of the present invention. In this embodiment, the file cache data corresponding to the file path information does not exist locally, but the file cache data corresponding to the file identifier code is used as an example. Referring to Figure 7, this embodiment includes:

 Step 701: The request processing module receives the web request.

 Step 702: The request processing module parses the web request, obtains corresponding file path information, and sends a cache request to the local cache module to carry the file path information.

 Step 703: The local cache module queries the local file cache data by using the file path information.

 Step 704: If there is no corresponding file cache data, the local cache module sends a query file identifier request to the file metadata storage unit, and carries the file path information.

 Step 705: The file metadata storage unit returns a file identification code to the local cache module.

 The file metadata storage unit stores a correspondence between the file path information and the file identification code, and the file identification code corresponding to the file path information carried in the query file identification code request can be queried by the corresponding relationship.

 Step 706: The local cache module queries, by using a file identifier, whether file cache data exists locally. Step 707: If there is corresponding file cache data, the local cache module establishes a correspondence between the file path information and the file cache data.

Step 708: The local cache module returns file cache data to the request processing module. The corresponding relationship between the file identifier and the file cache data is saved in the local cache module. The corresponding relationship can be used to query whether the file identifier has corresponding file cache data. Further, if there is corresponding file cache data, it can be returned to the file cache data. Moreover, in order to obtain the file cache data faster, the correspondence between the file path information and the file cache data may also be established by using the file identifier.

 FIG. 8 is a schematic flowchart of a method according to a fourth embodiment of the present invention. In this embodiment, file cache data corresponding to file path information does not exist locally, and file cache data corresponding to the file identifier code does not exist as an example. Referring to Figure 8, this embodiment includes:

 Step 801: The request processing module receives the web request.

 Step 802: The request processing module parses the web request, obtains corresponding file path information, and sends a cache request to the local cache module to carry the file path information.

 Step 803: The local cache module queries the local file cache data by using the file path information.

 Step 804: If the corresponding file cache data does not exist, the local cache module sends a query file identifier request to the file metadata storage unit, and carries the file path information.

 Step 805: The file metadata storage unit box local cache module returns a file identification code.

 The file metadata storage unit stores a correspondence between the file path information and the file identification code, and the file identification code corresponding to the file path information carried in the query file identification code request can be queried by the corresponding relationship.

 Step 806: The local cache module queries the local file cache code for the file cache data. Step 807: If there is no corresponding file cache data, the local cache module sends a cache query request to the shared cache unit, carrying the file path information or the file identifier code.

 Step 808: The shared cache unit executes an internal process to obtain file cache data.

 For details, refer to the embodiment shown in Figures 9-15.

Step 809: The shared cache unit returns the obtained file cache data to the local cache module. Step 810: The local cache module establishes a correspondence between the file path information and the file cache data. And the correspondence between the file identification code and the file cache data.

 Step 811: The local cache module returns file cache data to the request processing module.

 FIG. 9 is a schematic diagram of an internal process 1 of a shared cache unit according to an embodiment of the present invention. In this embodiment, file cache data corresponding to file path information is obtained as an example. Referring to Figure 9, this embodiment includes:

 Step 901: The local cache module sends a file cache read request to the file cache module, and carries the file path information.

 Step 902: The file cache module queries whether the file cache data corresponding to the file path information exists locally.

 Step 903: If yes, the file caching module returns the corresponding file cache data to the local cache module.

 The file cache module stores the correspondence between the file path information and the file cache data. The corresponding relationship can be used to query whether the file path information carried in the file cache read request has corresponding file cache data. Further, if there is corresponding file cache data, the file cache data can be returned.

 FIG. 10 is a schematic diagram of an internal process 2 of a shared cache unit according to an embodiment of the present invention. In this embodiment, file cache data corresponding to file path information is obtained as an example. Referring to FIG. 10, this embodiment includes:

 Step 1001: The local cache module sends a file cache read request to the file cache module, and carries the file path information.

 Step 1002: The file cache module queries whether the file cache data corresponding to the file path information exists locally.

 Step 1003: If the corresponding file cache data does not exist, the file cache module sends a query file identifier request to the file metadata storage unit, and carries the file path information.

 Step 1004: The file metadata storage unit returns a file identification code to the file cache module.

Wherein, the file metadata storage unit stores the corresponding relationship between the file path information and the file identification code The file identification code corresponding to the file path information carried in the query file identifier code request may be queried by the corresponding relationship.

 Step 1005: The file caching module queries whether there is file cache data corresponding to the file identification code locally.

 Step 1006: If there is corresponding file cache data, the file cache module establishes a correspondence between the file path information and the file cache data.

 Step 1007: The file cache module returns file cache data to the local cache module.

 The file cache module stores a correspondence between the file identifier code and the file cache data. Through the corresponding relationship, the file identifier code can be queried for the corresponding file cache data. Further, if there is corresponding file cache data, it can be returned to the file cache data. Moreover, in order to obtain the file cache data faster, the correspondence between the file path information and the file cache data may be established by using the file identification code.

 FIG. 11 is a schematic diagram of the internal process 3 of the shared cache unit in the embodiment of the present invention. In this embodiment, the file cache data corresponding to the file path information and the file cache data corresponding to the file identifier code are not present. Referring to FIG. 11, this embodiment includes:

 Step 1101: The local cache module sends a file cache read request to the file cache module, and carries the file path information.

 Step 1102: The file cache module queries whether the file cache data corresponding to the file path information exists locally.

 Step 1103: If the corresponding file cache data does not exist, the file cache module sends a query file identifier code request to the file metadata storage unit, and carries the file path information.

 Step 1104: The file metadata storage unit returns a file identification code to the file cache module.

 The file metadata storage unit stores a correspondence between the file path information and the file identification code, and the file identification code corresponding to the file path information carried in the query file identification code request can be queried by the corresponding relationship.

Step 1105: The file cache module queries, by using the file identifier code, whether file cache data exists locally. Step 1106: If the corresponding file cache data does not exist, the file cache module sends a read file data request to the file system, and carries the file identifier code.

 Step 1107: The file system returns file data corresponding to the file identification code to the file cache module.

 Or the file cache module carries the file path information in the read file data request sent by the file system, and the file system obtains the corresponding file data according to the file path information.

 Step 1108: The file cache module caches the file data returned by the file system, establishes a correspondence between the file path information and the file cache data, and a correspondence between the file identification code and the file cache data.

 Step 1109: The file caching module returns file cache data to the local cache module.

 The schematic diagram of the internal flow of the shared cache in the embodiment shown in FIG. 9-11 is not limited to the foregoing embodiment, and the file cache information may be carried in the file cache read request sent by the local cache module to the file cache module. The file identifier code is also carried. In this case, the corresponding step 1003-1004 does not need to be executed, and the file cache code is directly queried according to the carried file identifier code. In addition, the file identifier reading request carries the file identifier code, and does not carry the file path information. In this case, steps 902, 903, steps 1002-1004, and steps 1102-1104 do not need to be performed, and are directly carried according to the query. Whether the file identifier has file cache data.

 The above is the file cache as an example. The corresponding cache data is file cache data. The following takes the script cache as an example, and the corresponding cache data is the script cache data.

 FIG. 12 is a schematic diagram of the internal flow of the shared cache unit in the embodiment of the present invention. This example takes the script cache data corresponding to the file path information and the file cache information as an example. Referring to FIG. 12, this embodiment includes:

 Step 1201: The local cache module sends a script cache read request to the script cache module, and carries file path information.

Step 1202: The script cache module queries whether the script cache data corresponding to the file path information exists locally. Step 1203: If yes, the script cache module returns the corresponding script cache data to the local cache module.

 The corresponding relationship between the file path information and the script cache data is saved in the script cache module. The corresponding relationship can be used to query whether the file path information carried in the file cache read request has corresponding script cache data. Further, if there is a corresponding script cache data, the script cache data can be returned.

 FIG. 13 is a schematic diagram of an internal process 5 of a shared cache unit according to an embodiment of the present invention. This example takes the script cache data corresponding to the file path information and the file path information as an example. Referring to Figure 13, this embodiment includes:

 Step 1301: The local cache module sends a script cache read request to the script cache module to carry file path information.

 Step 1302: The script cache module queries whether the script cache data corresponding to the file path information exists locally.

 Step 1303: If the corresponding script cache data does not exist, the script cache module sends a query file identifier request to the file metadata storage unit, and carries the file path information.

 Step 1304: The file metadata storage unit returns a file identification code to the script cache module.

 The file metadata storage unit stores a correspondence between the file path information and the file identification code, and the file identification code corresponding to the file path information carried in the query file identification code request can be queried by the corresponding relationship.

 Step 1305: The script cache module queries, by using the file identifier, whether the script cache data exists locally. Step 1306: If there is a corresponding script cache data, the script cache module establishes a correspondence between the file path information and the script cache data.

 Step 1307: The script cache module returns the script cache data to the local cache module.

The corresponding relationship between the file identifier and the script cache data is saved in the script cache module. The corresponding relationship can be used to query whether the file identifier has the corresponding script cache data. Further, if there is a corresponding script cache data, it can be returned to the script cache data. And, for the next time The script cache data can be obtained relatively quickly, and the correspondence between the file path information and the script cache data can also be established through the file identification code.

 FIG. 14 is a schematic diagram of the internal flow of the shared cache unit in the embodiment of the present invention. In this embodiment, the script cache data corresponding to the file path information and the script cache data corresponding to the file identifier code are not present. Referring to Figure 14, this embodiment includes:

 Step 1401: The local cache module sends a script cache read request to the script cache module, and carries the file path information.

 Step 1402: The script cache module queries whether the script cache data corresponding to the file path information exists locally.

 Step 1403: If the corresponding script cache data does not exist, the script cache module sends a query file identifier code request to the file metadata storage unit, and carries the file path information.

 Step 1404: The file metadata storage unit returns a file identification code to the script cache module.

 The file metadata storage unit stores a correspondence between the file path information and the file identification code, and the file identification code corresponding to the file path information carried in the query file identification code request can be queried by the corresponding relationship.

 Step 1405: The script cache module queries, by using the file identifier, whether the script cache data exists locally. Step 1406: If the corresponding script cache data does not exist, the script cache module sends a compile request to the script engine module to carry the file path information.

 Step 1407: The script engine module sends a read script data request to the file system, and carries a file identifier code corresponding to the file path information.

 Step 1408: The file system returns script data to the script engine module.

 Step 1409: The script engine module compiles the script data.

 Compilation refers to the conversion of script data into bytecode data or memory structure data that can be represented by a script after parsing the script file.

 Step 1410: The script engine module returns the compiled script data to the script cache module.

Step 1411: The script cache module caches the compiled script data, and establishes file path information and The correspondence between the script cache data and the correspondence between the file identifier and the script cache data.

 Step 1412: The script cache module returns the script cache data to the local cache module.

 The schematic diagram of the internal flow of the shared cache in the embodiment shown in FIG. 12-14 is not limited to the foregoing embodiment, and the file cache information may be carried in the script cache read request sent by the local cache module to the file cache module. The file identifier code is also carried. In this case, the corresponding step 1303-1304 does not need to be executed, and the file cache code is directly queried according to the carried file identifier code. In addition, the file cache read request carries the file identifier code, and does not carry the file path information. In this case, steps 1202, 1203, steps 1302-1304, and steps 1402-1404 need not be performed, directly according to the query. Whether the file identifier code is carried in the script cache data.

 For the corresponding relationship between the cache data, the file path, and the file identifier, refer to FIG. 15. FIG. 15 is a schematic diagram of correspondence between cache data, file path information, and file identification code according to an embodiment of the present invention. Suppose the cache data is datal, data2...dataN, the file path information is urll, url2...urlN, and the file identifier is keyl, key2...keyN. The corresponding relationship can be obtained by using a hash algorithm or other algorithms. The file identification code and the cache data are-corresponding relationships, for example, key1 corresponds to data3, and key3 corresponds to data3; cache data and file path information may be one-to-many or many-to-one or-correspondence, for example, urll and url2 Corresponds to data3.

 It can be understood that related features in the above methods and devices can be referred to each other. In addition, "first", "second", and the like in the above embodiments are used to distinguish the embodiments, and do not represent the disadvantages of the embodiments.

 A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to program instructions. The foregoing program may be stored in a computer readable storage medium, and when executed, the program includes The foregoing steps of the method embodiment; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting thereof; although the present invention has been described in detail with reference to the foregoing embodiments, It should be understood that: the technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the essence of the corresponding technical solutions. The spirit and scope of the technical programme.

Claims

Rights request
 A web request scheduling method, comprising:
 Parsing the received web request to obtain file path information, and sending a cache request, carrying the file path information;
 Obtaining a file identification code corresponding to the file path information, and obtaining and returning cache data corresponding to the file identification code, where the file identification code and the cache data are corresponding to each other.
 The method according to claim 1, wherein the obtaining the file identification code corresponding to the file path information comprises:
 The local cache module sends a query file identifier code request to the file metadata storage unit, where the query file identifier code request carries the file path information; and receives a file corresponding to the file path information returned by the file metadata storage unit An identifier, where the file metadata storage unit stores a correspondence between file path information and a file identification code;
 Or,
 The local cache module obtains the file identification code from the file identification code calculation module.
 The method according to claim 1 or 2, wherein the acquiring and returning the cached data corresponding to the file identification code comprises:
 If the cache data corresponding to the file identification code is stored in the local cache module, the locally saved cache data is returned.
 The method according to claim 1 or 2, wherein the acquiring and returning the cached data corresponding to the file identification code comprises:
 And if the cached data corresponding to the file identifier is not stored in the local cache module, the cache query request is sent to the shared cache unit, where the cache query request carries the file path information and/or the file identifier code;
 Receiving, by the shared cache unit, cache data corresponding to the file path information and/or the file identification code;
Returning the cached data, and establishing a correspondence between the file identification code and the cached data and the file Correspondence between path information and cached data.
 5. The method according to claim 4, further comprising:
 The shared cache unit receives the cache query request;
 Obtaining a file identifier corresponding to the file path information if the cache data corresponding to the file path information is not saved in the shared cache unit;
 And if the cached data corresponding to the file identifier is stored in the shared cache unit, returning corresponding cache data to the local cache module, and establishing a correspondence between the file path information and the cache data.
 6. The method according to claim 4, further comprising:
 The shared cache unit receives the cache query request;
 Obtaining a file identifier corresponding to the file path information if the cache data corresponding to the file path information is not saved in the shared cache unit;
 If the cache data corresponding to the file identification code is not saved in the shared cache unit, the data is obtained and cached from the file system;
 The shared cache unit returns corresponding cache data to the local cache module, and establishes a correspondence between the file path information and the cache data and a correspondence between the file identification code and the cache data.
 The method according to claim 5 or 6, wherein the obtaining the file identification code corresponding to the file path information comprises:
 And if the cache query request carries the file identifier, the file identifier is obtained from the cache query request;
 Or,
 If the file identification code is not carried in the cache query request, the received file metadata storage unit obtains the file identification code according to the file path information and the correspondence between the file path information and the file identification code;
 Or,
If the file identification code is not carried in the cache query request, the file identification code is calculated. The file identification code is obtained in the block.
 8. The method according to claim 6, wherein the obtaining and buffering data from the file system comprises:
 If the file cache data is to be obtained, the shared cache module sends a read file data request to the file system, where the read file data request carries the file identification code; and receives the file data returned by the file system according to the file identification code. ;
 Or,
 If the script cache data is to be obtained, the shared cache module sends a read script data request to the file system, where the read script data request carries the file identifier code; and receives the script data returned by the file system according to the file identifier code. Compile the script data and cache the compiled script data.
 9. A web request scheduling system, comprising:
 a request processing module, configured to parse the received web request to obtain file path information, and send a cache request, and carry the file path information;
 The local cache module is configured to obtain a file identifier corresponding to the file path information, obtain and return cache data corresponding to the file identifier, where the file identifier and the cache data are corresponding relationships.
 10. The system according to claim 9, further comprising:
 And a file metadata storage unit, configured to save a corresponding relationship between the file path information and the file identification code; the local cache module is configured to obtain a corresponding file identification code from the file metadata storage unit according to the file path information.
 11. The system according to claim 9, further comprising:
 a file identification code calculation module, configured to calculate a file identification code;
 The local cache module is specifically configured to obtain the file identification code from the file identifier calculation module.
12. A system according to any of claims 9-11, characterized in that The local cache module is specifically configured to: if the cache data corresponding to the file identification code is saved in the local cache module, return the locally saved cache data.
 The system according to any one of claims 9-11, further comprising: a shared cache unit, configured to return cached data to the local cache module according to the file path information and/or the file identification code ;
 The local cache module is specifically configured to: if the cached data corresponding to the file identifier is not stored in the local cache module, send a cache query request to the shared cache unit, where the cache query request carries the file path information And/or a file identification code; receiving cache data corresponding to the file path information and/or the file identification code returned by the shared cache unit; returning the cached data, and establishing a correspondence between the file identification code and the cached data, and Correspondence between file path information and cached data.
 14. The system of claim 13 wherein:
 The shared cache unit includes: a file cache module, configured to send a read file data request to the file system if the file cache data corresponding to the file path information is not saved, where the read file data request carries the file Identification code;
 The system further includes: a file system, configured to obtain and return corresponding file cache data to the file cache module according to the file identification code.
 The system according to claim 13, wherein the shared cache unit comprises: a script cache module, configured to: if the script cache data corresponding to the file path information is saved, to the local cache module Returns the corresponding script cache data.
 16. The system of claim 13 wherein:
 The shared cache unit includes: a script cache module and a script engine module, where the script cache module is configured to send a compile request to the script engine module if the script cache data corresponding to the file path information is not saved; The engine module sends a read script to the file system to request that the read script request carries the file identifier code;
The system further includes: a file system, configured to acquire and feed the foot according to the file identification code The engine module returns corresponding script data;
 The script engine module is further configured to compile the script data; the script cache module is further configured to receive and cache the compiled script data.
PCT/CN2010/079740 2010-12-14 2010-12-14 Web request scheduling method and system WO2011110042A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2010/079740 WO2011110042A1 (en) 2010-12-14 2010-12-14 Web request scheduling method and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
PCT/CN2010/079740 WO2011110042A1 (en) 2010-12-14 2010-12-14 Web request scheduling method and system
CN2010800046912A CN102216930A (en) 2010-12-14 2010-12-14 Web request scheduling method and system

Publications (1)

Publication Number Publication Date
WO2011110042A1 true WO2011110042A1 (en) 2011-09-15

Family

ID=44562864

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2010/079740 WO2011110042A1 (en) 2010-12-14 2010-12-14 Web request scheduling method and system

Country Status (2)

Country Link
CN (1) CN102216930A (en)
WO (1) WO2011110042A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102929958A (en) * 2012-10-10 2013-02-13 无锡江南计算技术研究所 Metadata processing method, agenting and forwarding equipment, server and computing system
CN102970358B (en) * 2012-11-08 2016-06-15 百度在线网络技术(北京)有限公司 The control method of the local cache of mobile client and device in net dish
CN105812839B (en) * 2014-12-31 2018-12-14 华为技术有限公司 Video stream data acquisition, page data transmission method, system and network server

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1601526A (en) * 2003-09-23 2005-03-30 国际商业机器公司 Method and appts for searching within a computer network
CN1852170A (en) * 2006-03-21 2006-10-25 华为技术有限公司 Method and system for realizing apparatus management based on standard platform
CN101408899A (en) * 2008-11-21 2009-04-15 北京中企开源信息技术有限公司 Method and apparatus for switching website multiple data sources
US20090313241A1 (en) * 2008-06-16 2009-12-17 Cisco Technology, Inc. Seeding search engine crawlers using intercepted network traffic

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100591078C (en) * 2007-09-28 2010-02-17 北京金山软件有限公司 A website buffering method and device
CN101719936A (en) * 2009-12-09 2010-06-02 成都市华为赛门铁克科技有限公司 Method, device and cache system for providing file downloading service

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1601526A (en) * 2003-09-23 2005-03-30 国际商业机器公司 Method and appts for searching within a computer network
CN1852170A (en) * 2006-03-21 2006-10-25 华为技术有限公司 Method and system for realizing apparatus management based on standard platform
US20090313241A1 (en) * 2008-06-16 2009-12-17 Cisco Technology, Inc. Seeding search engine crawlers using intercepted network traffic
CN101408899A (en) * 2008-11-21 2009-04-15 北京中企开源信息技术有限公司 Method and apparatus for switching website multiple data sources

Also Published As

Publication number Publication date
CN102216930A (en) 2011-10-12

Similar Documents

Publication Publication Date Title
US20170004185A1 (en) Method and system for implementing collection-wise processing in a log analytics system
JP6181290B2 (en) Transaction ordering
US10664375B2 (en) Method and apparatus for stress management in a searchable data service
US9251163B2 (en) File sharing system and file sharing method
US8892415B2 (en) Model-based systems and methods for monitoring resources
CN102938731B (en) Exchange and integration device and method based on proxy cache adaptation model
US10324711B2 (en) System and method for the data management in the interaction between machines
US6591266B1 (en) System and method for intelligent caching and refresh of dynamically generated and static web content
CN101147379B (en) Systems and methods for performing caching of dynamically generated objects in a network
US9223867B2 (en) Methods and systems for data processing
CN102771080B (en) Use the system and method that the efficient media of buffer memory transmits
CN100428228C (en) System and method for caching data for a mobile application
TWI262406B (en) System, method and program storage device for dynamic caching of data based on queries performed by a local application
CN1755676B (en) System and method for batched indexing of network documents
US8296303B2 (en) Intelligent event query publish and subscribe system
Krause et al. Challenges in modelling and using quality of context (qoc)
EP1974295B1 (en) Method and apparatus for a distributed file storage and indexing service
US7849227B2 (en) Stream data processing method and computer systems
KR101079570B1 (en) Discovery Web Service
CN100596353C (en) Method and system for providing log service
CN105138592A (en) Distributed framework-based log data storing and retrieving method
US8380192B2 (en) Method and apparatus for managing profile information of mobile terminal in mobile communications system
JP2013246828A (en) Database for storing encoded triple, control part, method and system
US9405851B1 (en) Flexible caching
CN102163210B (en) The quick renewal of index metadata

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201080004691.2

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10847300

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10847300

Country of ref document: EP

Kind code of ref document: A1