CN103747030A - Nginx server intelligent cache method based on improved particle swarm optimization - Google Patents

Abstract

The invention relates to a cache method and especially relates to a Nginx server intelligent cache method based on improved particle swarm optimization. According to the Nginx server intelligent cache method based on the improved particle swarm optimization, an intelligent cache is parallelly added on the basis of not influencing original caching technology and efficiency of a Nginx server, thereby improving the efficiency of the Nginx server on the basis of the prior art of the Nginx server, and reducing server overhead to a greater degree.

Description

An intelligent caching method for Nginx server based on improved particle swarm optimization algorithm

technical field

The invention relates to a caching method, in particular to an intelligent caching method for an Nginx server based on an improved particle swarm algorithm.

Background technique

Nginx's existing caching technology is to save the accessed static resources after the client accesses the server, and the relevant content can be directly referenced next time, which improves efficiency to a limited extent and reduces the load on the server.

There are many defects in the traditional caching technology, for example: it does not support dynamic links with parameters, such as read.php?id=1, the Nginx server only saves the file name, so this link can only be saved as read. read.php?id=3 will return incorrect results. Another defect: there is no mechanism for cache expiration and cleaning inside the Nginx server. These cached files will be permanently stored on the hard disk. If there are a lot of things to be cached, it will take up all the hard disk space. For this reason, a shell script can be used to clean up regularly, and dynamic programs such as php can be written to do real-time updates. The traditional cache can only store 200 status codes, so status codes such as 301/302/404 returned by the backend will not be cached. If a pseudo-static link with a large number of visits happens to be deleted, it will continue to penetrate and cause subsequent The end bears a lot of pressure. The Nginx server will not automatically select the memory or hard disk as the storage medium, everything is determined by the configuration.

Contents of the invention

In order to solve the problems of the prior art, the invention provides a kind of Nginx server intelligent caching method based on the improved particle swarm optimization algorithm, and this caching technology is an intelligent caching method added in parallel on the basis of not affecting the original caching technology and efficiency of the Nginx server. Cache, which improves the efficiency of the Nginx server on the basis of the existing technology of the Nginx server, and reduces the overhead of the server to a greater extent.

The technical scheme adopted in the present invention is as follows:

A kind of Nginx server intelligent caching method based on the improved particle swarm optimization algorithm is to increase an intelligent cache in parallel on the basis of not affecting the original Nginx server cache. Access server resources, and optimize search results based on experience and artificially set parameters to save the most accessible resources in the smart cache. the

After receiving the access request from the client, the Nginx server will first retrieve it from the cache of the original Nginx server after it is verified to be legal. When the resource is found, the resource is returned, and the connection to the server is interrupted. If it is not found in the smart cache, it will not affect the normal access to the server.

Intelligent caching is based on the improved particle swarm algorithm of parameters to find the most accessible server resources.

The Nginx server hangs a callback function during the phase of responding to the request.

After the Nginx server is started, it applies for the cache and connects to the server.

The smart cache applies for space following the startup of the Ngnix server, sets an automatic cleaning threshold for the smart cache, and cleans up low-priority resources in a timely manner.

Smart caching not only caches static resources, but also caches some commonly used dynamic resources.

The parallel technology is used between the original Nginx server cache and the smart cache. When the smart cache cannot find resources, it will not affect the efficiency of the original Nginx server cache to obtain resources from the background service. In the case of large concurrency, the smart cache will reduce The speed gives up more resources to the main process of the Nginx server.

The principle that the present invention realizes is as follows:

1. After ngx_http_parse_request_line() returns NGX_OK, it means that the http request is legal. By judging the return value, when nginx_http_parse_header_line() returns NGX_HTTP_HEADER_DONE, it means that all request headers have been processed. The callback function of this patent is the handler after this stage Module call, this function starts to run here, analyzes the content of the request header in parallel and retrieves it in the cache, where the static resources will be retrieved in the static area of the cache, and the dynamic resources will be retrieved in the dynamic area. The contents of the cache include Table 1 As shown, when the returned content is faster than the nginx handler module itself, it will be directly returned to the client as the corresponding data, and its own retrieval will be terminated.

the

Table 1 Introduction to Algorithm Parameters

2. Initialize the module, apply for cache space and hard disk storage location, start the improved particle swarm optimization algorithm, initialize the algorithm and establish a socket connection with the background server, the algorithm goes to each server to obtain resources according to the set parameters, the weight of the algorithm and the number of learning times Can be set to reinitialize if it does not converge after a certain number of learnings.

3. The intelligent acquisition module, after the algorithm is initialized, establishes a connection with the back-end server, and then searches for resources on each server. The function of the intelligent learning algorithm here is to optimize the search. Classify the resources on the website, for example: the dynamic display of each page of a website has the highest priority; the screening content is screened according to the experience value, and the priority is second, and the obtained resources are classified and stored.

4. High-speed retrieval module: The storage of cache resources is stored in the form of classification and indexing. When retrieving resources, they are queried step by step according to classification. The algorithm used at the bottom layer is binary search method, which is easy to use and fast. No complex data structures are required. The search results are assembled (consistent with the request message), which is generally faster than Nginx's cache in dynamic scenarios, and can return results quickly when static resources cannot be found by Nginx.

5. When Nginx is started, the intelligent acquisition module of this function starts to work. According to the common access experience, the initialization conditions are set for the intelligent learning algorithm, and these conditions will be updated regularly according to user access. After actively connecting to the background server, since the number of servers is limited, intelligent resource acquisition is performed on the servers that can be connected at the same time, that is, the retrieval function is run on each server. According to the set number of iterations, the resource with the fastest convergence will be acquired into the cache.

6. Support caching of dynamic resources. According to the content of the http request header and the set access url, the resources of the server are obtained and stored in the cache. The difference from the traditional cache is that the cache stores all the keys of the http request. Content and detailed response data, so that in many cases Nginx connects to the server and waits for the server to execute the script language or even connect to the database and then respond, which improves the service efficiency of the Nginx server to a certain extent.

The present invention has the following characteristics:

1. Modify the Nginx source code and add a callback function;

2. Users can modify the configuration parameters according to the configuration file;

3. It can increase the access speed to a greater extent and reduce the load on the server;

4. The intelligent cache has a real-time cleaning function, and the resources with low priority will be cleaned up in time according to the set threshold.

The beneficial effects brought by the technical scheme provided by the invention are:

The present invention is to add a parallel intelligent cache on the basis of the original cache of the existing Nginx server. The idea of this intelligent cache is somewhat similar to the function of a web spider, but it is not fixed to fetch resources but decides which content to fetch by learning and optimizing. , so that resources with high priority can be obtained among limited servers. The difference from traditional caches is that this intelligent cache can save dynamic resources, and resources accessed by clients with parameters can also be obtained in the cache for the client access. On the basis of the existing technology of Nginx server caching, the work efficiency of Nginx server is improved, and the overhead of the server is reduced to a greater extent.

Description of drawings

Fig. 1 is a kind of principle architecture diagram of the Nginx server intelligent caching method based on the improved particle swarm algorithm of the present invention;

Fig. 2 is an algorithm schematic diagram of a kind of Nginx server intelligent caching method based on the improved particle swarm optimization algorithm of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings.

Embodiment one

Attached Figure 1 is a schematic diagram of the working principle of the smart cache. This function is a parallel function of the handler module added to the Nginx request processing. According to the detected running status of the Nginx server, it is judged whether to apply for the cache and connect to the server. After the Nginx server is started Take the initiative to connect to the background server, detect the status of the server and available resources, search for resources according to the intelligent learning algorithm, and return to the current cache, and determine whether to retrieve the resources in the cache by judging whether there is a request from the client.

The function flow is as follows:

#Define Ngx_CSIZE 4096

Ngx_Clever_Cache(ngx_string *info)

{

if (NGX_ON==status)

{

Pcache=pcalloc(Ngx_CSIZE);

Initial(sys);//Initialize system parameters

}

If (NULL!=*info)

{

Handler(*info)

Return Search(Pcache) }

}

As shown in Figure 2, this embodiment uses an improved particle swarm optimization algorithm (PSO) to retrieve server resources with the highest priority. The algorithm prototype: a group of birds are randomly searching for food. There is only one piece of food in this area. All birds don't know where the food is. But they know how far away the food is from their current location. So what is the optimal strategy for finding food. The simplest and most effective strategy is to search the area around the bird that is currently closest to food.

PSO is initialized as a swarm of random particles, and the solution to each optimization problem is a bird in the search space. We call them "particles". All examples have a fitness value determined by the function being optimized, and each particle also has a velocity that determines the direction and distance they fly. Then the particles follow the current optimal particle to search in the solution space. In this patent, PSO is improved, and experience values are added on the basis of the original parameters, so that each particle has a memory function, which will reduce the convergence rate. number of iterations. The experience value is generated during the operation of the optimal particle, which can be shared with the nearest particle, and so on.

The improved PSO formula is as follows:

Experience=1

v[] = v[] + [c1 * rand() * (pbest[] - present[]) + c2 * rand() * (gbest[] - present[]) ]*Experience present[] = present[] + v[]

Experience=persent[i-1]/persent[i]

The cache structure of this embodiment is shown in Table 2:

resource class priority storage location index resource content static resources the the the dynamic resources the the the

Table 2

Static resources: css, js, htm, html, jpg, gif, png, flv, swf, static return page.

Dynamic resources: data extracted from the database by calling scripts, or calculated data, such as index.php/?Id=3, to obtain a piece of data with an id of 3, instead of saving only the file name index.php as in the past, and It is to save all the information for matching when visiting again. This piece of data is stored in the cache. The premise is that the data with id 3 has not been modified after being stored in the cache. The retrieval program periodically verifies whether the data has been modified. If it is modified, the cache content will be updated. When requesting this data, it is not necessary to fetch it from the server. Instead, it finds this dynamic data based on the index, which greatly saves time.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (8)

1. A kind of Nginx server intelligent caching method based on the improved particle swarm optimization algorithm is to increase an intelligent cache in parallel on the basis of not affecting the original Nginx server cache, and the intelligent cache is obtained according to the particle swarm algorithm, and the Nginx server acts as an agent Access server resources under certain circumstances, and optimize search results based on experience and artificially set parameters to save the most accessible resources in the smart cache. 2. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 1 is characterized in that, Nginx server end receives the access request that client transmits, and after verification is legal, first arrives at former Nginx Search in the server cache, if you can't find it, then search it in the smart cache, and get it from the corresponding server at the same time, if you can find the resource, return the resource, interrupt the connection to the server, if you find it in the smart cache It will not affect the normal access to the server. 3. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 1, is characterized in that, described intelligent caching is to find the server resource that is most easily accessed according to the improved particle swarm optimization algorithm of parameter . 4. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 2, is characterized in that, described Nginx server end hangs callback function in response request stage. 5. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 1 and 2, is characterized in that, described Nginx server end promptly applies for cache after starting, connects server. 6. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 1, is characterized in that, described intelligent cache follows the Ngnix server end and starts and just applied for space, automatically clears up intelligent cache setting Threshold, clean up low-priority resources in a timely manner. 7. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 1, is characterized in that, described intelligent caching has not only cached static resources, also has cached part commonly used dynamic resources. 8. a kind of Nginx server intelligent caching method based on improved particle swarm optimization algorithm according to claim 1, is characterized in that, has adopted parallel technology between described former Nginx server cache and intelligent cache, can't find in intelligent cache In the case of resources, it will not affect the efficiency of the original Nginx server cache to the background service to obtain resources. In the case of large concurrency, the smart cache will slow down the speed and give more resources to the main process of the Nginx server.

Images