CN117880363A - Network data caching processing method and device, computer equipment and storage medium - Google Patents
Network data caching processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN117880363A CN117880363A CN202410139315.8A CN202410139315A CN117880363A CN 117880363 A CN117880363 A CN 117880363A CN 202410139315 A CN202410139315 A CN 202410139315A CN 117880363 A CN117880363 A CN 117880363A
- Authority
- CN
- China
- Prior art keywords
- data
- request
- webpage
- interface identifier
- identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 54
- 230000004044 response Effects 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000012986 modification Methods 0.000 abstract description 3
- 230000004048 modification Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Abstract
The invention provides a network data caching processing method, a device, computer equipment and a storage medium, wherein the method comprises the steps of acquiring a data request, analyzing the data request to acquire an interface identifier and a request parameter; detecting whether the interface identifier hits a preset interface identifier set or not; when the interface identifier hits a preset interface identifier set, calculating the interface identifier and the request parameter to obtain a corresponding data identifier; and acquiring the first webpage data corresponding to the data request from the database according to the data identifier. When the interface identifier hits the preset interface identifier set, the data identifier corresponding to the data request is calculated according to the interface identifier and the request parameter, and then the first webpage data corresponding to the data request is acquired from the database according to the data identifier, so that the webpage caching technology can be realized no matter the data request is requested in any request mode, the use limitation of the webpage caching is improved, and therefore the application has the characteristics of strong universality, and the terminal and the server can support caching without modification.
Description
Technical Field
The present invention relates to the field of internet information management technologies, and in particular, to a network data caching method, a device, a computer device, and a storage medium.
Background
With the development of internet technology, the user volume or access volume of websites is gradually increased, which brings great pressure to corresponding servers. When a large number of users access at the same time, the server needs to process a large number of requests, which results in a slow response speed and even a possible crash. Web page caching technology has become a key means to improve web site performance and user experience. The webpage cache is mainly used for storing webpage content to reduce requests to the original server, so that the webpage loading speed is accelerated.
However, in the prior art, the method for caching the web page generally loads the web page for the first time, and obtains the resource file, such as HTML, CSS, javaScript, of the web page from the server. And downloads and stores the resource file in a local cache. Further, when the user accesses the same webpage again, the browser can detect whether the local cache stores the resource file of the webpage or not; and when the existence of the resource file of the webpage in the local cache is detected, directly using the resource file of the local cache to load to form the webpage. But the access mode must implement the web page caching mode by the GET request mode, that is, based on the UL consistency in the GET request; however, in the POST request mode, due to the content diversity of the POST request, the web page cache of the PSOT request mode cannot be identified, that is, the web page cache mode in the prior art only supports the GET request mode, which results in less limitation of use of the web page cache.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a network data caching method, apparatus, computer device, and storage medium.
A network data caching processing method comprises the following steps:
acquiring a data request, and analyzing the data request to acquire an interface identifier and a request parameter;
detecting whether the interface identifier hits a preset interface identifier set or not;
when the interface identifier hits the preset interface identifier set, calculating the interface identifier and the request parameter to obtain a corresponding data identifier;
and acquiring first webpage data corresponding to the data request from a database according to the data identifier.
In one embodiment, after the obtaining, according to the data identifier, the first web page data corresponding to the request for data in the database, the method further includes:
calculating a first hash value corresponding to the first webpage data, and sending the first hash value and the data request to a server so that the server obtains second webpage data according to the data request, performs hash processing on the second webpage data to obtain a second hash value, and generates feedback data according to the first hash value, the second hash value and the second webpage data;
Receiving the feedback data sent by the server;
and updating the first webpage data according to the feedback data.
In one embodiment, after detecting whether the interface identifier hits in a preset set of interface identifiers, the method further comprises:
when the interface identifier is not hit in the preset interface identifier set, the data request is sent to a server;
and receiving third webpage data sent by the server in response to the data request.
In one embodiment, before the first web page data corresponding to the data request is obtained in the database according to the data identifier, the method further includes:
detecting whether the webpage data corresponding to the data request is cached in a database according to the data identifier;
when the data identification determines that the webpage data corresponding to the data request is not cached in the database, the data is sent to a server;
and receiving the webpage data sent by the server corresponding to the data request, and storing the webpage data in a database.
In one embodiment, the method further comprises:
acquiring the first webpage data and the cache time corresponding to the first webpage data from the database;
Detecting whether the cache time reaches a preset threshold value;
and deleting the first webpage data in a database when the caching time reaches the preset threshold value.
A network data caching processing method comprises the following steps:
receiving a data request and a first hash value which are simultaneously sent by a terminal, and responding to the data request to acquire second webpage data corresponding to the data request;
carrying out hash processing on the second webpage data to obtain a second hash value;
generating feedback data according to the first hash value, the second hash value and the second webpage data, and sending the feedback data to the terminal.
In one embodiment, the method further comprises:
receiving a data request sent by a terminal, and responding to the data request to obtain third webpage data;
and sending the third webpage data to the terminal.
A network data caching apparatus, comprising:
the first acquisition module is used for acquiring a data request and analyzing the data request to acquire an interface identifier and a request parameter;
the detection module is used for detecting whether the interface identifier hits a preset interface identifier set or not;
the calculation module is used for calculating the interface identifier and the request parameter to obtain a corresponding data identifier when the interface identifier hits the preset interface identifier set;
And the second acquisition module is used for acquiring the first webpage data corresponding to the request data request from the database according to the data identifier.
A network data caching apparatus, comprising:
the first receiving module is used for receiving a data request and a first hash value which are sent by a terminal at the same time, and responding to the data request to acquire second webpage data corresponding to the data request;
the second hash module is used for carrying out hash processing on the second webpage data to obtain a second hash value;
and the first sending module is used for generating feedback data according to the first hash value, the second hash value and the second webpage data and sending the feedback data to the terminal.
A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the network data caching method described in any one of the embodiments above when executing the computer program.
A computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the network data caching method described in any one of the embodiments above.
According to the network data caching processing method, the device, the computer equipment and the storage medium, when the interface identifier in the data request hits the preset interface identifier set, the data identifier corresponding to the data request is calculated according to the interface identifier and the request parameter, and further the first webpage data corresponding to the data request is obtained in the database according to the data identifier, so that the webpage caching technology can be realized no matter whether the data request is requested in any request mode, the use limitation of the webpage caching is improved, and therefore the application has the characteristics of strong universality, and the terminal and the server can support the caching without changing.
Drawings
FIG. 1 is a flow chart of a network data caching method according to an embodiment;
FIG. 2 is a flow chart of a network data caching method according to an embodiment;
FIG. 3 is a block diagram illustrating a network data cache processing apparatus according to an embodiment;
FIG. 4 is an internal block diagram of a computer device in one embodiment;
FIG. 5 is a block diagram illustrating a network data cache processing apparatus according to an embodiment;
FIG. 6 is a schematic diagram of a terminal start configuration flow in a network data caching method according to an embodiment;
FIG. 7 is a schematic diagram of a first access flow of a data request in a network data cache processing method according to an embodiment;
FIG. 8 is a schematic diagram of a data request in a network data cache processing method according to an embodiment as a secondary access flow;
fig. 9 is a schematic diagram of a process of deleting cold data in a network data cache processing method according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
Example 1
In this embodiment, as shown in fig. 1, a network data caching method is provided, which includes:
step 110, acquiring a data request, and analyzing the data request to acquire an interface identifier and a request parameter.
In this embodiment, the terminal receives the data request transmitted by the user at the input device, and thus the terminal acquires the data request. After the terminal acquires the data request, the data request is further analyzed to obtain an interface identifier and a request parameter corresponding to the data request.
For example, when a user opens an application program (App) on a terminal, and the user needs to obtain page data corresponding to a certain function in the application program (App) on the terminal, the user inputs a data request for obtaining a page corresponding to the function in an input device. At this time, the terminal will receive the data request of the page corresponding to the function, and the terminal directly obtains the data request.
Step 120, detecting whether the interface identifier hits in a preset interface identifier set.
In this embodiment, before detecting whether the data identifier exists in the preset interface identifier set, it should be noted that, in this embodiment, detecting whether the interface identifier hits the preset interface identifier set is to detect whether the data identifier exists in the preset interface identifier set, where the terminal needs to obtain the preset interface identifier set, where the preset interface identifier set is used to indicate that the content of the interface request corresponding to the corresponding data request has been cached in the database. For example, the preset interface identifier set may be set to a cached whitelist.
After the terminal obtains the data identifier corresponding to the data request, the terminal needs to detect whether the data identifier exists in the preset cache data identifier set, so as to determine whether the data request corresponding to the data request already exists in the database, and provide a data basis for step 130.
And 130, calculating the interface identifier and the request parameter to obtain a corresponding data identifier when the interface identifier hits the preset interface identifier set.
Since the application program (App) running on the terminal includes many pages corresponding to functions, for example, a first function corresponds to a first page, a second function corresponds to a second page and a third function corresponds to a third page, when the second function is implemented based on the first function, the second page is at a lower level of the first page, that is, there is a high probability that only part of contents of the second page and the first page are dissimilar, and the corresponding data requests all include the requested contents, that is, the contents of the whole URL; in this embodiment, in order to accurately query the database for the data content corresponding to the acquisition request, the data identifier corresponding to the data request is calculated according to the interface identifier and the request parameter.
For example, to more accurately determine whether the data identifier is in the preset cached data identifier set, in one embodiment, the calculating the data identifier corresponding to the data request includes: in this embodiment, since the hash value obtained after the hash processing has uniqueness and the hash value obtained by the whole hash processing has a large difference when the original data is only slightly changed, in this embodiment, the hash value corresponding to the data request is obtained by performing the hash processing on the interface identifier and the request parameter of the data request, and the hash value is used as the data identifier corresponding to the data request.
For example, the data request includes a user ID, an interface identification, and a request parameter, which may be expressed as user_id+api+req. The application (App) generates a request identity req_id from md5 (user_id+api+req).
In this embodiment, any hashing algorithm selected from md5, SHA-1, SHA-256, SHA-3, BLAKE2, RIPEMD or CRC may be used to hash the interface identifier and request parameters of the data request, which is not specifically limited herein.
And 140, acquiring first webpage data corresponding to the data request from a database according to the data identifier.
When the interface identifier exists in the preset interface identifier set, the webpage content corresponding to the identifier data request is cached in the database, so that the terminal can directly acquire the first webpage data corresponding to the data request in the database. In this embodiment, after the first webpage data corresponding to the data request is obtained in the local database, the webpage data is loaded, without obtaining the webpage data from the server, so that the page opening efficiency of the application program is effectively improved.
In this embodiment, when the interface identifier in the data request hits the preset interface identifier set, the data identifier corresponding to the data request is calculated according to the interface identifier and the request parameter, and then the first webpage data corresponding to the data request is obtained in the database according to the data identifier, so that the webpage caching technology can be realized no matter whether the data request is requested in any request mode, and the use limitation of the webpage cache is improved.
It should be understood that, although the steps in the flowchart of fig. 1 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 1 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of other steps or sub-steps of other steps.
In one embodiment, after the first web page data corresponding to the data request is obtained in the database according to the data identifier, the method further includes:
1-1) calculating a first hash value corresponding to the first webpage data, and sending the first hash value and the data request to a server, so that the server obtains second webpage data according to the data request, performs hash processing on the second webpage data to obtain a second hash value, and generates feedback data according to the first hash value, the second hash value and the second webpage data.
In order to verify whether the cached first webpage data in the terminal database is page data corresponding to the latest function of a data request in a latest application program (App), in this embodiment, after acquiring the first webpage data, the terminal acquires X-etag data of the first webpage data, performs hash processing on the X-etag data to obtain a first hash value corresponding to the first webpage data, and sends the first hash value and the data request to a server at the same time; the server obtains second webpage data according to the data request, hashes the second webpage data to obtain a second hash value, and generates feedback data according to the first hash value, the second hash value and the second webpage data. After the server generates the feedback data, the server generates the feedback data to the terminal.
In this embodiment, any hashing algorithm selected from md5, SHA-1, SHA-256, SHA-3, BLAKE2, RIPEMD or CRC may be used to hash the X-etag data, which is not specifically limited herein.
The specific process of generating feedback data by the server is as follows: the server receives the data request and responds to the data request to acquire second webpage data from a corresponding storage medium; then, acquiring the X-etag data of the second webpage data, and carrying out hash processing on the X-etag data of the second webpage data to obtain a second hash value; finally, whether the first hash value is consistent with the second hash value is detected, and when the first hash value is consistent with the second hash value, page data which indicates that the data request corresponds to the function in the latest application program (App) is not changed; when the first hash value is inconsistent with the second hash value, page data indicating that the data request corresponds to a function in the latest application (App) is changed.
In this embodiment, the feedback identifications include a need-to-update identification and a no-need-to-update identification. When the page is determined to have changed, generating a feedback identifier of the page data to be updated, and generating feedback data by combining the feedback identifier and the corresponding second webpage data. When the page is determined to be unchanged, generating a feedback identification which does not need to be updated in the page data, and taking the feedback identification as feedback data.
1-2) receiving the feedback data sent by the server.
1-3) updating the first webpage data according to the feedback data.
After the terminal acquires the feedback data, the terminal analyzes the feedback data to acquire a feedback identifier corresponding to the feedback data; when the feedback identification is the feedback identification which does not need to be updated, the terminal does not need to update the first webpage data in the corresponding database; when the feedback identification is the feedback identification which needs to be updated, the terminal firstly determines a database where the first webpage data is located, and then uses the second webpage data to assign value to the first webpage data to update the first webpage data.
In this embodiment, verifying the page data of the latest function corresponding to the data request corresponding to the application program (App) may help ensure that the security of the application program is updated and enhanced to resist potential attacks. It can also help ensure that the application can still run efficiently and stably after the new functionality is added. And the method can help to perfect the test cases and improve the test coverage rate, thereby better ensuring the overall quality and stability of the application program.
In one embodiment, after the detecting whether the interface identifier hits in a preset interface identifier set, the method further includes:
2-1) when the interface identifier is not in the preset interface identifier set, sending the data request to a server.
In this embodiment, when the interface identifier is not hit in the preset interface identifier set, the data request indicating the corresponding data request is not cached in the terminal database, and then the terminal sends the data request to the server directly or through the gateway, so that the terminal can obtain the data request corresponding to the data request. After the server receives the data request, the server responds to the data request to acquire third webpage data corresponding to the data request from a corresponding storage medium, and the third webpage data is sent to the terminal.
2-2) receiving third webpage data sent by the server in response to the data request.
2-3) loading the third webpage data.
In this embodiment, after the terminal receives the third webpage data, the third webpage data may be further loaded, so that an application program (App) running on the terminal realizes the aesthetic property and the completeness of the function display.
In this embodiment, when the data request is not in the preset cache data identifier set, the data request is directly sent to the server, so as to ensure that the third webpage data corresponding to the data request is obtained, and further, each functional page in the application program (App) can be normally displayed and operated.
In one embodiment, the method further comprises:
3-1) acquiring the first webpage data and the cache time corresponding to the first webpage data from the database.
In the embodiment, the data cached in the database is processed by adopting a time period limitation, specifically, first webpage data is acquired in the database, and then the corresponding caching time of the first webpage data is acquired, namely, the time period for which the first webpage data is stored in the database.
3-2) detecting whether the cache time reaches a preset threshold value.
After the buffer time corresponding to the first webpage data is obtained, the terminal obtains a preset threshold value, and detects whether the buffer time reaches the preset threshold value.
In this embodiment, the terminal may obtain the preset threshold value in the input device, or may directly obtain the preset threshold value in the storage medium, which is not specifically limited herein.
3-3) deleting the first webpage data in a database when the caching time reaches the preset threshold value.
In this embodiment, deleting the cached data cached in the database for a long time can make the operation of the database more efficient, thereby improving the query speed of the database and improving the efficiency of the network data caching processing method of this embodiment; and the larger the amount of data in the database, the more storage space is required. The expired data are not needed any more, and the deleting of the data can release the storage space, so that the size of the database is reduced, and the storage resources are saved. And the real-time performance and accuracy of the data in the database can be ensured.
In one embodiment, before the first web page data corresponding to the data request is obtained in the database according to the data identifier, the method further includes:
4-1) detecting whether the webpage data corresponding to the data request is cached in the database according to the data identifier.
In this embodiment, before the terminal obtains the first web page data corresponding to the data request in the database, it may also detect whether the web page data corresponding to the data request is already stored in the corresponding database; when the webpage data corresponding to the data request is cached in the corresponding database, the terminal can consider that the data request is not the first data request, and the corresponding cached data is stored in the database at the moment. When the webpage data corresponding to the data request is not stored in the corresponding database, the terminal can consider the data request as the first request, and the fact that the corresponding cache data does not exist in the database at the moment is indicated, so that the terminal can directly send the data request to the server and receive the webpage data sent by the server in response to the data request; at this time, the terminal does not need to detect whether the data identifier corresponding to the data request exists in the preset cache data identifier set, so that the execution efficiency of the network data cache processing method in the embodiment is improved.
4-2) when the data of the webpage corresponding to the data request is determined not to be cached in the database according to the data identification, the data is sent to a server.
When the data request is the first request, it means that the data request corresponding to the data request is not cached in the corresponding database, so that the data request needs to be sent to the server, so that the server receives the data request and responds to the data request.
4-3) receiving the web page data sent by the server corresponding to the data request and storing the web page data in a database.
In this embodiment, after the server responds to the data request, the server sends the acquired web page data corresponding to the data request to the terminal. At this time, the terminal receives the web page data transmitted by the server in response to the data request, loads the web page data, and stores the web page data in the database.
Example two
In this embodiment, as shown in fig. 2, a network data caching method is provided, which includes:
step 210, receiving a data request and a first hash value sent by a terminal at the same time, and responding to the data request to obtain second webpage data corresponding to the data request.
When the server receives the data request and the first hash value at the same time, the terminal at the moment needs to verify whether the webpage data corresponding to the data request is updated or not. And the server responds to the data request to acquire the second network page number corresponding to the data request.
In this embodiment, the first hash value is obtained by performing hash processing on the X-etag data of the first web page data acquired by the data request in the corresponding database by the terminal.
Step 230, hash processing is performed on the second web page data to obtain a second hash value.
After the server acquires the second webpage data, the server acquires the corresponding X-etag data in the second webpage data, and hashes the corresponding X-etag data of the second webpage data to obtain a second hash value.
In this embodiment, any hashing algorithm selected from md5, SHA-1, SHA-256, SHA-3, BLAKE2, RIPEMD or CRC may be used to hash the X-etag data, which is not specifically limited herein.
And 240, generating feedback data according to the first hash value, the second hash value and the second webpage data, and sending the feedback data to the terminal.
And after the server acquires the second hash value, detecting whether the first hash value is consistent with the second hash value. When the first hash value is consistent with the second hash value, the server determines that the webpage data corresponding to the data request is not updated, and at the moment, the server generates a feedback mark without updating, and takes the feedback mark without updating as feedback data. When the first hash value is inconsistent with the second hash value, the server determines that the webpage data corresponding to the data request is updated, at the moment, the server generates a feedback mark needing to be updated, and responds to the data request to acquire the corresponding second webpage data, and further generates feedback data by combining the feedback mark needing to be updated and the second webpage data. After the server generates feedback data, the corresponding feedback data is sent to the terminal, so that the terminal updates the first webpage data according to the feedback data.
In this embodiment, by calculating the data identifier corresponding to the data request, and then determining whether the data request corresponding to the data request is in the database according to the data identifier, the web page caching technology can be realized no matter whether the data request is requested in any request mode, and the use limitation of the web page cache is improved.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.
In one embodiment, the method further comprises:
5-1) receiving a data request sent by the terminal, and responding to the second data request to obtain third webpage data.
In this embodiment, when the server only receives the data request, the server may consider that the terminal does not need to store the web page data corresponding to the data request, and then the server may obtain the third web page data in response to the data request. Providing a data base for step 5-2) below.
5-2) sending the third webpage data to the terminal.
In this embodiment, when the server receives the data request, it indicates that the web page data corresponding to the data request sent by the terminal does not need to be cached. Therefore, after receiving the data request, the server directly responds to the data request to obtain third webpage data, and sends the third webpage data to the terminal to provide data for the terminal. And the integrity of the whole scheme is expanded.
Example III
In the prior art, page caching techniques typically include Http 304 caching, page caching, and interface caching. Among other things, the advantages and disadvantages of Http 304 cache, page cache, and interface cache are shown in the following table (table 1):
TABLE 1
As can be seen from the above table 1, the web page caching technology in the prior art has the following disadvantages:
1. http 304 cache: get requests are supported and cached data cannot be accessed offline.
2. For page caching: the universality is weak, and the customization is required for pages; the maintenance cost is relatively high, and when the interface return data changes, the service code needs to be changed.
3. For interface caching: the universality is not enough, the verification logic of a specific service interface is required to be modified, the difference among the services is different, and the service interface cannot be used as a universal method; the total data is returned, so that extra delay and flow waste are caused.
The network data caching processing method provided by the embodiment comprises the following characteristics:
1. the universality is strong, and the server controls the cache.
2. There is no need to add a time field to the response content.
3. non-Get requests are supported.
4. Offline access to cached data is supported.
5. And the access is accelerated, and the flow is saved.
In this embodiment, as shown in fig. 6 to 9, a network data caching method is provided, which includes:
before the network data caching method is executed, a corresponding configuration needs to be started, as shown in fig. 6, specifically:
1. the User (User) starts an application (App).
2. After the application (App) is started, a request for acquiring the cached white list is sent to the GateWay (GateWay).
3. The GateWay (GateWay) requests the configuration Center (Config Center) to acquire the cache white list. The configuration Center (Config Center) returns the cached whitelist to the GateWay (GateWay).
4. The GateWay (GateWay) saves the acquired cached white list into the memory.
5. The GateWay (GateWay) returns the cached whitelist to the application (App).
6. And the application program (App) stores the obtained cache white list into a memory variable and caches the memory variable into a file.
When the user makes the first access after the configuration is started, as shown in fig. 7, specifically:
1. The User (User) first accesses the HTTP interface of the application (App).
2. And the application program (App) decides whether to walk the caching process according to the hit condition of the caching white list.
1) If the interface exists in the buffered white list, the buffered white list is the preset buffered data identifier set in the above embodiment:
a. the application (App) generates a request identity req_id from md5 (user_id+api+req).
b. The application program (App) sends a request to the application database (App DB) to find the cached data by req_id.
c. The application database (App DB) checks the cache upon receipt of the request, finding that it is the first access and therefore no data is cached.
2) If the interface does not exist in the cached white list, the next step is continued.
3. The application (App) displays a popup window in the loading to the User (User).
4. The application (App) sends a request to the GateWay (GateWay) for data.
5. After receiving the request, the GateWay (GateWay) forwards the request to the Server (Server).
6. The Server (Server) processes the request and returns the data to the GateWay (GateWay).
7. The GateWay (GateWay) returns data to the application (App) and decides whether to construct the X-etag according to the setting of the cached whitelist.
8. After receiving the data, an application program (App) hides the popup in the loading process and starts to render the interface.
9. If X-etag is present in the response header, the application program (App) caches the data to the application database (App DB).
When the user makes a secondary access (i.e., not a first access), as shown in fig. 8, the following details are:
(1) the User (User) makes a secondary access to the application (App).
(2) And the application program (App) decides whether to walk the caching process according to the hit condition of the caching white list.
1) If the interface hits the cache white list:
a. the application (App) generates a request identity req_id from md5 (user_id+api+req).
b. The application program (App) sends a request to the application database (App DB) to find the cached data by req_id.
c. After receiving the request, the application database (App DB) checks the cache, finds that the access is secondary, and has cache data.
d. And the application program (App) returns the cache data to the User (User) to realize a second open interface.
2) If the interface misses the cache white list, the next step is continued.
(3) The application (App) sends a request to the GateWay (GateWay) for data.
(4) After receiving the request, the GateWay (GateWay) forwards the request to the Server (Server).
(5) The Server (Server) processes the request and returns the data to the GateWay (GateWay).
(6) The GateWay (GateWay) returns data to the application (App) and decides whether to construct the X-etag according to the setting of the cached whitelist.
(7) If the response status code is 304 (Not Modified), indicating that the cache has Not expired, the application (App) will refresh the last read time.
(8) Otherwise, if the response status code is not 304, indicating that there is new data, the application (App) refreshes the interface to the User (User).
(9) If X-etag is present in the response header, the application program (App) caches the data to the application database (App DB).
If no X-etag exists in the response header, indicating that the new data does not need to be cached, the application (App) deletes the cached data from the database.
In this embodiment, as shown in fig. 9, the cold data deletion (i.e., deleting the cache data in the database) includes:
(1) when a user cold starts an application (App) (the background is not counted back to the foreground), an asynchronous delete data task is triggered.
(2) The application program (App) starts a new thread, asynchronously deletes old data which is not accessed for more than 30 days, and avoids the application program (App) occupying excessive storage space.
In this embodiment, at least the following advantages are included:
1. the method has strong universality, and under the framework, the client and the server can support caching without changing.
2. The hot update, server (configuration center) dynamically controls whether the interface is cached.
3. The code is less intrusive, and no time field (control field in header, no modification of response content) is required to be added to the response content.
4. non-Get requests are supported, and req_id is generated according to md5 (user_id+api+req), in what way.
5. Offline access to cached data (local database, when available) is supported.
6. The access is accelerated, traffic is saved (hit cache, only code=304 is returned, and the whole content does not need to be returned).
Example IV
In this embodiment, as shown in fig. 3, a network data caching apparatus is provided, including: a first acquisition module 310, a first detection module 320, a calculation module 330, and a second acquisition module 340.
A first obtaining module 310, configured to obtain a data request, and parse the data request to obtain an interface identifier and a request parameter;
a first detection module 320, configured to detect whether the interface identifier hits a preset interface identifier set;
a calculating module 330, configured to calculate the interface identifier and the request parameter to obtain a corresponding data identifier when the interface identifier hits the preset interface identifier set;
the second obtaining module 340 is configured to obtain, in a database, first web page data corresponding to the data request according to the data identifier.
In this embodiment, the first obtaining module 310 obtains a data request, and parses the data request to obtain an interface identifier and a request parameter; the interface identification occurs to the detection module 320 and the request parameters are sent to the calculation module 330. The detection module 320 detects whether the interface identifier hits in a preset interface identifier set; when the interface identifier hits the preset interface identifier set, the calculation module 330 calculates the interface identifier and the request parameter to obtain a corresponding data identifier; the data identification occurs to the second acquisition module 340. The second obtaining module 340 obtains the first web page data corresponding to the data request from the database according to the data identifier. When an interface identifier in a data request hits a preset interface identifier set, calculating a data identifier corresponding to the data request according to the interface identifier and a request parameter, and further obtaining first webpage data corresponding to the data request from a database according to the data identifier, so that a webpage caching technology can be realized no matter whether the data request is requested in any request mode, and the use limitation of webpage caching is improved.
In one embodiment, the network data caching apparatus further comprises: the device comprises a first calculation module, a first receiving module, an updating module, a second sending module, a second receiving module, a detection request module, a third sending module, a third receiving module, a third obtaining module, a detection time module and a deleting module.
The first calculation module is used for calculating a first hash value corresponding to the first webpage data, sending the first hash value and the data request to a server, so that the server obtains second webpage data according to the data request, carrying out hash processing on the second webpage data to obtain a second hash value, and generating feedback data according to the first hash value, the second hash value and the second webpage data.
The first receiving module is used for receiving the feedback data sent by the server.
And the updating module is used for updating the first webpage data according to the feedback data.
And the second sending module is used for sending the data request to a server when the interface identifier is not in hit with the preset interface identifier set.
And the second receiving module is used for receiving third webpage data sent by the server in response to the data request.
And the detection request module is used for detecting whether the webpage data corresponding to the data request is cached in the database according to the data identifier.
And the third sending module is used for sending the data to a server when the data of the webpage corresponding to the data request is determined not to be cached in the database according to the data identification.
And the third receiving module is used for receiving the webpage data sent by the server corresponding to the data request and storing the webpage data in a database.
And the third acquisition module is used for acquiring the first webpage data and the cache time corresponding to the first webpage data from the database.
And the detection time module is used for detecting whether the cache time reaches a preset threshold value.
And the deleting module is used for deleting the first webpage data in the database when the caching time reaches the preset threshold value.
The specific limitation of the network data caching device can be referred to the limitation of the network data caching method hereinabove, and will not be described herein. The units in the above network data caching apparatus may be implemented wholly or partly by software, hardware or a combination thereof. The units can be embedded in hardware or independent of a processor in the computer equipment, and can also be stored in a memory in the computer equipment in a software mode, so that the processor can call and execute the operations corresponding to the units.
Example five
In this embodiment, as shown in fig. 5, there is provided a network data caching apparatus, including: a first receiving module 510, a second hashing module 520, and a first transmitting module 530.
The first receiving module 510 is configured to receive a data request and a first hash value sent by a terminal at the same time, and obtain second web page data corresponding to the data request in response to the data request.
And the second hash module 520 is configured to hash the second web page data to obtain a second hash value.
And a first sending module 530, configured to generate feedback data according to the first hash value, the second hash value, and the second web page data, and send the feedback data to the terminal.
In this embodiment, the first receiving module 510 receives a data request and a first hash value sent by a terminal at the same time, and obtains second webpage data corresponding to the data request in response to the data request; the second web page data is sent to the second hash module 520. The second hash module 520 hashes the second web page data to obtain a second hash value; the second hash value is sent to the first sending module 530. The first sending module 530 generates feedback data according to the first hash value, the second hash value and the second webpage data, and sends the feedback data to the terminal. By calculating the data identifier corresponding to the data request and then determining whether the data request corresponding to the data request is in the database according to the data identifier, the webpage caching technology can be realized no matter whether the data request is requested in any request mode, and the use limitation of webpage caching is improved.
In one embodiment, the network data caching apparatus further comprises: a response module and a fourth transmission module.
And the response module is used for receiving the data request sent by the terminal and responding to the data request to obtain third webpage data.
And the fourth sending module is used for sending the third webpage data to the terminal.
The specific limitation of the network data caching device can be referred to the limitation of the network data caching method hereinabove, and will not be described herein. The units in the above network data caching apparatus may be implemented wholly or partly by software, hardware or a combination thereof. The units can be embedded in hardware or independent of a processor in the computer equipment, and can also be stored in a memory in the computer equipment in a software mode, so that the processor can call and execute the operations corresponding to the units.
Example six
In this embodiment, a computer device is provided. The internal structure thereof can be shown in fig. 4. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program, and is deployed with a database for storing all relevant data related to the network data caching method. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used to communicate with other computer devices in which application software is deployed. The computer program is executed by a processor to implement a network data caching method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the structures shown in FIG. 4 are block diagrams only and do not constitute a limitation of the computer device on which the present aspects apply, and that a particular computer device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the network data caching method described in any one of the embodiments above when the computer program is executed.
Example seven
In this embodiment, a computer readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the steps of the network data caching method described in any one of the above embodiments.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. The network data caching processing method is characterized by comprising the following steps of:
acquiring a data request, and analyzing the data request to acquire an interface identifier and a request parameter;
detecting whether the interface identifier hits a preset interface identifier set or not;
when the interface identifier hits the preset interface identifier set, calculating the interface identifier and the request parameter to obtain a corresponding data identifier;
And acquiring first webpage data corresponding to the data request from a database according to the data identifier.
2. The method according to claim 1, wherein after the first web page data corresponding to the data request is obtained in the database according to the data identifier, the method further comprises:
calculating a first hash value corresponding to the first webpage data, and sending the first hash value and the data request to a server so that the server obtains second webpage data according to the data request, performs hash processing on the second webpage data to obtain a second hash value, and generates feedback data according to the first hash value, the second hash value and the second webpage data;
receiving the feedback data sent by the server;
and updating the first webpage data according to the feedback data.
3. The method of claim 1, wherein after detecting whether the interface identifier hits in a preset set of interface identifiers, the method further comprises:
when the interface identifier is not hit in the preset interface identifier set, the data request is sent to a server;
and receiving third webpage data sent by the server in response to the data request.
4. The method of claim 1, wherein before the first web page data corresponding to the data request is obtained in the database according to the data identifier, the method further comprises:
detecting whether the webpage data corresponding to the data request is cached in a database according to the data identifier;
when the data identification determines that the webpage data corresponding to the data request is not cached in the database, the data is sent to a server;
and receiving the webpage data sent by the server corresponding to the data request, and storing the webpage data in a database.
5. The method according to any one of claims 1 or 2, further comprising:
acquiring the first webpage data and the cache time corresponding to the first webpage data from the database;
detecting whether the cache time reaches a preset threshold value;
and deleting the first webpage data in a database when the caching time reaches the preset threshold value.
6. The network data caching processing method is characterized by comprising the following steps of:
receiving a data request and a first hash value which are simultaneously sent by a terminal, and responding to the data request to acquire second webpage data corresponding to the data request;
Carrying out hash processing on the second webpage data to obtain a second hash value;
generating feedback data according to the first hash value, the second hash value and the second webpage data, and sending the feedback data to the terminal.
7. The method of claim 6, wherein the method further comprises:
receiving a data request sent by a terminal, and responding to the data request to obtain third webpage data;
and sending the third webpage data to the terminal.
8. A network data caching apparatus, comprising:
the first acquisition module is used for acquiring a data request and analyzing the data request to acquire an interface identifier and a request parameter;
the detection module is used for detecting whether the interface identifier hits a preset interface identifier set or not;
the calculation module is used for calculating the interface identifier and the request parameter to obtain a corresponding data identifier when the interface identifier hits the preset interface identifier set;
and the second acquisition module is used for acquiring the first webpage data corresponding to the data request from the database according to the data identifier.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410139315.8A CN117880363A (en) | 2024-01-31 | 2024-01-31 | Network data caching processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410139315.8A CN117880363A (en) | 2024-01-31 | 2024-01-31 | Network data caching processing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117880363A true CN117880363A (en) | 2024-04-12 |
Family
ID=90583008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410139315.8A Pending CN117880363A (en) | 2024-01-31 | 2024-01-31 | Network data caching processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117880363A (en) |
-
2024
- 2024-01-31 CN CN202410139315.8A patent/CN117880363A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112073405B (en) | Webpage data loading method and device, computer equipment and storage medium | |
| US10042948B2 (en) | Identifying correlated components of dynamic content | |
| US9055124B1 (en) | Enhanced caching of network content | |
| US9147067B2 (en) | Security method and apparatus | |
| CN105824909B (en) | Page generation method and device | |
| CN110866198B (en) | Static resource caching method, system, device, computer equipment and storage medium | |
| US10909104B2 (en) | Caching of updated network content portions | |
| US10091289B2 (en) | Provisional execution of dynamic content component | |
| CN105930528B (en) | Webpage caching method and server | |
| US10645192B2 (en) | Identifying content files in a cache using a response-based cache index | |
| CN108683668B (en) | Resource checking method, device, storage medium and equipment in content distribution network | |
| US20150363369A1 (en) | Method and Apparatus of Processing Nested Fragment Caching of a Web Page | |
| WO2019127890A1 (en) | Vulnerability scanning method, device, computer apparatus, and storage medium | |
| CN112153170B (en) | Method, device and equipment for accessing server and storage medium | |
| CN108256014B (en) | Page display method and device | |
| CN108920703B (en) | HTTP cache optimization method and device | |
| US20190045026A1 (en) | Offline use of network application | |
| CN110555184A (en) | resource caching method and device, computer equipment and storage medium | |
| CN112181532B (en) | Page resource loading method and device, electronic equipment and readable storage medium | |
| WO2020252486A1 (en) | Proactive conditioned prefetching and origin flooding mitigation for content delivery | |
| CN112104703A (en) | Cookie management method, intermediate node and webvpn system | |
| CN111783005A (en) | Method, apparatus and system for displaying web page, computer system and medium | |
| CN117880363A (en) | Network data caching processing method and device, computer equipment and storage medium | |
| CN110460685B (en) | User unique identifier processing method and device, computer equipment and storage medium | |
| CN114629875A (en) | Active detection domain name brand protection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |