CN110311975B - Data request processing method and device - Google Patents
Data request processing method and device Download PDFInfo
- Publication number
- CN110311975B CN110311975B CN201910574217.6A CN201910574217A CN110311975B CN 110311975 B CN110311975 B CN 110311975B CN 201910574217 A CN201910574217 A CN 201910574217A CN 110311975 B CN110311975 B CN 110311975B
- Authority
- CN
- China
- Prior art keywords
- data
- client
- sending
- stored
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the application provides a data request processing method and a data request processing device, which belong to the technical field of computers, wherein the method is applied to a reverse proxy server and comprises the following steps: receiving a data request sent by a client, wherein the data request carries a data identifier of data to be acquired; judging whether first data corresponding to the data identification is stored locally; and if the first data is locally stored, sending the first data to the client. By the method and the device, the processing efficiency of the data request can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data request processing method and apparatus.
Background
When a user wants to obtain target data, the user may perform a search operation at the client, so that the client of the user sends a data request to the reverse proxy server, where the target data may be video data of a certain video, the data request may carry a data parameter of the target data, and the data parameter may be a data identifier. The reverse proxy server may, upon receiving the data request, forward the data request to a storage server storing the target data. The storage server may determine target data corresponding to the data identifier of the target data according to a correspondence between pre-stored data identifiers and data. The storage server may then send the target data to a reverse proxy server, which in turn sends the target data to the user's client.
However, when the data requests currently being processed by the storage server are excessive or the storage server is abnormal in operation, the storage server cannot determine the target data corresponding to a certain data request within the preset response time, the storage server may return an error message to the reverse proxy server, and then the reverse proxy server may send a preset search failure message to the client to prompt the user that the target data cannot be obtained. Therefore, when the data requests currently being processed by the storage server are too many or the storage server works abnormally, the reverse proxy server cannot send the target data to the client, and the efficiency of processing the data requests is low.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method and an apparatus for processing a data request, so as to improve the processing efficiency of the data request. The specific technical scheme is as follows:
in a first aspect, a data request processing method is provided, where the method is applied to a reverse proxy server, where the reverse proxy server is communicatively connected to a storage server, and the reverse proxy server caches part of data in the storage server, and the method includes:
receiving a data request sent by a client, wherein the data request carries a data identifier of data to be acquired;
judging whether first data corresponding to the data identification is stored locally;
and if the first data is locally stored, sending the first data to the client.
Optionally, the method further includes:
if the first data is not stored locally, sending the data request to the storage server;
and when second data sent by the storage server is received, sending the second data to the client, and locally storing the second data.
Optionally, if the first data is stored locally, sending the first data to the client includes:
and if a busy message sent by the storage server is received and the first data is locally stored, sending the first data to the client, wherein the busy message is used for indicating that the storage server cannot process a data request.
Optionally, the sending the first data to the client includes:
acquiring the stored duration of the first data;
if the stored time length is smaller than a preset storage time length threshold value, the first data are sent to the client;
if the stored time length is greater than a preset storage time length threshold value, sending the data request to the storage server;
and when an error message which is sent by the storage server and used for indicating that the data request processing fails is received, sending the first data to the client.
Optionally, the method further includes:
according to a preset updating period, determining redundant data meeting a preset processing condition, wherein the preset processing condition comprises that the request times in a preset time period are lower than a preset request time threshold;
and deleting the redundant data.
In a second aspect, a data request processing apparatus is provided, where the apparatus is applied to a reverse proxy server, where the reverse proxy server is communicatively connected to a storage server, and the reverse proxy server caches part of data in the storage server, and the apparatus includes:
the receiving module is used for receiving a data request sent by a client, wherein the data request carries a data identifier of data to be acquired;
the judging module is used for judging whether first data corresponding to the data identification is stored locally;
and the first sending module is used for sending the first data to the client when the first data is locally stored.
Optionally, the apparatus further comprises:
the second sending module is used for sending the data request to the storage server when the first data is not stored locally;
and the third sending module is used for sending the second data to the client side when receiving the second data sent by the storage server, and locally storing the second data.
Optionally, the first sending module includes:
and the first sending submodule is used for sending the first data to the client when a busy message sent by the storage server is received and the first data is locally stored, wherein the busy message is used for indicating that the storage server cannot process a data request.
Optionally, the first sending module includes:
the acquisition submodule is used for acquiring the stored duration of the first data;
the second sending submodule is used for sending the first data to the client when the stored time length is smaller than a preset storage time length threshold;
the third sending submodule is used for sending the data request to the storage server when the stored time length is greater than a preset storage time length threshold value;
the second sending submodule is further configured to send the first data to the client when receiving an error message sent by the storage server and used for indicating that data request processing fails.
Optionally, the apparatus further comprises:
the determining module is used for determining redundant data meeting a preset processing condition according to a preset updating period, wherein the preset processing condition comprises that the request times in a preset time period are lower than a preset request time threshold;
and the deleting module is used for deleting the redundant data.
In a third aspect, a reverse proxy server is provided, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing the method steps of any data request processing method when executing the program stored in the memory.
In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of the data request processing methods.
The embodiment of the application provides a data request processing method and device, which are applied to a reverse proxy server. The reverse proxy server can receive a data request sent by the client, wherein the data request carries a data identifier of data to be acquired; judging whether first data corresponding to the data identification is stored locally; and if the first data are locally stored, sending the first data to the client. The reverse proxy server can store the data to be acquired and can directly send the data to be acquired to the client, so that the processing efficiency of the data request can be improved.
Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a flowchart of a data request processing method according to an embodiment of the present application;
fig. 2 is a flowchart of a data request processing method according to an embodiment of the present application;
fig. 3 is a flowchart of a data request processing method according to an embodiment of the present application;
fig. 4 is a flowchart of a data request processing method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data request processing apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a reverse proxy server according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
The embodiment of the application provides a data request processing method, which is applied to a reverse proxy server, wherein the reverse proxy server can be an Nginx server. The reverse proxy server is in communication connection with the storage server, and part of data in the storage server may be cached in the reverse proxy server. In this embodiment, the historical data may be data that has been sent to the corresponding client by the reverse proxy server in response to the data request of each client. For the convenience of management, the reverse proxy server may further store a corresponding relationship between the history data and the data identifier.
In the embodiment of the application, the reverse proxy server can have the function of storing data in a way of logically expanding the reverse proxy server. In one possible implementation, the data in the reverse proxy server may be stored in a redis database.
As shown in fig. 1, a specific processing procedure of the data request processing method provided in the embodiment of the present application includes:
The data request may carry data parameters of the data to be acquired, such as data identifier. The data to be acquired may be video data of a certain video, and the data to be acquired may also be audio data of a certain audio.
In an implementation, when a user wants to obtain certain data, the user may perform a search operation at the client to cause the user's client to send a data request to the reverse proxy server. Then, the reverse proxy server may receive a data request sent by the client, and obtain a data identifier of the data to be obtained carried by the data request.
In a possible implementation manner, the client may generate a data request in the form of an http (hypertext transfer protocol) request, and a header of the http request may include data parameters such as a cache parameter and a timeout time. The cache parameter is used to indicate related information of the data to be acquired, for example, the cache parameter may be a data identifier of the data to be acquired or version information of the data to be acquired, the timeout time is used to indicate a longest waiting time for an http request to wait for a request response, and the timeout time is, for example, 1 s. The reverse proxy server may obtain the data identifier of the data to be obtained carried by the http request after receiving the http request.
And 102, judging whether first data corresponding to the data identification is stored locally.
In implementation, the reverse proxy server may search the data identifier of the data to be acquired in the corresponding relationship between the locally stored historical data and the data identifier, and if the data identifier of the data to be acquired is found in the corresponding relationship, take the historical data corresponding to the data identifier as the first data, and determine that the first data corresponding to the data identifier of the data to be acquired is locally stored.
If the data identifier is not found in the corresponding relationship, it is determined that the first data corresponding to the data identifier of the data to be acquired is not stored locally, and the processing procedure of the reverse proxy server will be described in detail later.
And 103, if the first data is locally stored, sending the first data to the client.
In an implementation, the reverse proxy server may send the first data to the client if the first data is stored locally.
In the embodiment of the present application, the reverse proxy server may store therein history data that has been sent to each client. The reverse proxy server may obtain a data identifier carried by the data request after receiving the data request sent by the client, then determine whether the local stores first data corresponding to the data identifier, and send the first data to the client if the first data exists. According to the embodiment of the application, by expanding the cache function of the reverse proxy server, when the reverse proxy server receives a data request requesting the same data, the reverse proxy server can directly obtain the first data corresponding to the data request from the local, and then send the first data to the client. Therefore, the problem that the storage server needs to process the data requests received by all the reverse proxy servers and possibly cannot respond to all the data requests in time is solved, the load pressure of the storage server can be reduced, and the processing efficiency of the data requests is improved.
Alternatively, as shown in fig. 2, if the reverse proxy server does not locally store the first data, the server may perform the following steps:
The storage server stores data to be acquired required by a user, and also stores the corresponding relation between the data and the data identifier.
In an implementation, the reverse proxy server may send a data request to the storage server if the first data is not stored locally.
In implementation, after receiving a data request sent by the reverse proxy server, the storage server may determine second data corresponding to the data identifier of the data to be acquired according to the correspondence between the data and the data identifier, and then send the second data to the reverse proxy server.
At this time, the reverse proxy server may receive the second data transmitted by the storage server, and then, the reverse proxy server may transmit the second data to the client and locally store the second data.
In a possible implementation manner, the storage server may store a preset response time, for example, 1 s. If the data request currently processed by the storage server is excessive, or the storage server works abnormally, the storage server returns an error message to the reverse proxy server when the storage server cannot determine the second data corresponding to the data identifier of the data to be acquired within the preset response time.
The reverse proxy server may send a preset search failure message to the client after receiving the error message, so as to prompt that the user fails to acquire data.
In the embodiment of the application, when the reverse proxy server does not locally store the first data (namely, the data to be acquired) corresponding to the data identifier of the data to be acquired, the reverse proxy server sends a data request to the storage server, receives the second data determined by the storage server, and sends the second data to the client. Therefore, when the reverse proxy server does not locally store the data to be acquired, the data to be acquired is acquired through the storage server, the cache function of the reverse proxy server can be fully utilized, the problem that the storage server needs to process all data requests received by the reverse proxy server and possibly cannot respond to all the data requests in time is solved, the processing efficiency of the data requests can be improved, the load pressure of the storage server is reduced, and the maintenance cost of the storage server is reduced.
Optionally, when the data requests currently being processed by the storage server are too many, or the storage server works abnormally, the storage server may send a busy message to the reverse proxy server. After receiving the busy message, the reverse proxy server may determine that the working status of the storage server is that the data request cannot be processed, and in this case, if the first data is locally stored, the reverse proxy server may send the first data to the client.
In the embodiment of the application, when the first data are determined to be locally stored, if the reverse proxy server receives a busy message sent by the storage server, the reverse proxy server can directly send the first data to the client, so that the problems that the storage server needs to process data requests received by all the reverse proxy servers and possibly cannot respond to all the data requests in time can be avoided, the load pressure of the storage server can be reduced, and the processing efficiency of the data requests is improved.
Optionally, a storage duration threshold of the historical data may be preset in the reverse proxy server. If the first data is locally stored, the reverse proxy server may determine whether to send the first data to the client according to the storage duration threshold, as shown in fig. 3, where the specific processing procedure includes:
In an implementation, the reverse proxy server may obtain a stored duration of the first data after determining that the first data is locally stored, and then the reverse proxy server may compare the stored duration of the first data with a storage duration threshold.
In an implementation, the reverse proxy server may send the first data to the client if the stored duration is less than a preset storage duration threshold.
For example, the storage duration threshold is 48 hours, the stored duration of the first data is 24 hours, the reverse proxy server compares the stored duration of 24 hours with the storage duration threshold of 48 hours, and determines that the stored duration is less than the preset storage duration threshold, the reverse proxy server may send the first data to the client.
Optionally, if the stored duration is greater than the preset storage duration threshold, the specific processing procedure of the reverse proxy server includes:
step 1, sending a data request to a storage server.
And step 2, when an error message which is sent by the storage server and used for indicating that the data request processing fails is received, sending the first data to the client.
In implementation, when an error message which is sent by the storage server and used for indicating that the data request processing fails is received, the first data is sent to the client.
In one possible implementation, the reverse proxy server may store target data parameters, such as a cache parameter, a timeout time, contained in the http request header before sending the data request in the form of an http request to the storage server.
The storage server may obtain a target data parameter included in the http request header after receiving the http request, and when the storage server determines that the data to be obtained cannot be determined within the preset response time, the storage server may generate an error message including the target data parameter, and then send the error message to the reverse proxy server.
The reverse proxy server may obtain the target data parameter included in the error message, compare the target data parameter with the locally stored target data parameter, and if the target data parameter is the same as the locally stored target data parameter, the reverse proxy server may send the locally stored first data to the client. If not, the reverse proxy server may send a preset search failure message to the client to prompt the user that the data acquisition fails.
And 3, when receiving second data sent by the storage server, sending the second data to the client, deleting the first data, and locally storing the second data.
In implementation, after receiving a data request sent by the reverse proxy server, the storage server may determine second data corresponding to the data identifier of the data to be acquired according to the correspondence between the data and the data identifier, and then send the second data to the reverse proxy server.
At this time, the reverse proxy server may receive the second data transmitted by the storage server, and then, the reverse proxy server may transmit the second data to the client, delete the locally stored first data, and locally store the second data.
In the embodiment of the application, when the stored duration of the first data is greater than the threshold of the stored duration, the reverse proxy server may send a data request to the storage server, so that when the storage server sends the second data, the first data stored locally is updated, and the second data is sent to the client; meanwhile, the reverse proxy server may also send the first data to the client when the storage server sends an error message indicating that the data request processing failed. Compared with the prior art that a reverse proxy server sends a search failure message to a client to prompt that data acquisition fails, when a storage server cannot respond to a data request in time, the reverse proxy server can send first data to the client, so that the client does not display a preset search failure page any more but displays the first data, and user experience can be improved.
Optionally, because the storage space of the reverse proxy server is limited, the reverse proxy server may periodically clean the locally stored historical data, as shown in fig. 4, the specific processing procedure includes:
The reverse proxy server may be preset with an update period, for example, 30 days. The reverse proxy server may also be preset with a preset processing condition, where the preset processing condition may be that the number of requests in the update period is lower than a preset request number threshold, and the preset processing condition may also be that the occupied storage space is greater than a preset storage space threshold.
In implementation, the reverse proxy server may determine, as the redundant data, historical data satisfying a preset processing condition among the locally stored historical data according to a preset update period.
For example, the threshold of the number of requests is set to 5, and the reverse proxy server may use, as redundant data, historical data with the number of requests smaller than 5 in the locally stored historical data after the current running time reaches 30 days.
In the embodiment of the application, the reverse proxy server determines the redundant data meeting the preset processing conditions according to the updating period, and then deletes the redundant data, so that the reverse proxy server can cache the data which are more interesting to the user in the limited storage space. Therefore, the data to be acquired can be directly determined locally and sent to the client, the problem that the storage server needs to process data requests received by all reverse proxy servers and possibly cannot respond to partial data requests in time is solved, the load pressure of the storage server can be reduced, and the processing efficiency of the data requests is improved.
An embodiment of the present application further provides a data request processing apparatus, where the apparatus is applied to a reverse proxy server, where the reverse proxy server is in communication connection with a storage server, and the reverse proxy server caches part of data in the storage server, as shown in fig. 5, the apparatus includes:
a receiving module 510, configured to receive a data request sent by a client, where the data request carries a data identifier of data to be acquired;
the judging module 520 is configured to judge whether first data corresponding to the data identifier is stored locally;
a first sending module 530, configured to send the first data to the client when the first data is stored locally.
Optionally, the apparatus further comprises:
the second sending module is used for sending the data request to the storage server when the first data is not stored locally;
and the third sending module is used for sending the second data to the client side when receiving the second data sent by the storage server, and locally storing the second data.
Optionally, the first sending module includes:
and the first sending submodule is used for sending the first data to the client when a busy message sent by the storage server is received and the first data is locally stored, wherein the busy message is used for indicating that the storage server cannot process a data request.
Optionally, the first sending module includes:
the acquisition submodule is used for acquiring the stored duration of the first data;
the second sending submodule is used for sending the first data to the client when the stored time length is smaller than a preset storage time length threshold;
the third sending submodule is used for sending the data request to the storage server when the stored time length is greater than a preset storage time length threshold value;
the second sending submodule is further configured to send the first data to the client when receiving an error message sent by the storage server and used for indicating that data request processing fails.
Optionally, the apparatus further comprises:
the determining module is used for determining redundant data meeting a preset processing condition according to a preset updating period, wherein the preset processing condition comprises that the request times in a preset time period are lower than a preset request time threshold;
and the deleting module is used for deleting the redundant data.
The data request processing device provided by the embodiment of the application is applied to a reverse proxy server. The reverse proxy server can receive a data request sent by the client, wherein the data request carries a data identifier of data to be obtained; judging whether first data corresponding to the data identification is stored locally; and if the first data is locally stored, sending the first data to the client. The reverse proxy server can store the data to be acquired and can directly send the data to be acquired to the client, so that the processing efficiency of the data request can be improved.
The embodiment of the present application further provides a reverse proxy server, as shown in fig. 6, which includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604,
a memory 603 for storing a computer program;
the processor 601 is configured to implement the following steps when executing the program stored in the memory 603:
receiving a data request sent by a client, wherein the data request carries a data identifier of data to be acquired;
judging whether first data corresponding to the data identification is stored locally;
and if the first data is locally stored, sending the first data to the client.
Optionally, the method further includes:
if the first data is not stored locally, sending the data request to the storage server;
and when second data sent by the storage server is received, sending the second data to the client, and locally storing the second data.
Optionally, if the first data is stored locally, sending the first data to the client includes:
and if a busy message sent by the storage server is received and the first data is locally stored, sending the first data to the client, wherein the busy message is used for indicating that the storage server cannot process a data request.
Optionally, the sending the first data to the client includes:
acquiring the stored duration of the first data;
if the stored time length is smaller than a preset storage time length threshold value, the first data are sent to the client;
if the stored time length is greater than a preset storage time length threshold value, sending the data request to the storage server;
and when an error message which is sent by the storage server and used for indicating that the data request processing fails is received, sending the first data to the client.
Optionally, the method further includes:
according to a preset updating period, determining redundant data meeting a preset processing condition, wherein the preset processing condition comprises that the request times in a preset time period are lower than a preset request time threshold;
and deleting the redundant data.
The communication bus mentioned above for the reverse proxy server may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the reverse proxy server and other devices.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The reverse proxy server provided by the embodiment of the application can receive a data request sent by a client, wherein the data request carries a data identifier of data to be acquired; judging whether first data corresponding to the data identification is stored locally; and if the first data is locally stored, sending the first data to the client. The reverse proxy server can store the data to be acquired and can directly send the data to be acquired to the client, so that the processing efficiency of the data request can be improved.
In yet another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above data request processing methods.
In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the data request processing methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims (10)
1. A data request processing method applied to a reverse proxy server, wherein the reverse proxy server is communicatively connected to a storage server, and the reverse proxy server caches a part of data in the storage server, the method comprising:
receiving a data request sent by a client, wherein the data request carries a data identifier of data to be acquired;
judging whether first data corresponding to the data identification is stored locally;
if the first data are stored locally, the first data are sent to the client;
the sending the first data to the client comprises:
acquiring the stored duration of the first data;
if the stored time length is smaller than a preset storage time length threshold value, the first data are sent to the client;
if the stored time length is greater than a preset storage time length threshold value, storing a target data parameter in the data request, and sending the data request to the storage server, wherein the target data parameter comprises at least one of a cache parameter and timeout time;
when an error message which is sent by the storage server and used for indicating that data request processing fails is received, acquiring a target data parameter contained in the error message, comparing the target data parameter contained in the error message with the target data parameter in the stored data request, if the target data parameter contained in the error message is the same as the target data parameter contained in the stored data request, sending the first data to the client, and if the target data parameter contained in the error message is not the same as the target data parameter contained in the stored data request, sending a preset search failure message to the client;
when second data sent by the storage server is received, sending the second data to a client, deleting the first data, and locally storing the second data; the second data is data acquired by the storage server responding to the data request.
2. The method of claim 1, further comprising:
if the first data is not stored locally, sending the data request to the storage server;
and when second data sent by the storage server is received, sending the second data to the client, and locally storing the second data.
3. The method of claim 1, wherein sending the first data to the client if the first data is stored locally comprises:
and if a busy message sent by the storage server is received and the first data is locally stored, sending the first data to the client, wherein the busy message is used for indicating that the storage server cannot process a data request.
4. The method of claim 1, further comprising:
according to a preset updating period, determining redundant data meeting a preset processing condition, wherein the preset processing condition comprises that the request times in a preset time period are lower than a preset request time threshold;
and deleting the redundant data.
5. A data request processing apparatus, wherein the apparatus is applied to a reverse proxy server, the reverse proxy server is communicatively connected to a storage server, and the reverse proxy server caches a part of data in the storage server, and the apparatus comprises:
the receiving module is used for receiving a data request sent by a client, wherein the data request carries a data identifier of data to be acquired; the judging module is used for judging whether first data corresponding to the data identification is stored locally;
the first sending module is used for sending the first data to the client when the first data is locally stored;
the first transmitting module further comprises:
the acquisition submodule is used for acquiring the stored duration of the first data;
the second sending submodule is used for sending the first data to the client when the stored time is smaller than a preset storage time threshold;
the third sending submodule is used for storing a target data parameter in the data request and sending the data request to the storage server when the stored time length is greater than a preset storage time length threshold, wherein the stored target data parameter comprises at least one of a cache parameter and timeout time;
the second sending submodule is further configured to, when an error message sent by the storage server and used for indicating that data request processing fails is received, obtain a target data parameter included in the error message, compare the target data parameter included in the error message with the target data parameter in the stored data request, send the first data to the client if the target data parameter included in the error message is the same as the target data parameter in the stored data request, and send a preset search failure message to the client if the target data parameter included in the error message is not the same as the target data parameter in the stored data request; when second data sent by the storage server is received, sending the second data to a client, deleting the first data, and locally storing the second data; the second data is data acquired by the storage server responding to the data request.
6. The apparatus of claim 5, further comprising:
the second sending module is used for sending the data request to the storage server when the first data is not stored locally;
and the third sending module is used for sending the second data to the client side when receiving the second data sent by the storage server, and locally storing the second data.
7. The apparatus of claim 5, wherein the first sending module comprises:
and the first sending submodule is used for sending the first data to the client when a busy message sent by the storage server is received and the first data is locally stored, wherein the busy message is used for indicating that the storage server cannot process a data request.
8. The apparatus of claim 5, further comprising:
the determining module is used for determining redundant data meeting preset processing conditions according to a preset updating period, wherein the preset processing conditions comprise that the number of requests in a preset time period is lower than a preset request threshold;
and the deleting module is used for deleting the redundant data.
9. The reverse proxy server is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 4 when executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910574217.6A CN110311975B (en) | 2019-06-28 | 2019-06-28 | Data request processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910574217.6A CN110311975B (en) | 2019-06-28 | 2019-06-28 | Data request processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110311975A CN110311975A (en) | 2019-10-08 |
| CN110311975B true CN110311975B (en) | 2022-06-03 |
Family
ID=68079317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910574217.6A Active CN110311975B (en) | 2019-06-28 | 2019-06-28 | Data request processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110311975B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110839074A (en) * | 2019-11-07 | 2020-02-25 | 北京首钢自动化信息技术有限公司 | Data request receiving and processing method and device |
| CN110955688A (en) * | 2019-12-23 | 2020-04-03 | 北京奇艺世纪科技有限公司 | Proxy server, data query method and device, electronic equipment and storage medium |
| CN114390098B (en) * | 2020-10-21 | 2024-09-27 | 北京金山云网络技术有限公司 | Data transmission method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106909690A (en) * | 2017-03-07 | 2017-06-30 | 四川驹马企业管理有限公司 | Network data caching method |
| CN106911735A (en) * | 2015-12-22 | 2017-06-30 | 北京奇虎科技有限公司 | Data capture method and device |
| CN107222567A (en) * | 2017-07-07 | 2017-09-29 | 北京京东尚科信息技术有限公司 | Method, device and the service cluster of processing data request |
| CN109831536A (en) * | 2019-03-22 | 2019-05-31 | 成都六四三六五科技有限公司 | A kind of front end load balancing service degradated system and method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160075698A (en) * | 2013-10-28 | 2016-06-29 | 후아웨이 테크놀러지 컴퍼니 리미티드 | Method, device and system for redirecting data service proxy |
| CN107943594B (en) * | 2016-10-13 | 2021-11-12 | 北京京东尚科信息技术有限公司 | Data acquisition method and device |
| CN109922053A (en) * | 2019-02-22 | 2019-06-21 | 北京三快在线科技有限公司 | Data transmission method, device, electronic equipment and readable storage medium storing program for executing |
-
2019
- 2019-06-28 CN CN201910574217.6A patent/CN110311975B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106911735A (en) * | 2015-12-22 | 2017-06-30 | 北京奇虎科技有限公司 | Data capture method and device |
| CN106909690A (en) * | 2017-03-07 | 2017-06-30 | 四川驹马企业管理有限公司 | Network data caching method |
| CN107222567A (en) * | 2017-07-07 | 2017-09-29 | 北京京东尚科信息技术有限公司 | Method, device and the service cluster of processing data request |
| CN109831536A (en) * | 2019-03-22 | 2019-05-31 | 成都六四三六五科技有限公司 | A kind of front end load balancing service degradated system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110311975A (en) | 2019-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110311975B (en) | Data request processing method and device | |
| CN111585867B (en) | Message processing method and device, electronic equipment and readable storage medium | |
| CN108696581B (en) | Distributed information caching method and device, computer equipment and storage medium | |
| CN112087644B (en) | Pull stream request processing method, device and system, electronic equipment and storage medium | |
| CN108647240B (en) | Method and device for counting access amount, electronic equipment and storage medium | |
| CN109450969B (en) | Method and device for acquiring data from third-party data source server and server | |
| CN111046310A (en) | Page processing method, device, server and computer readable storage medium | |
| CN112437148B (en) | Service request processing method and device, service server and system | |
| CN110430070B (en) | Service state analysis method, device, server, data analysis equipment and medium | |
| CN115080272A (en) | Business data processing method, electronic device and storage medium | |
| CN114629883B (en) | Service request processing method and device, electronic equipment and storage medium | |
| CN114691611A (en) | Directory refreshing processing method and device, electronic equipment and storage medium | |
| CN108132836B (en) | Task distribution method and device and electronic equipment | |
| CN112653736B (en) | Parallel source returning method and device and electronic equipment | |
| US20170286440A1 (en) | Method, business processing server and data processing server for storing and searching transaction history data | |
| CN112069386A (en) | Request processing method, device, system, terminal and server | |
| CN110955460A (en) | Service process starting method and device, electronic equipment and storage medium | |
| CN114090268B (en) | Container management method and container management system | |
| CN111753233B (en) | Third-party H5 page loading method and device and computer readable storage medium | |
| CN113242245A (en) | TCP message flow limiting method, device, server and storage medium | |
| CN113138943B (en) | Method and device for processing request | |
| CN109995855B (en) | Data acquisition method, device and terminal | |
| CN113014456A (en) | Monitoring data transmission system, method, device, equipment and storage medium | |
| CN111291127A (en) | Data synchronization method, device, server and storage medium | |
| CN113572824B (en) | Dynamic updating method and device for timeout duration, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |