CN113810455A - Back-end data access system, method, device and storage medium - Google Patents

Back-end data access system, method, device and storage medium Download PDF

Info

Publication number
CN113810455A
CN113810455A CN202110059589.2A CN202110059589A CN113810455A CN 113810455 A CN113810455 A CN 113810455A CN 202110059589 A CN202110059589 A CN 202110059589A CN 113810455 A CN113810455 A CN 113810455A
Authority
CN
China
Prior art keywords
target data
cache
data
end server
long connection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110059589.2A
Other languages
Chinese (zh)
Inventor
宋振亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Tuoxian Technology Co Ltd
Original Assignee
Beijing Jingdong Tuoxian Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Tuoxian Technology Co Ltd filed Critical Beijing Jingdong Tuoxian Technology Co Ltd
Priority to CN202110059589.2A priority Critical patent/CN113810455A/en
Publication of CN113810455A publication Critical patent/CN113810455A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention provides a back-end data access system, a method, a device and a storage medium, and the technical scheme is as follows: the method comprises the steps that a front-end device initiates a short connection request aiming at target data to a rear-end server to establish short connection with the rear-end server, the rear-end server searches target data in a cache, under the condition that the target data are not found in the cache, an asynchronous task for obtaining the target data from a data source is started, a miss cache mark is returned to the front-end device through the short connection, and therefore the front-end device initiates a long connection request aiming at the target data to the rear-end server to establish long connection with the rear-end server, the rear-end server stores the target data obtained through the asynchronous task into the cache, and the target data obtained through the asynchronous task are returned to the front-end device through the long connection. The invention can reduce the request times to the back-end server, reduce the occupation of network resources and reduce the load of the back-end server.

Description

Back-end data access system, method, device and storage medium
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a back-end data access system, method, apparatus, and storage medium.
Background
In the prior art, the access process of a user to backend data is as follows: the user initiates a request to the back-end server through the front-end equipment, the back-end server searches data in the cache firstly, if the data is found, the data is directly returned to the front-end equipment, if the data is not found, the data is obtained from the data source and is put into the cache, and then the data obtained from the data source is returned to the front-end equipment. If there are multiple data sources, or the data sources themselves can only return request results in batches, or it is time-consuming to obtain data from the data sources, the front-end device needs to initiate multiple requests to obtain complete data.
Taking communication between a web front end and a back end server as an example, the web front end initiates an http request to the back end server, and if the http request hits a cache of the back end server, data is directly returned; if the cache of the back-end server is not hit, the back-end server is required to obtain data from the data source. Because the http request has overtime, if one http request cannot acquire complete data, the http request needs to be continuously initiated again to continuously acquire the data until the complete data is acquired.
It can be seen that the front-end device not only increases the number of requests to the back-end server to cause network resource occupation, but also increases the load of the back-end server by initiating the requests many times to obtain complete data from the back-end server.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a back-end data access system, method, apparatus and storage medium, which can reduce the number of requests to a back-end server, reduce the occupation of network resources, and reduce the load of the back-end server.
In order to achieve the purpose, the invention provides the following technical scheme:
a back-end data access system, comprising: a front-end device and a back-end server;
the front-end equipment is used for initiating a short connection request aiming at target data to a back-end server; the cache miss processing system is used for receiving a miss cache mark returned by the back-end server through short connection with the front-end equipment and initiating a long connection request aiming at target data to the back-end server; the system comprises a front-end device and a back-end server, wherein the front-end device is used for receiving target data returned by the back-end server through long connection with the front-end device;
the back-end server is used for receiving a short link request aiming at target data initiated by the front-end equipment to establish the short connection with the front-end equipment, searching the target data in a cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short connection if the target data is not searched in the cache; the long connection request aiming at the target data and initiated by the front-end equipment is received to establish the long connection with the front-end equipment, the target data acquired through the asynchronous task is stored in a cache, and the target data acquired through the asynchronous task is returned to the front-end equipment through the long connection.
The first back-end data access method is applied to front-end equipment in a back-end data access system, and comprises the following steps:
initiating a short connection request aiming at target data to a back-end server in the system so that the back-end server receives the short connection request to establish short connection with the front-end equipment, searching the target data in a cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short connection under the condition that the target data is not searched in the cache;
receiving a miss cache mark returned by the back-end server through the short connection, initiating a long connection request aiming at target data to the back-end server so that the back-end server receives the long connection request to establish long connection with the front-end equipment, storing the target data acquired through the asynchronous task into a cache, and returning the target data acquired through the asynchronous task to the front-end equipment through the long connection;
and receiving target data returned by the back-end server through the long connection.
A second backend data access method, applied to a backend server in a backend data access system, includes:
receiving a short link request aiming at target data initiated by front-end equipment in the system to establish a short link with the front-end equipment, searching the target data in a cache, if the target data is not searched in the cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short link so that the front-end equipment initiates a long link request aiming at the target data to a back-end server;
receiving a long link request aiming at target data initiated by the front-end equipment to establish a long connection with the front-end equipment, storing the target data acquired through the asynchronous task into a cache, and returning the target data acquired through the asynchronous task to the front-end equipment through the long connection.
A first back-end data access apparatus, applied to a front-end device in a back-end data access system, the apparatus comprising:
a short connection initiating unit, configured to initiate a short connection request for target data to a back-end server in the system, so that the back-end server receives the short connection request to establish a short connection with the front-end device, searches for the target data in a cache, starts an asynchronous task for obtaining the target data from a data source and returns a miss cache flag to the front-end device through the short connection if the target data is not found in the cache;
a long connection initiating unit, configured to initiate a long connection request for target data to a backend server after the receiving unit receives a miss cache flag returned by the backend server through the short connection, so that the backend server receives the long connection request to establish a long connection with the front-end device, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device through the long connection;
a receiving unit, configured to receive a miss cache flag returned by the backend server through the short connection; and the target data is used for receiving the target data returned by the back-end server through the long connection.
A second backend data access apparatus, applied to a backend server in a backend data access system, the apparatus comprising:
the short connection establishing unit is used for receiving a short connection request aiming at target data and initiated by front-end equipment in the system so as to establish short connection between the back-end server and the front-end equipment;
a long connection establishing unit, configured to receive a long connection request for target data initiated by the front-end device to establish a long connection between the back-end server and the front-end device;
a data obtaining unit, configured to search for target data in a cache after the short connection establishing unit establishes the short connection between the backend server and the front-end device, and if the target data is not found in the cache, start an asynchronous task that obtains the target data from a data source and return a miss cache flag to the front-end device through the short connection, so that the front-end device initiates a long connection request for the target data to the backend server; the long connection establishing unit is used for storing the target data acquired by the asynchronous task into a cache after establishing the long connection between the back-end server and the front-end equipment, and returning the target data acquired by the asynchronous task to the front-end equipment through the long connection.
An electronic device, comprising: the system comprises at least one processor and a memory connected with the at least one processor through a bus; the memory stores one or more computer programs executable by the at least one processor; the at least one processor, when executing the one or more computer programs, implements the steps in the first back-end data access method described above, or implements the steps in the second back-end data access method described above.
A computer readable storage medium storing one or more computer programs which, when executed by a processor, implement the steps of the first back-end data access method described above, or implement the steps of the second back-end data access method described above.
According to the technical scheme, the front-end equipment initiates a short connection request aiming at the target data to the back-end server, when the back-end server does not find the target data in the cache, an asynchronous task for acquiring the target data from the data source is started, the front-end equipment is informed of the missing of the cache, the front-end equipment then initiates a long connection request aiming at the target data to the back-end server, and therefore the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end equipment through long connection. It can be seen that, since it is time consuming for the back-end server to obtain the target data from the data source, and the front-end device initiates the long connection to the back-end server so that the front-end device only needs to initiate two connection requests to obtain the complete target data, and does not need to initiate the short connection for multiple times to obtain the complete data, the number of requests to the back-end server can be reduced, the network resource occupation can be reduced, and the load of the back-end server can be reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a schematic diagram of a backend data access system architecture according to an embodiment of the present invention;
FIG. 2 is a flow chart of a backend data access method of an embodiment of the present invention;
FIG. 3 is a flowchart of a second backend data access method according to an embodiment of the present invention;
FIG. 4 is a flow chart of a three backend data access method of an embodiment of the present invention;
FIG. 5 is a flow chart of a four back-end data access method according to an embodiment of the present invention;
FIG. 6 is a flow chart of a five backend data access method of an embodiment of the present invention;
FIG. 7 is a flowchart of a six backend data access method according to an embodiment of the present invention;
FIG. 8 is a block diagram of a backend data access device according to an embodiment of the present invention;
FIG. 9 is a block diagram of a second backend data access device according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a schematic architecture diagram of a backend data access system according to an embodiment of the present invention, and as shown in fig. 1, the system includes: a front-end device 10, a back-end server 20, and caches and data sources that serve the back-end server 20.
In this embodiment, the front-end device 10 is configured to initiate a short connection request for target data to the back-end server 20. The back-end server 20 is configured to receive a short link request for the target data initiated by the front-end device 10 to establish the short connection with the front-end device 10, search for the target data in the cache, start an asynchronous task for obtaining the target data from the data source if the target data is not found in the cache, and return a miss cache flag to the front-end device 10 through the short connection.
In this embodiment, the front-end device 10 is configured to receive a miss cache flag returned by the back-end server 20 through a short connection with the front-end device 10, and initiate a long connection request for the target data to the back-end server 20. The back-end server 20 is configured to receive a long connection request for target data initiated by the front-end device 10 to establish the long connection with the front-end device 10, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device 10 through the long connection.
In this embodiment, the front-end device 10 is configured to receive the target data returned by the back-end server 20 through the long connection with the front-end device 10, so as to achieve access to the target data.
As can be seen from the system shown in fig. 1, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server does not find the target data in the cache, starts an asynchronous task for obtaining the target data from the data source, and notifies the front-end device that the cache is not hit, so that the front-end device then initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data obtained from the data source through the asynchronous task to the front-end device through the long connection. The back-end server is time-consuming to obtain the target data from the data source, and the front-end equipment can obtain the complete target data only by initiating two connection requests through initiating the long connection to the back-end server, and does not need to obtain the complete data by initiating the short connection for multiple times continuously, so that the number of requests to the back-end server can be reduced, the network resource occupation is reduced, and the load of the back-end server is reduced.
The embodiment of the present invention further provides a second backend data access system, where the second backend data access system has the same architecture as the system shown in fig. 1, that is: the second backend data access system also comprises: a front-end device 10, a back-end server 20, and caches and data sources that serve the back-end server 20.
In this embodiment, the front-end device 10 may be triggered by the user to initiate a short connection request for the target data to the back-end server 20, thereby opening the data access flow.
In this embodiment, the front-end device 10 is configured to initiate a short connection request for target data to the back-end server 20. The back-end server 20 is configured to receive a short link request for target data initiated by the front-end device 10 to establish the short connection with the front-end device 10, search for the target data in the cache, return the searched target data to the front-end device 10 through the short connection if the target data is found in the cache, and return a hit cache tag to the front-end device 10 through the short connection at the same time; if the target data is not found in the cache, an asynchronous task for obtaining the target data from the data source is started and a miss cache mark is returned to the front-end device 10 through the short connection.
In this embodiment, when the target data is stored in the cache of the back-end server 20, the front-end device 10 is configured to receive a hit cache flag and the target data returned by the back-end server 20 through a short connection with the front-end device 10, and display the target data to the user, so as to implement access to the target data.
In this embodiment, in the case that the target data is not stored in the cache of the back-end server 20, the front-end device 10 is configured to receive a miss cache flag returned by the back-end server 20 through a short connection with the front-end device 10, and initiate a long connection request for the target data to the back-end server 20. The back-end server 20 is configured to receive a long connection request for target data initiated by the front-end device 10 to establish the long connection with the front-end device 10, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device 10 through the long connection. Accordingly, the front-end device 10 is configured to receive the target data returned by the back-end server 20 through the long connection with the front-end device 10, and may display the target data to the user, so as to achieve access to the target data.
In this embodiment, the front-end device 10 may be a web client, the back-end server may be a web server, and the short connection may be a short connection based on an http request; the long connection may be a webSocket request based long connection.
As can be seen from the above, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server finds the target data in the cache, the target data can be directly returned through the short connection; when the back-end server does not find the target data in the cache, starting an asynchronous task for acquiring the target data from the data source, and informing the front-end equipment of missing the cache, so that the front-end equipment initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end equipment through long connection. It can be seen that when the target data is stored in the cache of the back-end server, the target data can be directly returned to the front-end device, and the front-end device only needs to initiate a connection request once; when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, and the time is long, for this reason, the front-end equipment initiates a long connection mode to the back-end server, so that the front-end equipment only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection continuously for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server.
The embodiment of the present invention further provides a third backend data access system, where the third backend data access system has the same architecture as the system shown in fig. 1, that is: the third back-end data access system also includes: a front-end device 10, a back-end server 20, and caches and data sources that serve the back-end server 20.
In this embodiment, the front-end device 10 is configured to initiate a short connection request for target data to the back-end server 20. The back-end server 20 is configured to receive a short link request for target data initiated by the front-end device 10 to establish the short connection with the front-end device 10, search for the target data in the cache, return the searched target data to the front-end device 10 through the short connection if the target data is found in the cache, and return a hit cache tag to the front-end device 10 through the short connection at the same time; if the target data is not found in the cache, an asynchronous task for obtaining the target data from the data source is started and a miss cache mark is returned to the front-end device 10 through the short connection.
In this embodiment, when the target data is stored in the cache of the back-end server 20, the front-end device 10 is configured to receive a hit cache flag and the target data returned by the back-end server 20 through a short connection with the front-end device 10, and display the target data to the user, so as to implement access to the target data.
In this embodiment, in the case that the target data is not stored in the cache of the back-end server 20, the front-end device 10 is configured to receive a miss cache flag returned by the back-end server 20 through a short connection with the front-end device 10, and initiate a long connection request for the target data to the back-end server 20. The back-end server 20 is configured to receive a long connection request for target data initiated by the front-end device 10 to establish the long connection with the front-end device 10, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device 10 through the long connection. Accordingly, the front-end device 10 is configured to receive the target data returned by the back-end server 20 through the long connection with the front-end device 10, so as to achieve access to the target data.
In the existing implementation, after data is stored in the cache, a validity period is usually set for the data, and the data is always stored in the cache in the validity period, and after the validity period is invalid, the data is deleted from the cache. Since the validity period of the data does not change after the setting, even if the data is frequently accessed, the data belonging to the hot spot is deleted from the cache after the validity period is expired, so that when the data is accessed again, the data needs to be acquired from the data source again, and the access delay is caused.
In order to solve the above problem, in this embodiment, when the backend server 20 stores the target data acquired through the asynchronous task in the cache, the cache validity duration of the target data may be further set to be a first preset duration, and after the cache validity duration of the target data is set to be the first preset duration, the following operations may be further performed: and periodically updating the effective cache duration of the target data until the effective cache duration of the target data is invalid.
In this embodiment, the periodically updating the effective caching duration of the target data by the backend server 20 may specifically include: and counting the accessed times of the target data in each updating period, determining a new effective cache time length according to the principle that the accessed times are in direct proportion to the effective cache time length when the updating period is ended, and updating the effective cache time length of the target data into the new effective cache time length.
For example, a function f (x) a × x + b for calculating the effective buffer duration may be set according to a principle that the number of times of access is proportional to the effective buffer duration, where x is the number of times of access to the target data in an update period, and a is a coefficient greater than 0, and may be set according to the service requirement; b is the base buffer duration. Then, at the end of each update cycle, a new cache effective duration may be calculated using the function, and the effective cache duration of the target data may be updated to the new cache effective duration. It can be seen that, as the number of times that target data is accessed in a certain update period is greater, the new effective cache duration calculated by using the above function at the end of the update period is longer, so that the longer the target data is stored in the cache, the longer the hot spot data is stored in the cache can be satisfied.
If the cache validity duration of the target data expires at a time before the end of the update period, it may be determined that the cache validity duration of the target data expires at the time, and the target data may be deleted from the cache at the time. For example, assuming that one update cycle is 10 minutes, and the new valid cache time calculated at the end of the previous update cycle is 7 minutes, and thus the valid cache time of the target data is updated to 7 minutes, the valid cache time of the target data expires 7 minutes after the start of the present cycle (the valid cache time of the target data is not updated since the present cycle has not ended), and the target data may be deleted from the cache. Therefore, in this embodiment, the back-end server 20 is further configured to delete the target data from the cache when the cache validity duration of the target data is invalid.
As can be seen from the above, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server finds the target data in the cache, the target data can be directly returned through the short connection; when the back-end server does not find the target data in the cache, starting an asynchronous task for acquiring the target data from the data source, and informing the front-end equipment of missing the cache, so that the front-end equipment initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end equipment through long connection. It can be seen that when the target data is stored in the cache of the back-end server, the target data can be directly returned to the front-end device, and the front-end device only needs to initiate a connection request once; when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, and the time is long, for this reason, the front-end equipment initiates a long connection mode to the back-end server, so that the front-end equipment only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection continuously for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server. In addition, in this embodiment, the backend server also periodically updates the effective cache duration of the target data, so that the target data can be stored in the cache for a longer time as the hot spot data, thereby shortening the access delay to the hot spot data.
The embodiment of the present invention further provides a fourth backend data access system, where the fourth backend data access system has the same architecture as the system shown in fig. 1, that is: the fourth back-end data access system also includes: a front-end device 10, a back-end server 20, and caches and data sources that serve the back-end server 20.
In this embodiment, the front-end device 10 is configured to initiate a short connection request for target data to the back-end server 20. The back-end server 20 is configured to receive a short link request for the target data initiated by the front-end device 10 to establish the short connection with the front-end device 10, search for the target data in the cache, start an asynchronous task for obtaining the target data from the data source if the target data is not found in the cache, and return a miss cache flag to the front-end device 10 through the short connection.
In this embodiment, the front-end device 10 is configured to receive a miss cache flag returned by the back-end server 20 through a short connection with the front-end device 10, and initiate a long connection request for the target data to the back-end server 20. The back-end server 20 is configured to receive a long connection request for target data initiated by the front-end device 10 to establish the long connection with the front-end device 10, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device 10 through the long connection.
In this embodiment, the front-end device 10 is configured to receive the target data returned by the back-end server 20 through the long connection with the front-end device 10, and may present the target data to the user, so as to implement access to the target data.
In practical applications, when the front-end device 10 initiates a large number of connection requests (short connection requests or long connection requests) to the back-end server 20 in a short time due to malicious access or access peak, a great processing pressure is applied to the back-end server 20, and in order to ensure normal operation of the back-end server 20, a certain prevention means is usually adopted for such a situation, for example, current limiting processing is performed on the connection requests initiated by the front-end device 10.
In this embodiment, when receiving the short connection request initiated by the front-end device 10, the back-end server 20 may further accumulate the number of connection requests received within the second preset time, and if the accumulated result exceeds the preset threshold, perform current limiting processing on the connection request initiated by the front-end device 10.
In this embodiment, when receiving the long connection request initiated by the front-end device 10, the back-end server 20 may further accumulate the number of connection requests received within the second preset time period last, and if the accumulated result exceeds the preset threshold, perform current limiting processing on the connection request initiated by the front-end device 10.
In this embodiment, the current limiting process may specifically be a current limiting measure such as denial of service, queuing or waiting, or degradation.
As can be seen from the above, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server does not find the target data in the cache, starts an asynchronous task for acquiring the target data from the data source, and notifies the front-end device that the cache is not hit, so that the front-end device then initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end device through the long connection. It can be seen that, when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, which is time-consuming, and for this reason, the front-end device initiates a long connection to the back-end server so that the front-end device only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server. In addition, in this embodiment, when the backend server further receives too many connection requests within a set time, normal operation of the backend server is ensured by performing current-limiting processing on the connection request initiated by the front-end device, and the stability of the system can be improved.
The embodiment of the present invention further provides a fifth backend data access system, where the fifth backend data access system has the same architecture as the system shown in fig. 1, that is: the fifth back-end data access system also comprises: a front-end device 10, a back-end server 20, and caches and data sources that serve the back-end server 20.
In this embodiment, the front-end device 10 is configured to initiate a short connection request for target data to the back-end server 20. The back-end server 20 is configured to receive a short link request for the target data initiated by the front-end device 10 to establish the short connection with the front-end device 10, search for the target data in the cache, start an asynchronous task for obtaining the target data from the data source if the target data is not found in the cache, and return a miss cache flag to the front-end device 10 through the short connection.
In this embodiment, the front-end device 10 is configured to receive a miss cache flag returned by the back-end server 20 through a short connection with the front-end device 10, and initiate a long connection request for the target data to the back-end server 20. The back-end server 20 is configured to receive a long connection request for target data initiated by the front-end device 10 to establish the long connection with the front-end device 10, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device 10 through the long connection.
In this embodiment, the front-end device 10 is configured to receive the target data returned by the back-end server 20 through the long connection with the front-end device 10, and may present the target data to the user, so as to implement access to the target data.
In practical applications, when an access request is initiated for data of different service types, the amount of returned data usually varies, and some variations are very large. In the case of a small amount of returned data, the backend server 20 may generally return all the data to the front-end device 10 at one time, and in the case of a large amount of returned data, the backend server 20 may not return all the data to the front-end device 10 at one time, and therefore the data is generally returned to the front-end device 10 in batches.
In this embodiment, in order to predict the amount of return data in advance, the front-end device 10 may carry the request traffic type in the long connection request when initiating the long connection to the back-end server 20. And the back-end server 20 may pre-configure the data backhaul method corresponding to each service type. The data back transmission mode comprises full back transmission and incremental back transmission; the full feedback refers to one-time transmission of complete data; incremental backtracking refers to batch transmission of complete data.
In this embodiment, the back-end server 20 returns the target data acquired through the asynchronous task to the front-end device 10 through the long connection, and specifically includes: determining a data returning mode corresponding to the request service type carried by the long connection request, returning the target data acquired by the asynchronous task to the front-end equipment 10 through the long connection at one time when the data returning mode is incremental returning, and returning the target data acquired by the asynchronous task to the front-end equipment 10 through the long connection in batches when the data returning mode is incremental returning;
correspondingly, the front-end device 10 is configured to receive the target data returned by the back-end server 20 through the long connection at one time, and display the received target data to the user; and is further configured to receive target data returned by the backend server 20 through the long connection in batches, and display the target data received in each batch to the user in an incremental manner.
As can be seen from the above, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server does not find the target data in the cache, starts an asynchronous task for acquiring the target data from the data source, and notifies the front-end device that the cache is not hit, so that the front-end device then initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end device through the long connection. It can be seen that, when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, which is time-consuming, and for this reason, the front-end device initiates a long connection to the back-end server so that the front-end device only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server. In addition, in this embodiment, the background server may also adaptively return the target data to the front-end device in one or more batches according to the request service type of the front-end device.
The embodiment of the present invention further provides a backend data access method applied to a front-end device in the backend data access system, and a backend data access method applied to a backend server in the backend data access system, which is described in detail below with reference to the accompanying drawings.
Referring to fig. 2, fig. 2 is a flowchart of a backend data access method according to an embodiment of the present invention, where the method is applied to a front-end device in a backend data access system, and as shown in fig. 2, the method mainly includes the following steps:
step 201, initiating a short connection request for target data to a back-end server in the system, so that the back-end server receives the short connection request to establish a short connection with the front-end device, searching for the target data in a cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end device through the short connection under the condition that the target data is not found in the cache;
in this embodiment, the front-end device may be triggered by the user to initiate a short connection request for the target data to the back-end server, thereby opening the data access flow.
In this embodiment, when the back-end server searches for the target data in the cache, if the target data is found, the target data does not need to be obtained from the data source, but the target data found in the cache may be directly returned to the front-end device through the short connection. When the front-end equipment receives the target data returned by the back-end server through the short connection, the target data can be displayed to the user, and therefore the target data can be accessed.
Step 202, receiving a miss cache flag returned by the back-end server through the short connection, initiating a long connection request for target data to the back-end server, so that the back-end server receives the long connection request to establish a long connection with the front-end device, storing the target data acquired through the asynchronous task in a cache, and returning the target data acquired through the asynchronous task to the front-end device through the long connection;
in this embodiment, the front-end device 10 may be a web client, the back-end server may be a web server, and the short connection may be a short connection based on an http request; the long connection may be a webSocket request based long connection.
And 203, receiving the target data returned by the back-end server through the long connection.
In this embodiment, the front-end device receives the target data returned by the back-end server through the long connection, and may display the target data to the user, thereby implementing access to the target data.
As can be seen from the method shown in fig. 2, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server does not find the target data in the cache, starts an asynchronous task for obtaining the target data from the data source, and notifies the front-end device of a miss of the cache, so that the front-end device then initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data obtained from the data source through the asynchronous task to the front-end device through the long connection. The back-end server is time-consuming to obtain the target data from the data source, and the front-end equipment can obtain the complete target data only by initiating two connection requests through initiating the long connection to the back-end server, and does not need to obtain the complete data by initiating the short connection for multiple times continuously, so that the number of requests to the back-end server can be reduced, the network resource occupation is reduced, and the load of the back-end server is reduced.
Referring to fig. 3, fig. 3 is a flowchart of a second backend data access method according to an embodiment of the present invention, where the method is applied to a backend server in a backend data access system, and as shown in fig. 3, the method mainly includes the following steps:
step 301, receiving a short link request for target data initiated by a front-end device in the system to establish a short link with the front-end device, searching for the target data in a cache, if the target data is not found in the cache, starting an asynchronous task for obtaining the target data from a data source and returning a miss cache mark to the front-end device through the short link, so that the front-end device initiates a long link request for the target data to a back-end server;
step 302, receiving a long link request for target data initiated by the front-end device to establish a long connection with the front-end device, storing the target data acquired by the asynchronous task in a cache, and returning the target data acquired by the asynchronous task to the front-end device through the long connection.
As can be seen from the method shown in fig. 3, in this embodiment, the front-end device initiates a short connection request for the target data to the back-end server, starts an asynchronous task for acquiring the target data from the data source when the back-end server does not find the target data in the cache, and notifies the front-end device of a miss of the cache, so that the front-end device initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source by the asynchronous task to the front-end device through the long connection. The back-end server is time-consuming to obtain the target data from the data source, and the front-end equipment can obtain the complete target data only by initiating two connection requests through initiating the long connection to the back-end server, and does not need to obtain the complete data by initiating the short connection for multiple times continuously, so that the number of requests to the back-end server can be reduced, the network resource occupation is reduced, and the load of the back-end server is reduced.
Referring to fig. 4, fig. 4 is a flowchart of a three-backend data access method according to an embodiment of the present invention, where the method is applied to a backend server in a backend data access system, and as shown in fig. 4, the method mainly includes the following steps:
step 401, receiving a short link request for target data initiated by a front-end device in the system to establish a short connection with the front-end device, searching for the target data in a cache, if the target data is found in the cache, executing step 402, and if the target data is not found in the cache, executing step 403;
in this embodiment, the front-end device may be triggered by the user to initiate a short connection request for the target data to the back-end server, thereby opening the data access flow.
Step 402, returning the searched target data to the front-end device through the short connection, so that the front-end device displays the received target data to the user, and going to step 405.
Step 403, starting an asynchronous task for obtaining target data from a data source and returning a miss cache mark to the front-end device through the short connection, so that the front-end device initiates a long connection request for the target data to a back-end server;
step 404, receiving a long link request for target data initiated by the front-end device to establish a long connection with the front-end device, storing the target data acquired by the asynchronous task in a cache, and returning the target data acquired by the asynchronous task to the front-end device through the long connection.
And step 405, ending the process.
As can be seen from the method shown in fig. 4, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server finds the target data in the cache, the target data can be directly returned through the short connection; when the back-end server does not find the target data in the cache, starting an asynchronous task for acquiring the target data from the data source, and informing the front-end equipment of missing the cache, so that the front-end equipment initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end equipment through long connection. It can be seen that when the target data is stored in the cache of the back-end server, the target data can be directly returned to the front-end device, and the front-end device only needs to initiate a connection request once; when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, and the time is long, for this reason, the front-end equipment initiates a long connection mode to the back-end server, so that the front-end equipment only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection continuously for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server.
Referring to fig. 5, fig. 5 is a flowchart of a four backend data access method according to an embodiment of the present invention, where the method is applied to a backend server in a backend data access system, and as shown in fig. 5, the method mainly includes the following steps:
step 501, receiving a short link request for target data initiated by a front-end device in the system to establish a short connection with the front-end device, searching for the target data in a cache, executing step 502 if the target data is found in the cache, and executing step 503 if the target data is not found in the cache;
step 502, returning the found target data to the front-end device through the short connection, so that the front-end device displays the received target data to the user, and going to step 509.
Step 503, starting an asynchronous task for acquiring target data from a data source and returning a miss cache mark to the front-end device through the short connection, so that the front-end device initiates a long connection request for the target data to a back-end server;
step 504, receiving a long connection request aiming at target data initiated by the front-end equipment to establish a long connection with the front-end equipment, and executing step 505 and step 508;
in this embodiment, step 505 and step 508 may be executed simultaneously, without being in sequence.
Step 505, storing the target data acquired by the asynchronous task into a cache, and setting the effective cache duration of the target data to be a first preset duration;
in this embodiment, when the target data is stored in the cache, the cache validity duration of the target data is set to an initial value, that is, a first preset duration, and then the cache validity duration of the target data may be periodically updated according to the access condition (for example, the access frequency, that is, the number of times of access per unit time) of the target data, so that the hot data may be stored in the cache for a longer time, thereby reducing the access delay to the hot data.
Step 506, periodically updating the effective cache duration of the target data until the effective cache duration of the target data is invalid;
in the existing implementation, after data is stored in the cache, a validity period is usually set for the data, and the data is always stored in the cache in the validity period, and after the validity period is invalid, the data is deleted from the cache. Since the validity period of the data does not change after the setting, even if the data is frequently accessed, the data belonging to the hot spot is deleted from the cache after the validity period is expired, so that when the data is accessed again, the data needs to be acquired from the data source again, and the access delay is caused.
In order to solve the above problem, in this embodiment, after setting the cache validity duration of the target data to be the first preset duration, the backend server may periodically update the cache validity duration of the target data until the cache validity duration of the target data is invalid.
In this embodiment, the periodically updating, by the backend server, the effective cache duration of the target data may specifically include: and counting the accessed times of the target data in each updating period, determining a new effective cache time length according to the principle that the accessed times are in direct proportion to the effective cache time length when the updating period is ended, and updating the effective cache time length of the target data into the new effective cache time length.
For example, a function f (x) a × x + b for calculating the effective buffer duration may be set according to a principle that the number of times of access is proportional to the effective buffer duration, where x is the number of times of access to the target data in an update period, and a is a coefficient greater than 0, and may be set according to the service requirement; b is the base buffer duration. Then, at the end of each update cycle, a new cache effective duration may be calculated using the function, and the effective cache duration of the target data may be updated to the new cache effective duration. It can be seen that, as the number of times that target data is accessed in a certain update period is greater, the new effective cache duration calculated by using the above function at the end of the update period is longer, so that the longer the target data is stored in the cache, the longer the hot spot data is stored in the cache can be satisfied.
Step 507, when the effective time length of the cache of the target data is invalid, deleting the target data from the cache, and going to step 509.
In this embodiment, if the cache validity period of the target data expires at a time before the end of the update period, it may be determined that the cache validity period of the target data expires at this time, at which point the target data may be deleted from the cache. For example, assuming that the update period is 10 minutes, the new valid cache time calculated at the end of the previous update period is 7 minutes, and thus the valid cache time of the target data is updated to 7 minutes, the valid cache time of the target data expires 7 minutes after the start of the present period (the valid cache time of the target data is not updated since the present period has not ended), and the target data may be deleted from the cache.
And step 508, returning the target data acquired by the asynchronous task to the front-end equipment through the long connection.
Step 509, the process ends.
As can be seen from the method shown in fig. 5, in this embodiment, the front-end device first initiates a short connection request for the target data to the back-end server, and when the back-end server finds the target data in the cache, the target data can be directly returned through the short connection; when the back-end server does not find the target data in the cache, starting an asynchronous task for acquiring the target data from the data source, and informing the front-end equipment of missing the cache, so that the front-end equipment initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source through the asynchronous task to the front-end equipment through long connection. It can be seen that when the target data is stored in the cache of the back-end server, the target data can be directly returned to the front-end device, and the front-end device only needs to initiate a connection request once; when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, and the time is long, for this reason, the front-end equipment initiates a long connection mode to the back-end server, so that the front-end equipment only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection continuously for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server. In addition, in this embodiment, the backend server also periodically updates the effective cache duration of the target data, so that the target data can be stored in the cache for a longer time as the hot spot data, thereby shortening the access delay to the hot spot data.
Referring to fig. 6, fig. 6 is a flowchart of a five-backend data access method according to an embodiment of the present invention, where the method is applied to a backend server in a backend data access system, and as shown in fig. 5, the method mainly includes the following steps:
601, receiving a short link request aiming at target data initiated by a front-end device in the system to establish a short connection with the front-end device, and executing step 602 and step 603;
in this embodiment, step 602 and step 603 may be performed simultaneously, without being in a sequential order.
Step 602, accumulating the number of connection requests received within the second preset time period, and if the accumulation result exceeds a preset threshold, performing current limiting processing on the connection request initiated by the front-end device, and going to step 607.
Step 603, searching target data in the cache, if the target data is not searched in the cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short connection, so that the front-end equipment initiates a long connection request for the target data to a back-end server;
step 604, receiving a long connection request aiming at target data initiated by the front-end equipment to establish a long connection with the front-end equipment, and executing step 605 and step 606;
in this embodiment, step 605 and step 606 may be performed simultaneously, without being in a sequential order.
Step 605, accumulating the number of times of the connection request received within the second preset time period, if the accumulated result exceeds a preset threshold, performing current limiting processing on the connection request initiated by the front-end equipment, and going to step 607.
Step 606, storing the target data obtained by the asynchronous task in a cache, and returning the target data obtained by the asynchronous task to the front-end device through the long connection, and then going to step 607.
Step 607, the process ends.
As can be seen from the method shown in fig. 6, in this embodiment, the front-end device initiates a short connection request for the target data to the back-end server, starts an asynchronous task for acquiring the target data from the data source when the back-end server does not find the target data in the cache, and notifies the front-end device of a miss of the cache, so that the front-end device initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source by the asynchronous task to the front-end device through the long connection. It can be seen that, when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, which is time-consuming, and for this reason, the front-end device initiates a long connection to the back-end server so that the front-end device only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server. In addition, in this embodiment, when the backend server further receives too many connection requests within a set time, normal operation of the backend server is ensured by performing current-limiting processing on the connection request initiated by the front-end device, and the stability of the system can be improved.
Referring to fig. 7, fig. 7 is a flowchart of a six backend data access method according to an embodiment of the present invention, where the method is applied to a backend server in a backend data access system, and as shown in fig. 7, the method mainly includes the following steps:
step 701, receiving a short link request for target data initiated by a front-end device in the system to establish a short link with the front-end device, searching for the target data in a cache, if the target data is not found in the cache, starting an asynchronous task for obtaining the target data from a data source and returning a miss cache mark to the front-end device through the short link, so that the front-end device initiates a long link request for the target data to a back-end server;
step 7021, receiving a long connection request for target data initiated by the front-end device to establish a long connection with the front-end device, and executing step 7022 and step 7023.
Step 7022, storing the target data obtained by the asynchronous task in a cache, and going to step 703.
Step 7023, determining a data backhaul method corresponding to the request service type carried by the long connection request, and when the data backhaul method is incremental backhaul, returning the target data acquired by the asynchronous task to the front-end device through the long connection at one time, so that the front-end device displays the received target data to a user; when the data returning mode is incremental returning, returning the target data acquired by the asynchronous task to the front-end device in batches through the long connection, so that the front-end device displays the target data received in each batch to the user in an incremental mode, and going to step 703.
In practical applications, when an access request is initiated for data of different service types, the amount of data returned by a backend server usually varies, and some variations are very large. In the case of a small amount of returned data, the backend server can usually return all the data to the front-end device at one time, and in the case of a large amount of returned data, the backend server cannot return all the data to the front-end device at one time, and therefore the data are usually returned to the front-end device in batches.
In this embodiment, in order to predict the amount of return data in advance, the front-end device may carry the request traffic type in the long connection request when initiating the long connection to the back-end server. And the back-end server can pre-configure a data return mode corresponding to each service type, so that when a long connection request initiated by the front-end equipment is received, the data return mode corresponding to the request service type carried by the long connection request can be determined, and then target data acquired from a data source is returned to the front-end equipment according to the data return mode. The data back transmission mode comprises full back transmission and incremental back transmission; the full feedback refers to one-time transmission of complete data; incremental backtracking refers to batch transmission of complete data.
The above step 7023 is a specific refinement of "returning the target data acquired by the asynchronous task to the front-end device through the long connection" in step 302 shown in fig. 3.
The above steps 7021 to 7023 are a specific refinement of the capture 302 shown in fig. 3.
And step 703, ending the process.
As can be seen from the method shown in fig. 7, in this embodiment, the front-end device initiates a short connection request for the target data to the back-end server, starts an asynchronous task for acquiring the target data from the data source when the back-end server does not find the target data in the cache, and notifies the front-end device of a miss of the cache, so that the front-end device initiates a long connection request for the target data to the back-end server, and the back-end server can return the target data acquired from the data source by the asynchronous task to the front-end device through the long connection. It can be seen that, when the target data is not stored in the cache of the back-end server, the target data needs to be acquired from the data source, which is time-consuming, and for this reason, the front-end device initiates a long connection to the back-end server so that the front-end device only needs to initiate two connection requests to acquire complete target data, and does not need to initiate a short connection for multiple times to acquire complete data, thereby reducing the number of requests to the back-end server, reducing the network resource occupation, and reducing the load of the back-end server. In addition, in this embodiment, the background server may also adaptively return the target data to the front-end device in one or more batches according to the request service type of the front-end device.
An embodiment of the present invention further provides a backend data access apparatus, which is applied to a front-end device in a backend data access system, and as shown in fig. 8, the apparatus includes:
a short connection initiating unit 801, configured to initiate a short connection request for target data to a backend server in the system, so that the backend server receives the short connection request to establish a short connection with the front-end device, searches for the target data in a cache, starts an asynchronous task for obtaining the target data from a data source and returns a miss cache flag to the front-end device through the short connection if the target data is not found in the cache;
a long connection initiating unit 802, configured to initiate a long connection request for target data to the backend server after the receiving unit 803 receives a miss cache flag returned by the backend server through the short connection, so that the backend server receives the long connection request to establish a long connection with the front-end device, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device through the long connection;
a receiving unit 803, configured to receive a miss cache flag returned by the backend server through the short connection; and the target data is used for receiving the target data returned by the back-end server through the long connection.
In the arrangement shown in figure 8 of the drawings,
a short connection initiating unit, further configured to: and under the condition that the target data is found in the cache, returning the target data found in the cache to the front-end equipment through the short connection.
An embodiment of the present invention further provides another backend data access device, which is applied to a backend server in a backend data access system, and as shown in fig. 9, the device includes:
a short connection establishing unit 901, configured to receive a short connection request for target data initiated by a front-end device in the system to establish a short connection between the back-end server and the front-end device;
a long connection establishing unit 902, configured to receive a long connection request for target data initiated by the front-end device to establish a long connection between the back-end server and the front-end device;
a data processing unit 903, configured to search for target data in the cache after the short connection between the backend server and the front-end device is established by the short connection establishing unit, and if the target data is not found in the cache, start an asynchronous task that obtains the target data from a data source and return a miss cache flag to the front-end device through the short connection, so that the front-end device initiates a long connection request for the target data to the backend server; the long connection establishing unit is used for storing the target data acquired by the asynchronous task into a cache after establishing the long connection between the back-end server and the front-end equipment, and returning the target data acquired by the asynchronous task to the front-end equipment through the long connection.
In the arrangement shown in figure 9 of the drawings,
the data processing unit 903 is further configured to: and if the target data is found in the cache, returning the found target data to the front-end equipment through the short connection, so that the front-end equipment displays the received target data to the user.
In the arrangement shown in figure 9 of the drawings,
the data processing unit 903 is configured to further set a cache effective duration of the target data to be a first preset duration when the target data acquired by the asynchronous task is stored in the cache;
the data processing unit 903, after setting the cache effective duration of the target data to be a first preset duration, is further configured to: and periodically updating the effective cache duration of the target data until the effective cache duration of the target data is invalid.
In the arrangement shown in figure 9 of the drawings,
the data processing unit 903, which periodically updates the effective cache duration of the target data, includes:
and counting the accessed times of the target data in each updating period, determining a new effective cache time length according to the principle that the accessed times are in direct proportion to the effective cache time length when the updating period is ended, and updating the effective cache time length of the target data into the new effective cache time length.
In the arrangement shown in figure 9 of the drawings,
the data processing unit 903 is further configured to: and when the effective caching duration of the target data is invalid, deleting the target data from the cache.
In the arrangement shown in figure 9 of the drawings,
the data processing unit 903, when receiving the short connection request initiated by the front-end device, further accumulates the number of connection requests received within a second preset time period, and if the accumulated result exceeds a preset threshold, performs current limiting processing on the connection request initiated by the front-end device;
the data processing unit 903, when receiving the long connection request initiated by the front-end device, further accumulates the number of connection requests received within a second preset time period, and if the accumulated result exceeds a preset threshold, performs current limiting processing on the connection request initiated by the front-end device.
In the arrangement shown in figure 9 of the drawings,
the long connection request carries a request service type;
the data processing unit 903 is further configured to: pre-configuring a data back-transmission mode corresponding to each service type, wherein the data back-transmission mode comprises full back-transmission and incremental back-transmission;
the data processing unit 903, where the target data obtained through the asynchronous task is returned to the front-end device through the long connection, includes: determining a data returning mode corresponding to the request service type carried by the long connection request, and when the data returning mode is incremental returning, returning the target data acquired by the asynchronous task to the front-end equipment through the long connection at one time so that the front-end equipment can display the received target data to a user; when the data returning mode is incremental returning, returning the target data acquired by the asynchronous task to the front-end equipment in batches through the long connection, so that the front-end equipment displays the target data received in each batch to a user in an incremental mode.
An embodiment of the present invention further provides an electronic device, as shown in fig. 10, where the electronic device includes: at least one processor 1001, and a memory 1002 connected to the at least one processor 1001 through a bus; the memory 1002 stores one or more computer programs executable by the at least one processor 1001; the at least one processor 1001, when executing the one or more computer programs, implements the steps in the back-end data access method illustrated in any of the flowcharts of fig. 2-7.
Embodiments of the present invention also provide a computer-readable storage medium, which stores one or more computer programs that, when executed by a processor, implement the steps in the backend data access method shown in any of the flowcharts in fig. 2 to 7.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (19)

1. A back-end data access system, the system comprising: the system comprises front-end equipment, a back-end server, a cache and a data source, wherein the cache and the data source serve the back-end server;
the front-end equipment is used for initiating a short connection request aiming at target data to a back-end server; the cache miss processing system is used for receiving a miss cache mark returned by the back-end server through short connection with the front-end equipment and initiating a long connection request aiming at target data to the back-end server; the system comprises a front-end device and a back-end server, wherein the front-end device is used for receiving target data returned by the back-end server through long connection with the front-end device;
the back-end server is used for receiving a short link request aiming at target data initiated by the front-end equipment to establish the short connection with the front-end equipment, searching the target data in a cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short connection if the target data is not searched in the cache; the long connection request aiming at the target data and initiated by the front-end equipment is received to establish the long connection with the front-end equipment, the target data acquired through the asynchronous task is stored in a cache, and the target data acquired through the asynchronous task is returned to the front-end equipment through the long connection.
2. The system of claim 1,
the back-end server is used for returning the searched target data to the front-end equipment through the short connection if the target data is searched in the cache;
and the front-end equipment is used for receiving the target data returned by the back-end server through the short connection and displaying the received target data to the user.
3. The system of claim 2,
the back-end server further sets the effective caching duration of the target data to be a first preset duration when the target data acquired through the asynchronous task is stored in a cache;
the back-end server, after setting the cache validity duration of the target data to a first preset duration, further includes: and periodically updating the effective cache duration of the target data until the effective cache duration of the target data is invalid.
4. The system of claim 3,
the back-end server periodically updates the effective cache duration of the target data, and comprises the following steps:
and counting the accessed times of the target data in each updating period, determining a new effective cache time length according to the principle that the accessed times are in direct proportion to the effective cache time length when the updating period is ended, and updating the effective cache time length of the target data into the new effective cache time length.
5. The system of claim 3,
the back-end server is further configured to: and when the effective caching duration of the target data is invalid, deleting the target data from the cache.
6. The system of claim 1,
the back-end server further accumulates the connection request times received within the latest second preset time when receiving the short connection request initiated by the front-end equipment, and performs current limiting processing on the connection request initiated by the front-end equipment if the accumulated result exceeds a preset threshold;
and the back-end server further accumulates the connection request times received in the second preset time when receiving the long connection request initiated by the front-end equipment, and performs current limiting processing on the connection request initiated by the front-end equipment if the accumulated result exceeds a preset threshold.
7. The system of claim 1,
the long connection request carries a request service type;
the back-end server is used for pre-configuring a data back-transmission mode corresponding to each service type, wherein the data back-transmission mode comprises full back-transmission and incremental back-transmission;
the back-end server returns the target data acquired by the asynchronous task to the front-end equipment through the long connection, and the method comprises the following steps: determining a data returning mode corresponding to the request service type carried by the long connection request, returning the target data acquired by the asynchronous task to the front-end equipment through the long connection at one time when the data returning mode is incremental returning, and returning the target data acquired by the asynchronous task to the front-end equipment through the long connection in batches when the data returning mode is incremental returning;
the front-end equipment is used for receiving the target data returned by the back-end server through the long connection at one time and displaying the received target data to the user; and the system is used for receiving the target data returned by the back-end server in batches through the long connection and displaying the target data received in each batch to the user in an incremental manner.
8. A back-end data access method is applied to front-end equipment in a back-end data access system, and is characterized in that the method comprises the following steps:
initiating a short connection request aiming at target data to a back-end server in the system so that the back-end server receives the short connection request to establish short connection with the front-end equipment, searching the target data in a cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short connection under the condition that the target data is not searched in the cache;
receiving a miss cache mark returned by the back-end server through the short connection, initiating a long connection request aiming at target data to the back-end server so that the back-end server receives the long connection request to establish long connection with the front-end equipment, storing the target data acquired through the asynchronous task into a cache, and returning the target data acquired through the asynchronous task to the front-end equipment through the long connection;
and receiving target data returned by the back-end server through the long connection.
9. A back-end data access method is applied to a back-end server in a back-end data access system, and is characterized in that the method comprises the following steps:
receiving a short link request aiming at target data initiated by front-end equipment in the system to establish a short link with the front-end equipment, searching the target data in a cache, if the target data is not searched in the cache, starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short link so that the front-end equipment initiates a long link request aiming at the target data to a back-end server;
receiving a long link request aiming at target data initiated by the front-end equipment to establish a long connection with the front-end equipment, storing the target data acquired through the asynchronous task into a cache, and returning the target data acquired through the asynchronous task to the front-end equipment through the long connection.
10. The method of claim 9,
the method further comprises the following steps: and if the target data is found in the cache, returning the found target data to the front-end equipment through the short connection, so that the front-end equipment displays the received target data to the user.
11. The method of claim 10,
when the target data acquired through the asynchronous task is stored in a cache, further setting the effective cache duration of the target data as a first preset duration;
after setting the cache validity duration of the target data to be a first preset duration, the method further includes: and periodically updating the effective cache duration of the target data until the effective cache duration of the target data is invalid.
12. The method of claim 11,
the method for regularly updating the effective caching duration of the target data comprises the following steps:
and counting the accessed times of the target data in each updating period, determining a new effective cache time length according to the principle that the accessed times are in direct proportion to the effective cache time length when the updating period is ended, and updating the effective cache time length of the target data into the new effective cache time length.
13. The method of claim 11,
the method further comprises the following steps: and when the effective caching duration of the target data is invalid, deleting the target data from the cache.
14. The method of claim 9,
when a short connection request initiated by the front-end equipment is received, further accumulating the number of connection requests received within a second preset time recently, and if the accumulated result exceeds a preset threshold, performing current limiting processing on the connection request initiated by the front-end equipment;
and when a long connection request initiated by the front-end equipment is received, further accumulating the number of times of the connection request received in the second preset time period, and if the accumulated result exceeds a preset threshold, performing current limiting processing on the connection request initiated by the front-end equipment.
15. The method of claim 9,
the long connection request carries a request service type;
the method further comprises the following steps: pre-configuring a data back-transmission mode corresponding to each service type, wherein the data back-transmission mode comprises full back-transmission and incremental back-transmission;
the target data acquired by the asynchronous task is returned to the front-end equipment through the long connection, and the method comprises the following steps: determining a data returning mode corresponding to the request service type carried by the long connection request, and when the data returning mode is incremental returning, returning the target data acquired by the asynchronous task to the front-end equipment through the long connection at one time so that the front-end equipment can display the received target data to a user; when the data returning mode is incremental returning, returning the target data acquired by the asynchronous task to the front-end equipment in batches through the long connection, so that the front-end equipment displays the target data received in each batch to a user in an incremental mode.
16. A backend data access apparatus, applied to a front-end device in a backend data access system, the apparatus comprising:
a short connection initiating unit, configured to initiate a short connection request for target data to a back-end server in the system, so that the back-end server receives the short connection request to establish a short connection with the front-end device, searches for the target data in a cache, starts an asynchronous task for obtaining the target data from a data source and returns a miss cache flag to the front-end device through the short connection if the target data is not found in the cache;
a long connection initiating unit, configured to initiate a long connection request for target data to a backend server after the receiving unit receives a miss cache flag returned by the backend server through the short connection, so that the backend server receives the long connection request to establish a long connection with the front-end device, store the target data obtained through the asynchronous task in a cache, and return the target data obtained through the asynchronous task to the front-end device through the long connection;
a receiving unit, configured to receive a miss cache flag returned by the backend server through the short connection; and the target data is used for receiving the target data returned by the back-end server through the long connection.
17. A backend data access apparatus, applied to a backend server in a backend data access system, the apparatus comprising:
the short connection establishing unit is used for receiving a short connection request aiming at target data and initiated by front-end equipment in the system so as to establish short connection between the back-end server and the front-end equipment;
a long connection establishing unit, configured to receive a long connection request for target data initiated by the front-end device to establish a long connection between the back-end server and the front-end device;
the data processing unit is used for searching target data in the cache after the short connection establishing unit establishes the short connection between the back-end server and the front-end equipment, and starting an asynchronous task for acquiring the target data from a data source and returning a miss cache mark to the front-end equipment through the short connection if the target data is not searched in the cache so as to enable the front-end equipment to initiate a long connection request aiming at the target data to the back-end server; the long connection establishing unit is used for storing the target data acquired by the asynchronous task into a cache after establishing the long connection between the back-end server and the front-end equipment, and returning the target data acquired by the asynchronous task to the front-end equipment through the long connection.
18. An electronic device, comprising: the system comprises at least one processor and a memory connected with the at least one processor through a bus; the memory stores one or more computer programs executable by the at least one processor; characterized in that the at least one processor, when executing the one or more computer programs, implements the steps in the method of claim 8 or implements the steps in the method of any one of claims 9-15.
19. A computer readable storage medium, characterized in that the computer readable storage medium stores one or more computer programs which, when executed by a processor, implement the steps in the method of claim 8 or implement the steps in the method of any of claims 9-15.
CN202110059589.2A 2021-01-18 2021-01-18 Back-end data access system, method, device and storage medium Pending CN113810455A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110059589.2A CN113810455A (en) 2021-01-18 2021-01-18 Back-end data access system, method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110059589.2A CN113810455A (en) 2021-01-18 2021-01-18 Back-end data access system, method, device and storage medium

Publications (1)

Publication Number Publication Date
CN113810455A true CN113810455A (en) 2021-12-17

Family

ID=78892881

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110059589.2A Pending CN113810455A (en) 2021-01-18 2021-01-18 Back-end data access system, method, device and storage medium

Country Status (1)

Country Link
CN (1) CN113810455A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115080625A (en) * 2022-07-21 2022-09-20 成都薯片科技有限公司 Caching method, device and equipment based on Spring Cache framework and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200045546A1 (en) * 2017-11-03 2020-02-06 Huawei Technologies Co., Ltd. Internet of Things Communication Method, Apparatus, and System
CN110795457A (en) * 2019-09-24 2020-02-14 苏宁云计算有限公司 Data caching processing method and device, computer equipment and storage medium
CN111125175A (en) * 2019-12-20 2020-05-08 北京奇艺世纪科技有限公司 Service data query method and device, storage medium and electronic device
CN111225010A (en) * 2018-11-27 2020-06-02 北京京东尚科信息技术有限公司 Data processing method, data processing system and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200045546A1 (en) * 2017-11-03 2020-02-06 Huawei Technologies Co., Ltd. Internet of Things Communication Method, Apparatus, and System
CN111225010A (en) * 2018-11-27 2020-06-02 北京京东尚科信息技术有限公司 Data processing method, data processing system and device
CN110795457A (en) * 2019-09-24 2020-02-14 苏宁云计算有限公司 Data caching processing method and device, computer equipment and storage medium
CN111125175A (en) * 2019-12-20 2020-05-08 北京奇艺世纪科技有限公司 Service data query method and device, storage medium and electronic device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115080625A (en) * 2022-07-21 2022-09-20 成都薯片科技有限公司 Caching method, device and equipment based on Spring Cache framework and storage medium

Similar Documents

Publication Publication Date Title
KR100791628B1 (en) Method for active controlling cache in mobile network system, Recording medium and System thereof
CN102984286B (en) Method and device and system of domain name server (DNS) for buffering updating
CN111221469B (en) Method, device and system for synchronizing cache data
CN107332908B (en) Data transmission method and system
CN108984553B (en) Caching method and device
CN108153783B (en) Data caching method and device
CN109167840B (en) Task pushing method, node autonomous server and edge cache server
CN111177601B (en) Page rendering processing method, device, equipment and readable storage medium
CN107301215B (en) Search result caching method and device and search method and device
CN107665235B (en) Cache processing method and device, computer equipment and storage medium
CN112035766A (en) Webpage access method and device, storage medium and electronic equipment
EP3842955A2 (en) File directory traversal method, apparatus, device, and medium
CN113810455A (en) Back-end data access system, method, device and storage medium
CN110659971B (en) Transaction data processing method and device
CN107992489B (en) Data processing method and server
CN113656178B (en) Data processing method, device, equipment and readable storage medium
CN110891090B (en) Request method, device, server, system and storage medium
CN103049445A (en) Method and device for inquiring pictorial information and picture state server
US9158732B2 (en) Distributed cache system for delivering contents to display apparatus
WO2017097027A1 (en) Data processing method, apparatus and system based on internet content adaptation protocol
CN113596177A (en) Method and device for analyzing IP address of intelligent household equipment
CN112887447A (en) Domain name query method and device and domain name server
KR102093166B1 (en) A method for reducing connection time to website and an apparatus for the method
CN113746868B (en) Method, device, equipment and medium for optimizing server performance
CN107229699B (en) System and method for improving search experience of mobile terminal user

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination