CN113806655A - Data processing method and device, electronic equipment and storage medium - Google Patents

Data processing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113806655A
CN113806655A CN202111129273.2A CN202111129273A CN113806655A CN 113806655 A CN113806655 A CN 113806655A CN 202111129273 A CN202111129273 A CN 202111129273A CN 113806655 A CN113806655 A CN 113806655A
Authority
CN
China
Prior art keywords
data
request
indexdb
cache
cache data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111129273.2A
Other languages
Chinese (zh)
Inventor
张欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202111129273.2A priority Critical patent/CN113806655A/en
Publication of CN113806655A publication Critical patent/CN113806655A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/957Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/252Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The application discloses a data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first request of a front-end page; the first request is used for requesting to acquire first data of a target server; judging whether first cache data which are not invalidated exist in an index database (IndexDB); the first cache data corresponds to the first data; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.

Description

Data processing method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.
Background
In the related art, the browser may employ an Application Cache (Application Cache) and a Cache mechanism of LocalStorage to Cache the front-end data. The Application Cache is a Cache mechanism provided by the HyperText Markup Language 5(HTML5, Hypertext Markup Language 5) technology at the earliest, and can set resources needing to be cached by a browser by configuring a manifest file, so that a network (Web) Application program can run offline; in other words, in the case that the user is in an offline state (i.e., the client that installed the browser does not access the network), even if the user clicks the refresh button of the browser, the Web application can be loaded and run normally. Compared with the Application Cache, the LocalStorage is a new characteristic which is provided by the HTML5 technology and is used for local storage, a Cache mechanism of the LocalStorage is adopted, the browser can create a long-term effective Cache file for each website, and the Cache file can be stored in the local of the client in a text form, so that the problems that the browser Cookie occupies insufficient bandwidth and storage space can be solved.
However, in the related art, the caching mechanism of the browser needs to be optimized.
Disclosure of Invention
In order to solve the related technical problems, embodiments of the present application provide a data processing method, an apparatus, an electronic device, and a storage medium.
The technical scheme of the embodiment of the application is realized as follows:
the embodiment of the application provides a data processing method, which comprises the following steps:
acquiring a first request of a front-end page; the first request is used for requesting to acquire first data of a target server;
judging whether first cache data which are not invalidated exist in an index database (IndexDB); the first cache data corresponds to the first data;
returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB;
and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
In the foregoing solution, the determining whether there is any inefficacy first cache data in IndexDB includes:
judging whether the first cache data exists in the IndexDB by using a first identifier corresponding to the first data to obtain a first judgment result;
under the condition that the first judgment result represents that the first cache data exists in the IndexDB, judging whether the first cache data is invalid or not by using at least one second identifier corresponding to the first request to obtain a second judgment result; the second identification represents a data version corresponding to the first request.
In the foregoing solution, the determining whether the first cache data is invalid by using at least one second identifier corresponding to the first request includes:
obtaining at least one third identifier corresponding to the first cache data from the IndexDB; the third identification represents a data version corresponding to the first cache data;
comparing the at least one second identifier with the at least one third identifier;
determining that the first cached data is not stale if the at least one second identifier is consistent with the at least one third identifier;
determining that the first cached data has failed if the at least one second identifier is inconsistent with the at least one third identifier.
In the foregoing solution, the acquiring the first data from the target server includes:
injecting first configuration information into the first request through a first function to obtain a second request; the first configuration information comprises at least one second identifier; the second identification represents a data version corresponding to the first request; the second request is used for requesting to acquire the first data;
sending the second request to the target server; and receiving first data returned by the target server based on the second request.
In the foregoing solution, when the first data is returned to the front-end page, the method further includes:
generating first cache data by using the first data when the first cache data does not exist in the IndexDB;
or,
updating the first cache data with the first data if the first cache data has failed.
In the foregoing solution, the generating first cache data by using the first data includes:
performing first processing on the first data based on a first preset rule so that the first processed first data meet a first preset condition;
and generating first cache data by using the first processed first data.
In the above scheme, the front page is in a first working mode; the method further comprises the following steps:
judging whether the current network environment meets a second preset condition or not, and judging whether the front-end page meets a third preset condition or not;
under the condition that the current network environment does not meet the second preset condition and/or under the condition that the front-end page does not meet the third preset condition, controlling the front-end page to enter the first working mode;
controlling the front-end page to enter a second working mode under the condition that the current network environment meets the second preset condition and the front-end page meets the third preset condition; wherein,
and under the condition that the front-end page is in a second working mode, acquiring all data requested by the front-end page from the IndexDB.
An embodiment of the present application further provides a data processing apparatus, including:
an acquisition unit, configured to acquire a first request of a front-end page; the first request is used for requesting to acquire first data of a target server;
the processing unit is used for judging whether inefficacy first cache data exist in the IndexDB or not; the first cache data corresponds to the first data; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
An embodiment of the present application further provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of any of the above methods when running the computer program.
Embodiments of the present application also provide a storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of any one of the above methods.
The data processing method, the data processing device, the electronic equipment and the storage medium provided by the embodiment of the application acquire a first request of a front-end page; the first request is used for requesting to acquire first data of a target server; judging whether inefficacy first cache data exists in the IndexDB or not; the first cache data corresponds to the first data; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page. According to the scheme of the embodiment of the application, the IndexDB is used for storing the cache data of the browser, when the browser requests the server for the data, the corresponding cache data are directly returned to the front-end page under the condition that the corresponding cache data are not invalid, and the data do not need to be obtained from the server, so that the number of requests sent to the server by the browser can be reduced, the dependence of front-end application (namely Web application program) on the server can be reduced, the loading time of the front-end page can be reduced, and the use efficiency of the browser can be improved.
Drawings
FIG. 1 is a schematic flow chart illustrating a data processing method according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a data processing flow of a data processing system according to an embodiment of the present application;
FIG. 3 is a schematic flow chart illustrating a process for intercepting a Hypertext Transfer Protocol (HTTP) request by a data processing system according to an embodiment of the present application;
FIG. 4 is a diagram illustrating a persistent data structure according to an embodiment of the present application;
FIG. 5 is a schematic diagram illustrating a flow of persistent data generation according to an embodiment of the present application;
fig. 6 is a schematic diagram of a workflow of providing a data caching service by IndexDB according to an embodiment of the present application;
FIG. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples.
In the related art, although the browser may employ the caching mechanisms of the Application Cache and the LocalStorage to Cache the front-end data, both caching mechanisms have defects. Specifically, the Application Programming Interface (API) related to the Application Cache is unreasonable in design, limited in supported functions and not timely updated, and does not support functions such as performing data refinement control by using a JavaScript technology (i.e., classifying data so that the granularity of the data meets a specific condition); moreover, when these APIs are used, some preset rules need to be strictly followed, and the flexibility and usability of the APIs are poor and have uncertainty.
For the caching mechanism of LocalStorage, the technology only supports a single data type, and when the browser needs to use other types of data, conversion of the data type is needed. For example, when the data type supported by LocalStorage is limited to a String (String) type in advance, if the browser needs to use JavaScript Object Notation (JSON), the String type data needs to be converted into JSON type data. In addition, the LocalStorage does not support data reading in the privacy mode of the browser, that is, the browser cannot load cache data in the privacy mode.
Based on this, in various embodiments of the present application, the IndexDB is used to store the cache data of the browser, and when the browser requests data from the server, the corresponding cache data is directly returned to the front-end page without being invalidated, and data does not need to be acquired from the server, so that the number of requests sent by the browser to the server can be reduced, thereby reducing the dependency of the front-end application (i.e., the Web application) on the server, reducing the loading time of the front-end page, and further improving the use efficiency of the browser. Meanwhile, compared with a Cache mechanism of the Application Cache, the relevant API of the IndexDB supports more functions, has better flexibility and usability and weaker uncertainty. Compared with the caching mechanism of LocalStorage, IndexDB has capacity advantage, so that the cache data in IndexDB can be stored for a longer time, in other words, the persistence of the cache data can be realized by utilizing IndexDB; moreover, the IndexDB can store diversified data, namely, can simultaneously support various types of data storage, and the browser does not need to convert data types when loading data, so that the loading time of a front-end page can be further reduced; in addition, IndexDB has no limitation on the usage condition, and the cached data in IndexDB can be read even if the browser is in the privacy mode.
The embodiment of the application provides a data processing method, which is applied to electronic equipment, such as a client provided with a browser and used for a mobile phone, a computer and the like; as shown in fig. 1, the method includes:
step 101: acquiring a first request of a front-end page;
here, the first request is used for requesting to acquire first data of a target server;
step 102: judging whether inefficacy first cache data exists in the IndexDB or not;
here, the first cache data corresponds to the first data, in other words, the first cache data is generated using the first data;
step 103: returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB;
step 104: and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
In practical application, the data processing method provided by the embodiment of the present application can be implemented by an application program developed by using a JavaScript technology And an Asynchronous JavaScript And XML (AJAX, Asynchronous JavaScript And XML) technology.
In practical applications, the data type of the first cache data may be set according to requirements, in other words, the data type supported by the IndexDB may be set according to requirements, such as JSON, Binary Large Object (BLOB), and the like.
In step 101, during actual application, the first request is used to request to obtain first data of a target server, which may be understood as that the first request is information sent to the target server; accordingly, the obtaining of the first request may be understood as intercepting the first request before the first request is issued. Here, the manner of intercepting the first request may be set based on a communication manner between the front-end page and the target server (such as HTTP communication).
In practical application, the first function may be used to intercept the first request, where the first function may also be referred to as a request interceptor, an HTTP interceptor, and the like.
In step 102, during actual application, the data stored in the target server may be updated, in other words, the data stored in the target server may have a version change, and once the version change occurs to the data, the corresponding cached data in the IndexDB may be invalidated. Therefore, whether inefficacy first cache data exists in the IndexDB may be determined by using the data version corresponding to the data requested by the first request, that is, the data version corresponding to the first data.
Based on this, in an embodiment, the determining whether there is the first cache data that is not invalidated in the IndexDB may include:
judging whether the first cache data exists in the IndexDB by using a first identifier corresponding to the first data to obtain a first judgment result;
under the condition that the first judgment result represents that the first cache data exists in the IndexDB, judging whether the first cache data is invalid or not by using at least one second identifier corresponding to the first request to obtain a second judgment result; the second identification represents a data version corresponding to the first request.
In actual application, the first identifier may be a flag, such as ETagID, generated by the target server for the first data; the ETagiD refers to parameters automatically carried in the interaction process of the browser and the target server, and comprises ETag in a Response Header (Response Header) and If-None-Match in a Request Header (Request Header). Here, the manner of obtaining the first identifier corresponding to the first data may be determined according to a service requirement, and the embodiment of the present application is not limited by this.
In practical application, the determining, by using the first identifier corresponding to the first data, whether the first cached data exists in the IndexDB may be understood as: judging whether the first identifier exists in the IndexDB, and if the first identifier exists in the IndexDB, determining that the first judgment result represents that the first cache data exists in the IndexDB; if the first identifier does not exist in the IndexDB, it may be determined that the first determination result indicates that the first cache data does not exist in the IndexDB, so that it may be determined that the first cache data that has not failed does not exist in the IndexDB.
In actual application, the second identifier represents a data version corresponding to the first request, and may be understood as the second identifier represents a version of a data packet requested by the first request, that is, a data version corresponding to the first data. Here, the second identifier may be a version number, a request timestamp, cache expiration time, and the like, and the number of the second identifiers and a manner of determining the second identifiers may be determined according to a service requirement, which is not limited in comparison in the embodiment of the present application.
In an embodiment, the determining, by using at least one second identifier corresponding to the first request, whether the first cache data is invalid may include:
obtaining at least one third identifier corresponding to the first cache data from the IndexDB; the third identification represents a data version corresponding to the first cache data;
comparing the at least one second identifier with the at least one third identifier;
determining that the first cached data is not stale if the at least one second identifier is consistent with the at least one third identifier;
determining that the first cached data has failed if the at least one second identifier is inconsistent with the at least one third identifier.
Here, the number of the second identifiers is the same as the number of the third identifiers, and the specific number may be set according to a service requirement.
In actual application, the first cache data is generated by using the first data, that is, before the first request, the front-end page requests the target server to acquire the first data, generates the first cache data by using the first data, and generates at least one third identifier by using a data version corresponding to the first data.
In step 104, in actual application, after intercepting the first request by using the first function, the at least one second identifier may be injected into the first request, and the processed request is sent to the target server to obtain the first data.
Based on this, in an embodiment, the obtaining the first data from the target server may include:
injecting first configuration information into the first request through a first function to obtain a second request; the first configuration information comprises at least one second identifier; the second identification represents a data version corresponding to the first request; the second request is used for requesting to acquire the first data;
sending the second request to the target server; and receiving first data returned by the target server based on the second request.
Here, the first configuration information may further include the first identifier.
In practical application, the first function may specifically inject the first configuration information into the Request Header of the first Request.
In actual application, after the first data is acquired from the target server, the first data may be cached or updated to the IndexDB in order to provide data caching support for a next request.
Based on this, in an embodiment, when the first data is returned to the front-end page, the method may further include:
generating first cache data by using the first data when the first cache data does not exist in the IndexDB;
or,
updating the first cache data with the first data if the first cache data has failed.
In actual application, the first cache data is generated by using the first data, or the first cache data is updated by using the first data, so that the IndexDB can provide data cache support for the next request, and thus the efficiency of using the browser by the user, that is, the friendliness of using the browser by the user, can be improved.
In practical applications, in order to enable the IndexDB to provide better data caching support, when the first cached data is generated by using the first data, specific processing, such as filtering, classifying, and the like, may be performed on the first data.
Based on this, in an embodiment, the generating the first cache data by using the first data may include:
performing first processing on the first data based on a first preset rule so that the first processed first data meet a first preset condition;
and generating first cache data by using the first processed first data.
In practical application, the first preset rule and the first preset condition may be set according to a service requirement, for example, the first processing may be filtering processing, data classification processing, and the like, and the first preset condition may be a relevant condition of data granularity, which is not limited in this embodiment of the present application.
In actual application, when the first cache data is generated, second configuration information corresponding to the first cache data may be generated and stored in the IndexDB, where the second configuration information may include the first identifier and the at least one third identifier.
In practical application, the browser may include multiple working modes, and in a case of a poor network environment (for example, the network speed is lower than a first threshold, which may be set according to a service requirement) or a network outage, the browser may directly read the required data from the IndexDB without interacting with the target server.
Based on this, in one embodiment, the front page may be in a first mode of operation; the method may further comprise:
judging whether the current network environment meets a second preset condition or not, and judging whether the front-end page meets a third preset condition or not;
under the condition that the current network environment does not meet the second preset condition and/or under the condition that the front-end page does not meet the third preset condition, controlling the front-end page to enter the first working mode;
controlling the front-end page to enter a second working mode under the condition that the current network environment meets the second preset condition and the front-end page meets the third preset condition; wherein,
and under the condition that the front-end page is in a second working mode, acquiring all data requested by the front-end page from the IndexDB.
In actual application, under the condition that the front-end page is in a first working mode, intercepting the first request by using the first function, and judging whether inefficacy first cache data exists in the IndexDB or not; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page. Here, it is understood that the first operation mode is an operation mode in which the front-end page negotiates (i.e., presence communication with) the target server for caching.
In practical application, the second preset condition and the third preset condition may be set according to a service requirement. Illustratively, the second preset condition may indicate that the corresponding client does not access the network, or indicate that the current network speed is lower than the first threshold; the third preset condition may indicate that the IndexDB cache stores all data required for loading the front-end page.
The data processing method provided by the embodiment of the application obtains a first request of a front-end page; the first request is used for requesting to acquire first data of a target server; judging whether inefficacy first cache data exists in the IndexDB or not; the first cache data corresponds to the first data; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page. According to the scheme of the embodiment of the application, the IndexDB is used for storing the cache data of the browser, when the browser requests the server for the data, the corresponding cache data are directly returned to the front-end page under the condition that the corresponding cache data are not invalid, and the data do not need to be obtained from the server, so that the number of requests sent to the server by the browser can be reduced, the dependence of front-end application (namely Web application program) on the server can be reduced, the loading time of the front-end page can be reduced, and the use efficiency of the browser can be improved. Meanwhile, compared with a Cache mechanism of the Application Cache, the relevant API of the IndexDB supports more functions, has better flexibility and usability and weaker uncertainty. Compared with the caching mechanism of LocalStorage, IndexDB has capacity advantage, so that the cache data in IndexDB can be stored for a longer time, in other words, the persistence of the cache data can be realized by utilizing IndexDB; moreover, the IndexDB can store diversified data, namely, can simultaneously support various types of data storage, and the browser does not need to convert data types when loading data, so that the loading time of a front-end page can be further reduced; in addition, IndexDB has no limitation on the usage condition, and the cached data in IndexDB can be read even if the browser is in the privacy mode.
The present application will be described in further detail with reference to the following application examples.
An embodiment of the present application provides a data processing system, as shown in fig. 2, the system includes: a request interceptor (i.e., the first function described above), a cache adapter, and a response interceptor; the cache adapter comprises an enhanced adapter. When a user browses a front-end page, the front-end page initiates an HTTP request (i.e., the first request), and the browser executes (i.e., calls) a request interceptor to intercept the HTTP request and executes a dispatch request process (english may be denoted as displatcequest) and a conversion request data process (english may be denoted as transformData) on the intercepted HTTP request. Then, the browser processes the HTTP request by using (i.e., calling) the cache adapter, and determines whether cache data (i.e., the first cache data) corresponding to the HTTP request exists in IndexDB; if the corresponding cache data exists in the IndexDB, judging whether the corresponding cache data is expired (i.e. judging whether the corresponding cache data is invalid); if the corresponding cache data is not expired, returning the corresponding cache data as a response object of the HTTP request, and executing a subsequent process, namely, the browser executes a response interceptor, and the response interceptor converts the response data (namely, the response object). If cache data corresponding to the HTTP request does not exist in the IndexDB, the browser sends the HTTP request to a backend server (i.e., the target server) by using the enhanced adapter, saves a response object returned by the browser based on the request, and returns the saved response object to the front-end page to execute a subsequent process. In addition, when the corresponding cache data is expired, the browser also uses the enhanced adapter to send an HTTP request to the backend server, and saves a response object returned by the browser based on the request (i.e., updates the corresponding cache data in the IndexDB by using the response object), and simultaneously returns the saved response object to the front-end page to execute the subsequent process.
In the present application embodiment, "enhancing" of the enhanced adapter may be understood as fitting the data storage and usage functions of IndexDB to the underlying logic of the application. In the data processing system provided by the embodiment of the application, the IndexDB is used for storing the cache data of the browser, and because the IndexDB has a capacity advantage, the cache data in the IndexDB can be stored for a longer time, in other words, the IndexDB can be used for realizing the persistence of the cache data, that is, the IndexDB is used for providing reliable persistent cache data for the browser, so that the dependence of the front-end application on the server is reduced, and the use efficiency of the Web front-end is improved. In addition, when the corresponding cache data is expired, that is, when the version of the cache data is changed, the data synchronization function can be improved, and the response object returned by the server is asynchronously synchronized to the IndexDB, in other words, while providing the browser with normal data display, the same data is stored (i.e., updated) to the IndexDB in a multithreading mode, so as to provide the cache data for subsequent requests, thereby improving the browsing experience of the user.
The functions of the respective modules included in the data processing system of the present application embodiment are explained in detail below.
First, the data processing system of the present application embodiment provides an interception service of an HTTP request.
Specifically, in the embodiment of the present application, in order to achieve the effect of front-end data persistence, the HTTP request needs to be uniformly processed before being issued. In the related art, a uniform request (request) function is usually packaged to uniformly add information such as a token (english may be represented as token) for each HTTP request, but in a later stage, if cache time needs to be set for some GET (GET) requests or call frequency of some requests needs to be controlled, the request function needs to be continuously modified to expand corresponding functions, and if uniform processing is performed on responses, the request function becomes increasingly bulky and difficult to maintain. For this reason, a new solution is needed to interrupt HTTP, and since the HTTP protocol is based on request and response mode communication, the HTTP interrupt can be implemented by an HTTP interceptor. The interceptors may include a request interceptor and a response interceptor for processing the HTTP request and the HTTP response, respectively. Wherein, the request interceptor is used for: certain operations are uniformly performed before the HTTP request is sent, such as adding a token field in the request header. The response interceptor functions as: and after receiving the server response, uniformly executing some specific operations, such as finding that the response status code is 401, and automatically jumping to the login page.
In this application embodiment, as shown in fig. 3, the HTTP request mainly includes the following flows:
1) the processing request configures the object. Specifically, if the front end initiates an HTTP request, the browser starts an HTTP interception service, that is, intercepts the current HTTP request by using a request interceptor, and injects configuration information (i.e., the first configuration information) into a request header of the request; the configuration information may include a request version number, a request timestamp, a cache expiration time, an EtagID, and the like. Here, the configuration information may be understood as an identification set, and EtagID corresponds to the first identification; the request version number, the request timestamp and the cache expiration time are equivalent to the second identifier or the third identifier; the request timestamp and cache expiration time are used only to reflect the data version and are not used to calculate whether the cache is expired. In practical application, the type of the intercepted HTTP request may be set according to service requirements, such as GET, POST, PUT, DELETE, and the like.
2) An HTTP request is initiated. The flow may include a dispatch request process and a convert request data process. In addition, if the cache adapter determines that the corresponding cache data is in a normal working state by comparing the configuration information injected by the HTTP request (i.e., the first configuration information) and the configuration information corresponding to the corresponding cache data in the IndexDB (i.e., the second configuration information), i.e., the corresponding cache data is not invalidated, the corresponding cache data in the IndexDB is read instead of requesting the corresponding data from the server.
3) The responsive data object is processed. The process may include executing a response interceptor and converting response data by the response interceptor, and specifically, performing secondary processing (such as filtering, classifying, and the like) on corresponding cache data or data returned by the server, so that the user performs a unified operation on the data.
Second, the data processing system of the present application embodiment provides a persistent data generation service.
In particular, the persisted data generation service does not depart from the implementation of a cache, which in computing is a high-speed data storage layer in which subsets of data are stored, and is typically a transitory store, such that later when the data is requested again, it is faster than the main storage location from which the data is accessed. Data previously retrieved or calculated can be efficiently reused by caching. In the present application embodiment, the persistent data generation service is provided through the following APIs:
1) get (key) used for obtaining a value corresponding to a specified key value (key) from cache data in the IndexDB;
2) delete (key) for deleting the value corresponding to the specified key from the cache data in IndexDB;
3) clear () for emptying cached data;
4) set (key, value, maxAge) for storing key-value pairs, while supporting setting of the maximum time (i.e. the most significant duration) maxAge of the cache, which may be in milliseconds.
Illustratively, in the embodiment of the present application, as shown in fig. 5, the generation flow of the persistent data may include the following steps:
1) creating or opening a database in IndexDB;
2) creating an object store (objectStore) in the database;
3) setting a data retrieval type of a response interceptor to be BLOB;
4) initializing a database transaction;
5) the BLOB data is cached in the database.
Thirdly, the data processing system of the embodiment of the present application provides a data caching service based on IndexDB, and the data caching service includes two operation modes: a negotiated cache mode of operation (i.e., the first mode of operation described above) and a forced cache mode of operation (i.e., the second mode of operation described above).
As shown in fig. 6, in the negotiation cache working mode, the cache adapter determines whether cache data corresponding to the current HTTP request exists in the IndexDB; if the data does not exist, it is described that the browser first implements the cache service for the data requested by the HTTP request, and it is necessary to obtain the response data from the server through the enhanced adapter, perform refinement processing on the response data (that is, classify the data so that the granularity of the data meets a specific condition), and write the data after the refinement processing into the IndexDB to provide the cache data for the next request. If the indexDB has cache data corresponding to the current HTTP request, generating a data synchronization manager through an enhanced adapter, wherein the data synchronization manager is used for registering cache change (namely cache update) and performing comparison management on configuration information through a comparison management unit of the enhanced adapter, namely comparing whether the configuration information (namely the first configuration information) of the request is consistent with the configuration information (namely the second configuration information) corresponding to the cache data returned by the previous request in the indexDB; if the cache is consistent, the cache does not need to be updated; if the comparison result is inconsistent with the preset comparison result, the corresponding cache data needs to be updated according to the updating logic preset by the comparison management unit, namely, the response data is obtained from the server through the enhanced adapter, the asynchronous synchronous task is triggered, and the corresponding cache data in the IndexDB is updated by using the response data while the response data is presented to the user. Here, it is understood that the enhanced adapter includes: the system comprises a comparison management unit, a data synchronization manager and a data fine processing unit.
In the case of a poor client network environment (e.g., a network outage), the data processing system enters a forced cache mode of operation. And under the forced cache working mode, providing all request data of the front end by the IndexDB cache service, and reading refined cache data in the IndexDB cache service according to the configuration information corresponding to each request.
The scheme provided by the application embodiment has the following advantages:
1) the configuration information is injected into the HTTP request based on the HTTP interceptor, the persistent cache of the front-end data is realized based on the enhanced adapter and the IndexDB, and when the cache data in the IndexDB is not invalid, the cached data can be directly used by the browser without initiating a request to acquire the data from the server, so that the number of the HTTP requests can be reduced, the time for a user to wait for page loading can be reduced, and the user experience can be improved;
2) the IndexDB cache service can intelligently update cache data, can directly send a request to a background server under the condition that the cache data is invalid, and stores the returned data to the IndexDB after fine processing, so that data cache support is provided for the next request, and the browser using friendliness of a user can be improved.
In order to implement the method according to the embodiment of the present application, an embodiment of the present application further provides a data processing apparatus, as shown in fig. 7, the apparatus includes:
an obtaining unit 701, configured to obtain a first request of a front-end page; the first request is used for requesting to acquire first data of a target server;
a processing unit 702, configured to determine whether there is unreenabled first cache data in IndexDB; the first cache data corresponds to the first data; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
In an embodiment, the processing unit 702 is specifically configured to:
judging whether the first cache data exists in the IndexDB by using a first identifier corresponding to the first data to obtain a first judgment result;
under the condition that the first judgment result represents that the first cache data exists in the IndexDB, judging whether the first cache data is invalid or not by using at least one second identifier corresponding to the first request to obtain a second judgment result; the second identification represents a data version corresponding to the first request.
In an embodiment, the processing unit 702 is specifically configured to:
obtaining at least one third identifier corresponding to the first cache data from the IndexDB; the third identification represents a data version corresponding to the first cache data;
comparing the at least one second identifier with the at least one third identifier;
determining that the first cached data is not stale if the at least one second identifier is consistent with the at least one third identifier;
determining that the first cached data has failed if the at least one second identifier is inconsistent with the at least one third identifier.
In an embodiment, the processing unit 702 is further configured to:
injecting first configuration information into the first request through a first function to obtain a second request; the first configuration information comprises at least one second identifier; the second identification represents a data version corresponding to the first request; the second request is used for requesting to acquire the first data;
sending the second request to the target server; and receiving first data returned by the target server based on the second request.
In an embodiment, when the first data is returned to the front-end page, the processing unit 702 is further configured to:
generating first cache data by using the first data when the first cache data does not exist in the IndexDB;
or,
updating the first cache data with the first data if the first cache data has failed.
In an embodiment, the processing unit 702 is further configured to:
performing first processing on the first data based on a first preset rule so that the first processed first data meet a first preset condition;
and generating first cache data by using the first processed first data.
In one embodiment, the front end page is in a first mode of operation; the processing unit 702 is further configured to:
judging whether the current network environment meets a second preset condition or not, and judging whether the front-end page meets a third preset condition or not;
under the condition that the current network environment does not meet the second preset condition and/or under the condition that the front-end page does not meet the third preset condition, controlling the front-end page to enter the first working mode;
controlling the front-end page to enter a second working mode under the condition that the current network environment meets the second preset condition and the front-end page meets the third preset condition; wherein,
and under the condition that the front-end page is in a second working mode, acquiring all data requested by the front-end page from the IndexDB.
In practical applications, the obtaining unit 701 and the processing unit 702 may be implemented by a processor in a data processing apparatus.
It should be noted that: the data processing apparatus provided in the above embodiment is only illustrated by the division of the program modules when processing data, and in practical applications, the processing may be distributed to different program modules according to needs, that is, the internal structure of the apparatus may be divided into different program modules to complete all or part of the processing described above. In addition, the data processing apparatus and the data processing method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Based on the hardware implementation of the program module, and in order to implement the method according to the embodiment of the present application, an embodiment of the present application further provides an electronic device, as shown in fig. 8, where the electronic device 800 includes:
a communication interface 801 capable of performing information interaction with other electronic devices;
the processor 802 is connected with the communication interface 801 to realize information interaction with other electronic devices, and is used for executing the method provided by one or more technical schemes when running a computer program;
a memory 803 storing a computer program capable of running on the processor 802.
Specifically, the processor 802 is configured to:
acquiring a first request of a front-end page; the first request is used for requesting to acquire first data of a target server;
judging whether inefficacy first cache data exists in the IndexDB or not; the first cache data corresponds to the first data;
returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB;
and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
In an embodiment, the processor 802 is specifically configured to:
judging whether the first cache data exists in the IndexDB by using a first identifier corresponding to the first data to obtain a first judgment result;
under the condition that the first judgment result represents that the first cache data exists in the IndexDB, judging whether the first cache data is invalid or not by using at least one second identifier corresponding to the first request to obtain a second judgment result; the second identification represents a data version corresponding to the first request.
In an embodiment, the processor 802 is specifically configured to:
obtaining at least one third identifier corresponding to the first cache data from the IndexDB; the third identification represents a data version corresponding to the first cache data;
comparing the at least one second identifier with the at least one third identifier;
determining that the first cached data is not stale if the at least one second identifier is consistent with the at least one third identifier;
determining that the first cached data has failed if the at least one second identifier is inconsistent with the at least one third identifier.
In an embodiment, the processor 802 is further configured to:
injecting first configuration information into the first request through a first function to obtain a second request; the first configuration information comprises at least one second identifier; the second identification represents a data version corresponding to the first request; the second request is used for requesting to acquire the first data;
sending the second request to the target server; and receiving first data returned by the target server based on the second request.
In an embodiment, when returning the first data to the front-end page, the processor 802 is further configured to:
generating first cache data by using the first data when the first cache data does not exist in the IndexDB;
or,
updating the first cache data with the first data if the first cache data has failed.
In an embodiment, the processor 802 is further configured to:
performing first processing on the first data based on a first preset rule so that the first processed first data meet a first preset condition;
and generating first cache data by using the first processed first data.
In one embodiment, the front end page is in a first mode of operation; the processor 802 is further configured to:
judging whether the current network environment meets a second preset condition or not, and judging whether the front-end page meets a third preset condition or not;
under the condition that the current network environment does not meet the second preset condition and/or under the condition that the front-end page does not meet the third preset condition, controlling the front-end page to enter the first working mode;
controlling the front-end page to enter a second working mode under the condition that the current network environment meets the second preset condition and the front-end page meets the third preset condition; wherein,
and under the condition that the front-end page is in a second working mode, acquiring all data requested by the front-end page from the IndexDB.
It should be noted that: the process of the processor 802 specifically executing the above operations is detailed in the method embodiment, and is not described here again.
Of course, in practice, the various components in the electronic device 800 are coupled together by a bus system 804. It is understood that the bus system 804 is used to enable communications among the components. The bus system 804 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 804 in FIG. 8.
The memory 803 in the present embodiment is used to store various types of data to support the operation of the electronic device 800. Examples of such data include: any computer program for operating on the electronic device 800.
The method disclosed in the embodiments of the present application can be applied to the processor 802, or implemented by the processor 802. The processor 802 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 802. The Processor 802 may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 802 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 803, and the processor 802 reads the information in the memory 803 and performs the steps of the aforementioned methods in conjunction with its hardware.
In an exemplary embodiment, the electronic Device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, Micro Controllers (MCUs), microprocessors (microprocessors), or other electronic components for performing the foregoing methods.
It is to be appreciated that the memory 803 of the subject embodiment can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memories described in the embodiments of the present application are intended to comprise, without being limited to, these and any other suitable types of memory.
In an exemplary embodiment, the present application further provides a storage medium, specifically a computer storage medium, for example, a memory 803 storing a computer program, which can be executed by the processor 802 of the electronic device 800 to perform the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disk, or CD-ROM.
It should be noted that: "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The technical means described in the embodiments of the present application may be arbitrarily combined without conflict.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.

Claims (10)

1. A data processing method, comprising:
acquiring a first request of a front-end page; the first request is used for requesting to acquire first data of a target server;
judging whether ineffectiveness first cache data exist in an index database IndexDB or not; the first cache data corresponds to the first data;
returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB;
and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
2. The method of claim 1, wherein the determining whether there is any inefficient first cache data in IndexDB comprises:
judging whether the first cache data exists in the IndexDB by using a first identifier corresponding to the first data to obtain a first judgment result;
under the condition that the first judgment result represents that the first cache data exists in the IndexDB, judging whether the first cache data is invalid or not by using at least one second identifier corresponding to the first request to obtain a second judgment result; the second identification represents a data version corresponding to the first request.
3. The method according to claim 2, wherein the determining whether the first cache data is invalid by using at least one second identifier corresponding to the first request comprises:
obtaining at least one third identifier corresponding to the first cache data from the IndexDB; the third identification represents a data version corresponding to the first cache data;
comparing the at least one second identifier with the at least one third identifier;
determining that the first cached data is not stale if the at least one second identifier is consistent with the at least one third identifier;
determining that the first cached data has failed if the at least one second identifier is inconsistent with the at least one third identifier.
4. The method of any of claims 1 to 3, wherein said obtaining the first data from the target server comprises:
injecting first configuration information into the first request through a first function to obtain a second request; the first configuration information comprises at least one second identifier; the second identification represents a data version corresponding to the first request; the second request is used for requesting to acquire the first data;
sending the second request to the target server; and receiving first data returned by the target server based on the second request.
5. The method of any of claims 1 to 3, wherein when returning the first data to the front-end page, the method further comprises:
generating first cache data by using the first data when the first cache data does not exist in the IndexDB;
or,
updating the first cache data with the first data if the first cache data has failed.
6. The method of claim 5, wherein generating the first cache data using the first data comprises:
performing first processing on the first data based on a first preset rule so that the first processed first data meet a first preset condition;
and generating first cache data by using the first processed first data.
7. A method according to any one of claims 1 to 3, wherein the front end page is in a first mode of operation; the method further comprises the following steps:
judging whether the current network environment meets a second preset condition or not, and judging whether the front-end page meets a third preset condition or not;
under the condition that the current network environment does not meet the second preset condition and/or under the condition that the front-end page does not meet the third preset condition, controlling the front-end page to enter the first working mode;
controlling the front-end page to enter a second working mode under the condition that the current network environment meets the second preset condition and the front-end page meets the third preset condition; wherein,
and under the condition that the front-end page is in a second working mode, acquiring all data requested by the front-end page from the IndexDB.
8. A data processing apparatus, comprising:
an acquisition unit, configured to acquire a first request of a front-end page; the first request is used for requesting to acquire first data of a target server;
the processing unit is used for judging whether inefficacy first cache data exist in the IndexDB or not; the first cache data corresponds to the first data; returning first cache data to the front-end page when the first cache data which is not invalidated exists in the IndexDB; and under the condition that the inefficacy first cache data does not exist in the IndexDB, acquiring the first data from the target server, and returning the first data to the front-end page.
9. An electronic device, comprising: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is adapted to perform the steps of the method of any one of claims 1 to 7 when running the computer program.
10. A storage medium having a computer program stored thereon, the computer program, when being executed by a processor, performing the steps of the method of any one of claims 1 to 7.
CN202111129273.2A 2021-09-26 2021-09-26 Data processing method and device, electronic equipment and storage medium Pending CN113806655A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111129273.2A CN113806655A (en) 2021-09-26 2021-09-26 Data processing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111129273.2A CN113806655A (en) 2021-09-26 2021-09-26 Data processing method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113806655A true CN113806655A (en) 2021-12-17

Family

ID=78938511

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111129273.2A Pending CN113806655A (en) 2021-09-26 2021-09-26 Data processing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113806655A (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001075771A (en) * 1999-08-31 2001-03-23 Matsushita Electric Ind Co Ltd Picture data processing system in web browser
JP2004013635A (en) * 2002-06-07 2004-01-15 Nec Corp Web browser cache method, server, and information processing terminal
GB201211176D0 (en) * 2012-06-22 2012-08-08 5Th Tier Ltd Network communications
US8543554B1 (en) * 2010-08-10 2013-09-24 ScalArc Inc. Method and system for transparent database query caching
CN103473272A (en) * 2013-08-20 2013-12-25 小米科技有限责任公司 Data processing method, device and system
CN104111829A (en) * 2014-06-25 2014-10-22 小米科技有限责任公司 Data storage method and device
CN105045539A (en) * 2015-08-20 2015-11-11 百度在线网络技术(北京)有限公司 Control method and apparatus for data offline storage
CN106055637A (en) * 2016-05-30 2016-10-26 努比亚技术有限公司 Buffer memory data processing method and server
CN110362776A (en) * 2019-07-23 2019-10-22 深圳前海微众银行股份有限公司 Browser front-end data storage method, device, equipment and readable storage medium storing program for executing
CN112035205A (en) * 2020-08-31 2020-12-04 深圳前海微众银行股份有限公司 Data processing method, device, equipment and storage medium

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001075771A (en) * 1999-08-31 2001-03-23 Matsushita Electric Ind Co Ltd Picture data processing system in web browser
JP2004013635A (en) * 2002-06-07 2004-01-15 Nec Corp Web browser cache method, server, and information processing terminal
US8543554B1 (en) * 2010-08-10 2013-09-24 ScalArc Inc. Method and system for transparent database query caching
GB201211176D0 (en) * 2012-06-22 2012-08-08 5Th Tier Ltd Network communications
CN103473272A (en) * 2013-08-20 2013-12-25 小米科技有限责任公司 Data processing method, device and system
CN104111829A (en) * 2014-06-25 2014-10-22 小米科技有限责任公司 Data storage method and device
CN105045539A (en) * 2015-08-20 2015-11-11 百度在线网络技术(北京)有限公司 Control method and apparatus for data offline storage
CN106055637A (en) * 2016-05-30 2016-10-26 努比亚技术有限公司 Buffer memory data processing method and server
CN110362776A (en) * 2019-07-23 2019-10-22 深圳前海微众银行股份有限公司 Browser front-end data storage method, device, equipment and readable storage medium storing program for executing
CN112035205A (en) * 2020-08-31 2020-12-04 深圳前海微众银行股份有限公司 Data processing method, device, equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
艾斯卡尔・亚克甫;齐向伟;: "在使用代理时因浏览器缺陷引起的几种攻击的介绍", 新疆师范大学学报(自然科学版), no. 04, 30 December 2010 (2010-12-30), pages 77 - 82 *

Similar Documents

Publication Publication Date Title
WO2021180025A1 (en) Message processing method and apparatus, electronic device and medium
KR102591878B1 (en) Search method, search device, electronic device, storage medium and computer program
US20140006487A1 (en) Methods for making ajax web applications bookmarkable and crawable and devices thereof
CN106656920B (en) Processing method, device, storage medium and the processor of HTTP service
US11436066B2 (en) System for offline object based storage and mocking of rest responses
CN113010818A (en) Access current limiting method and device, electronic equipment and storage medium
CN109918191B (en) Method and device for preventing frequency of service request
CN111367596A (en) Method and device for realizing service data processing and client
CN112612988A (en) Page processing method and device, computer equipment and storage medium
CN106202082B (en) Method and device for assembling basic data cache
CN112764948A (en) Data transmission method, data transmission device, computer device, and storage medium
CN111294377B (en) Dependency network request sending method, terminal device and storage medium
CN111767053A (en) Front-end page data acquisition method and device
CN112926002A (en) Service request processing method and device
CN117194068A (en) Cross-process data transmission method, system, equipment and storage medium
CN114374657B (en) Data processing method and device
CN113806655A (en) Data processing method and device, electronic equipment and storage medium
CN113157738B (en) In-heap data cache synchronization method and device, computer equipment and storage medium
CN113824675B (en) Method and device for managing login state
CN113760274B (en) Front-end assembly logic injection method and device
CN115658171A (en) Method and system for solving dynamic refreshing of java distributed application configuration in lightweight mode
CN114461950A (en) Global caching method and device, electronic equipment and storage medium
CN106940660B (en) Method and device for realizing cache
CN114637969A (en) Target object authentication method and device
CN113779122A (en) Method and apparatus for exporting data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination