CN112100537A - Data acquisition method and application server based on local cache and distributed cache - Google Patents
Data acquisition method and application server based on local cache and distributed cache Download PDFInfo
- Publication number
- CN112100537A CN112100537A CN202010739659.4A CN202010739659A CN112100537A CN 112100537 A CN112100537 A CN 112100537A CN 202010739659 A CN202010739659 A CN 202010739659A CN 112100537 A CN112100537 A CN 112100537A
- Authority
- CN
- China
- Prior art keywords
- cache
- data identifier
- data
- local cache
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 claims description 32
- 238000012544 monitoring process Methods 0.000 claims description 7
- 230000006399 behavior Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 241000508269 Psidium Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
Abstract
The application relates to a data acquisition method based on local cache and distributed cache and an application server. The method comprises the following steps: the remote cache client module responds to the call from the user client for obtaining the cache data corresponding to the target data identification, and calls the local cache SDK module to obtain the cache data corresponding to the target data identification; the local cache SDK module responds to the call of the remote cache client module, checks whether the target data identification belongs to the hot spot data identification, acquires cache data corresponding to the target data identification from the local cache of the application server when the target data identification is determined to belong to the hot spot data identification, and calls back the cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identification from the distributed cache server when the target data identification is determined not to belong to the hot spot data identification, so that the influence on the cache performance of the distributed cache can be reduced under the concurrent access of high peaks.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data acquisition method based on a local cache and a distributed cache, and an application server.
Background
In a high-concurrency scene, the remote distributed cache is built to effectively deal with high-concurrency access, for example, in an order sale process of an e-commerce platform, the bottleneck of a database is solved by hitting cache data, and the overall performance is effectively improved. Redis is a cache component with very high performance, but the network IO (in/out) consumption caused by remote access to a Redis cache cluster can bring great influence on the cache performance of the Redis cache cluster, and especially in some scenes, besides remote access, a large amount of access from an intranet is also provided, which can greatly occupy the bandwidth of the intranet and influence the stability of an application layer.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a data acquisition method based on a local cache and a distributed cache and an application server.
The invention provides a data acquisition method based on local cache and distributed cache according to a first aspect, in one embodiment, the method is applied to an application server, and the application server comprises a remote cache client module and a local cache SDK module; the method comprises the following steps:
the remote cache client module responds to the call from the user client for obtaining the cache data corresponding to the target data identification, and calls the local cache SDK module to obtain the cache data corresponding to the target data identification;
and the local cache SDK module responds to the call of the remote cache client module, checks whether the target data identifier belongs to the hot spot data identifier, and acquires cache data corresponding to the target data identifier from a local cache of the application server when the target data identifier is determined to belong to the hot spot data identifier, or calls back a cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identifier from the distributed cache server when the target data identifier is determined not to belong to the hot spot data identifier.
In one embodiment, the method further comprises:
and the local cache SDK module generates a data identifier access event according to the target data identifier and sends the data identifier access event to the local cache server so that the local cache server executes hot data identifier discovery processing according to the data identifier access event.
In one embodiment, the method further comprises:
when receiving a hot data identifier list sent by a local cache server after hot data identifier discovery is performed, a local cache SDK module caches data corresponding to data identifiers included in the hot data identifier list into a local cache, wherein the hot data identifier list includes at least one data identifier.
In one embodiment, the remote cache client module is configured with a first preset interface;
when monitoring that the first preset interface has a corresponding calling behavior, the remote cache client module determines an associated data identifier corresponding to the calling behavior, and calls a data identifier failure interface of the local cache SDK module, so that the local cache SDK module performs failure processing on cache data corresponding to the associated data identifier in a local cache of the application server.
In one embodiment, the invalidation processing is performed on the cache data corresponding to the associated data identifier in the local cache of the application server by the local cache SDK module, which includes:
the local cache SDK module checks whether the associated data identification belongs to the hot spot data identification, when the associated data identification is determined to belong to the hot spot data identification, cache data corresponding to the associated data identification in the local cache are invalidated, a data identification invalidation event is generated according to the associated data identification, and the data identification invalidation event is sent to an external application server, so that the external application server invalidates the cache data corresponding to the data identification invalidation event in the local cache of the external application server.
The present invention provides according to a second aspect an application server, which in one embodiment comprises:
the remote cache client module is used for responding to the call from the user client for obtaining the cache data corresponding to the target data identification, and calling the local cache SDK module to obtain the cache data corresponding to the target data identification;
and the local cache SDK module is used for responding to the calling of the remote cache client module, checking whether the target data identifier belongs to the hot spot data identifier, and acquiring cache data corresponding to the target data identifier from a local cache of the application server when the target data identifier is determined to belong to the hot spot data identifier, or calling back a cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identifier from the distributed cache server when the target data identifier is determined not to belong to the hot spot data identifier.
In an embodiment, the local cache SDK module is further configured to generate a data identifier access event according to the target data identifier, and send the data identifier access event to the local cache server, so that the local cache server executes hot spot data identifier discovery processing according to the data identifier access event.
In an embodiment, the local cache SDK module is further configured to receive a hot spot data identifier list sent by the local cache server, and cache data corresponding to the data identifiers included in the hot spot data identifier list in the local cache, where the hot spot data identifier list includes at least one data identifier, the hot spot data identifier list is obtained by the local cache server after performing hot spot data identifier discovery processing, and the hot spot data identifier list includes at least one data identifier.
In one embodiment, the remote cache client module is further configured with a first preset interface; the local cache SDK module is configured with a data identification failure interface;
the remote cache client module is further used for determining the associated data identifier corresponding to the calling behavior when the first preset interface is monitored to have the corresponding calling behavior, and calling the data identifier failure interface to send the associated data identifier to the local cache SDK module;
and the local cache SDK module is further used for performing invalidation processing on cache data corresponding to the associated data identifier in the local cache of the application server when the associated data identifier is received through the data identifier invalidation interface.
In an embodiment, the local cache SDK module is configured to, when the local cache of the application server is used to perform invalidation processing on cache data corresponding to the associated data identifier, specifically, check whether the associated data identifier belongs to a hot data identifier, when it is determined that the associated data identifier belongs to the hot data identifier, perform invalidation processing on the cache data corresponding to the associated data identifier in the local cache of the application server, generate a data identifier invalidation event according to the associated data identifier, and send the data identifier invalidation event to the local cache SDK module of the external application server, so that the local cache SDK module of the external application server performs invalidation processing on the cache data corresponding to the data identifier invalidation event in the local cache of the external application server.
The data acquisition method based on the local cache and the distributed cache provided by the embodiment of the invention can be applied to an application server, the application server comprises a remote cache client module and a local cache SDK module, and the remote cache client module can respond to the call from a user client for acquiring the cache data corresponding to the target data identifier and call the local cache SDK module to acquire the cache data corresponding to the target data identifier; the local cache SDK module responds to the call of the remote cache client module, checks whether the target data identification belongs to the hot spot data identification, and acquires cache data corresponding to the target data identification from the local cache of the application server when the target data identification is determined to belong to the hot spot data identification, or recalls the cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identification from the distributed cache server when the target data identification is determined not to belong to the hot spot data identification.
Drawings
FIG. 1 is a diagram of an application environment of a data acquisition method based on a local cache and a distributed cache in an embodiment;
FIG. 2 is a schematic flow chart illustrating a data acquisition method based on a local cache and a distributed cache according to an embodiment;
FIG. 3 is an overall architecture diagram of a business system in one embodiment;
FIG. 4 is a diagram of a local cache architecture in a business system in one embodiment;
FIG. 5 is a diagram of a local cache architecture in a business system in accordance with another embodiment;
fig. 6 is a block diagram of a data acquisition apparatus based on a local cache and a distributed cache in an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Fig. 1 shows an application environment of a data acquisition method based on a local cache and a distributed cache in an embodiment of the present invention. The application environment may be a business system comprising user clients 10, application servers 20 and distributed cache servers 30. Of course, the business system may also include a database, which is not shown in the figure since the present embodiment does not involve interaction with the database.
The application server 20 deploys a local cache, extracts a target data identifier carried by a data acquisition request after the application server 20 receives the data acquisition request from the user client 10, then checks whether the target data identifier belongs to a hot spot data identifier, if the target data identifier belongs to the hot spot data identifier, acquires cache data corresponding to the target data identifier from the local cache of the application server, and if the target data identifier does not belong to the hot spot data identifier, the application server 20 requests the distributed cache server 30 to acquire the cache data corresponding to the target data identifier. It should be noted that the user client 10 may be, but is not limited to, various computing devices such as a personal computer, a notebook computer, a smart phone, a tablet computer, and a desktop computer, the application server 20 may be implemented by an independent server or a server cluster formed by a plurality of servers, the distributed cache server 30 includes a plurality of cache nodes, all of the cache nodes may be deployed in the same physical server, or each cache node may be implemented by an independent server or a server cluster formed by a plurality of servers. Further, the number of the user clients 10 and the application servers 20 may be plural.
Fig. 2 shows a data obtaining method based on a local cache and a distributed cache according to an embodiment of the present invention, and the following description will take the application server in fig. 1 as an example. The method comprises the following steps:
s110: and the remote cache client module responds to the call for obtaining the cache data corresponding to the target data identification from the user client, and calls the local cache SDK module to obtain the cache data corresponding to the target data identification.
S120: and the local cache SDK module responds to the call of the remote cache client module, checks whether the target data identifier belongs to the hot spot data identifier, and acquires cache data corresponding to the target data identifier from a local cache of the application server when the target data identifier is determined to belong to the hot spot data identifier, or calls back a cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identifier from the distributed cache server when the target data identifier is determined not to belong to the hot spot data identifier.
In this embodiment, the application server is deployed with a remote cache client module and a local cache SDK module. When the user client needs to obtain the cache data corresponding to the target data identifier, the remote cache client module can be called, and the remote cache client module can respond to the call of the user client to call the remote cache client module to obtain the cache data corresponding to the target data identifier. The target data identifier may be a key value (key) in key-value pair data (key-value), and the cache data corresponding to the target data identifier may be a value (value) corresponding to the key value (key).
The local cache SDK module responds to the call of the remote cache client module and checks whether the target data identification belongs to the hot spot data identification. Specifically, the local cache SDK module may maintain a hot spot data list (e.g., a key list), so as to query whether the target data identifier is in the hot spot data list, and if the target data identifier is in the hot spot data list, it may be determined that the target data identifier belongs to the hot spot data identifier, and otherwise, it may be determined that the target data identifier does not belong to the hot spot data identifier. When it is determined that the target data identifier belongs to the hot spot data identifier, the local cache SDK module may obtain cache data corresponding to the target data identifier from a local cache of the application server, where the local cache may be deployed in the local cache SDK module and maintained by the local cache SDK module, and specifically, a Redis or Guava storage component may be deployed as the local cache. And when the target data identification is determined not to belong to the hot spot data identification, the local cache SDK module calls back a cache data acquisition interface of the remote cache client module, so that the distributed cache server is accessed to acquire cache data corresponding to the target data identification.
By deploying the local cache and the distributed cache, part of access requests for the hot spot data identification can be placed in the local cache in front under the concurrent access of the peak, so that the impact on the distributed cache is reduced, the influence on the cache performance of the distributed cache during the high concurrency is reduced, and the cache performance of the distributed cache is further improved.
In one embodiment, the method further comprises:
and the local cache SDK module generates a data identifier access event according to the target data identifier and sends the data identifier access event to the local cache server so that the local cache server executes hot data identifier discovery processing according to the data identifier access event.
In this embodiment, when the target data identifier does not belong to the hot data identifier, the local cache SDK module reports the data identifier access event to the local cache server, so that the local cache server executes hot data identifier discovery processing according to the data identifier access event.
Specifically, the local cache SDK module may further include a communication module, and when the target data identifier does not belong to the hot spot data identifier, the local cache SDK module asynchronously reports the data identifier access event to the local cache server after being called by the remote cache client module.
Further, the method further comprises:
when receiving a hot data identifier list sent by a local cache server after hot data identifier discovery is performed, a local cache SDK module caches data corresponding to data identifiers included in the hot data identifier list into a local cache, wherein the hot data identifier list includes at least one data identifier.
In this embodiment, after receiving a data identifier access event reported by the local cache SDK module, the local cache server performs analysis and calculation on the data identifier of the entire service system, so as to detect a hot data identifier list (e.g., a key list) in the service system. Specifically, the local cache server may count the access amount of each data identifier, and then add the data identifier whose access amount exceeds a preset threshold value as a hot spot data identifier into the hot spot data identifier list.
After generating the hot spot data identification list, the local cache server side pushes the hot spot data identification list to a local cache SDK module of the application server, and after receiving the hot spot data identification list, the local cache SDK module updates a hot spot data list maintained by the local cache SDK module, and caches data corresponding to each data identification in the hot spot data identification list into a local cache of the application server.
In one embodiment, when monitoring that the first preset interface has a corresponding call behavior, the remote cache client module determines an associated data identifier corresponding to the call behavior, and calls a data identifier failure interface of the local cache SDK module, so that the local cache SDK module performs failure processing on cache data corresponding to the associated data identifier in the local cache of the application server.
In this embodiment, the cache space of the application server is limited, and therefore it is necessary to ensure that the data cached in the local cache is all hot data, and if the access amount of the data identifier corresponding to the cached data is reduced or the access amount is not high in all the data identifiers, it is necessary to exclude the hot data identifier. Specifically, a first preset interface (e.g., a set (), del () or expire () interface) may be configured at the remote cache client module of the application server, and when the first preset interface is called, the corresponding cache data in the local cache may be invalidated. More specifically, when monitoring that the first preset interface has a corresponding call behavior, the remote cache client module determines an associated data identifier corresponding to the call behavior, and calls a data identifier invalidation interface (e.g., invalid () interface) of the local cache SDK module, so that the local cache SDK module invalidates cache data corresponding to the associated data identifier in the local cache of the application server.
Further, in an embodiment, the invalidating, by the local cache SDK module, the cache data corresponding to the associated data identifier in the local cache of the application server includes:
the local cache SDK module checks whether the associated data identification belongs to the hot spot data identification;
when the associated data identification is determined to belong to the hot data identification, cache data corresponding to the associated data identification in the local cache is subjected to invalidation processing, a data identification invalidation event is generated according to the associated data identification, and the data identification invalidation event is sent to an external application server, so that the external application server performs invalidation processing on the cache data corresponding to the data identification invalidation event in the local cache.
In this embodiment, when monitoring that the first preset interface has a corresponding call behavior, the remote cache client module checks whether the associated data identifier belongs to the hot spot data identifier, and if the associated data identifier belongs to the hot spot data identifier, calls a data identifier failure interface of the local cache SDK module, so that the local cache SDK module performs failure processing on cache data corresponding to the associated data identifier in the local cache of the application server, and if the local cache SDK module is deployed in the hot spot data identifier, the remote cache client module does not need to perform operation.
Further, in a scenario where the application servers are multiple, if the associated data identifier belongs to a hot data identifier, the local cache SDK module further needs to push a data identifier failure event to the external application server, and specifically, the local cache SDK module of the application server pushes the data identifier failure event to the local cache SDK module in each external application server, so that the local cache SDK module of each external application server performs failure processing on the cache data corresponding to the associated data identifier in the local cache of the external application server, thereby ensuring that the hot data in all the application servers are consistent.
The application also provides an application embodiment. As shown in fig. 3, the overall architecture of the business system in this embodiment may be divided into three layers, namely, a storage layer, a proxy layer and an application layer. Wherein the storage tier comprises a plurality of (n) Redis clusters, each Redis cluster being deployable as a master-slave (slave) mode. The agent layer comprises a plurality of (n) application servers, application service appliances are realized by a JBOSS cluster (application server based on J2EE open source code), three jar packages of snf-redis-client, local-cache-sdk and jedis are deployed in each application server, wherein snf-redis-client provides a uniform cache API access entrance for the application layer and undertakes routing function forwarding work after distributed data horizontal segmentation, functional modules of 'hot spot discovery' and 'hot spot cache' are built in the local-cache-sdk, service functions are transparent, namely the original service functions can use the functions of the 'hot spot discovery' and 'hot spot cache' without changing, for the Java application service originally deployed in the application server, the global configuration of 'whether to start local cache' is only needed when snf-render-client package is quoted, the functions of hot spot discovery and local cache can be accessed and used without modifying codes, and the minimum intrusion to the Java application service system is realized. The application layer includes a plurality of (n) user clients, the user clients need to install corresponding application programs (i.e. JAVA APP in the figure), and the user clients can interact with snf-redis-client in the application server through the application programs.
Further, as shown in FIG. 4, which is an architecture diagram of a local cache, the application server includes snf-redis-client module and local-cache-sdk module. The snf-redis-client module may be defined with a plurality of interfaces including get (), getOrigin (), set ()/del ()/expire () interfaces. The local-cache-sdk module defines two interfaces, get (), invoke (), and in addition, the local-cache-sdk module may include a communication module and a core module, the communication module is configured with a key access reporting function and a hotspot discovery monitoring function, and the core module is configured with a hotspot maintenance function and a local cache maintenance function.
Specifically, when the Java application needs to obtain data, the snf-Redis-client module may be called to obtain a value (i.e., cache data) corresponding to a target key (i.e., target data identifier), and the snf-Redis-client module may, in response to the calling of the Java application, query the local-cache-sdk module whether the target key is a hot-spot key (i.e., hot-spot data identifier), and if the local-cache-sdk module determines that the target key is a hot-spot key, directly obtain the value corresponding to the target key in the local cache from the core module in the local-cache-sdk module, without accessing the distributed cache cluster, i.e., Redis cluster, so as to place an access request of the Java application at the application layer, which may effectively reduce the impact of a large number of access requests on the cache layer in a high concurrency scenario, and further, if the local-cache-sdk module determines that the target key is not a hot-spot key, the local-cache-sdk module will call back the getOrigin () interface of snf-Redis-client module so that the snf-Redis-client module will access the Redis cluster to obtain the value corresponding to the target key.
Further, after the snf-redis-client module calls the local-cache-sdk module each time, the local-cache-sdk module asynchronously reports the key access event to the local-cache-server cluster through the communication module of the local-cache-sdk module, so that the local-cache-server cluster can perform hotspot discovery according to reported data. The local-cache-server cluster can continuously collect key access events reported by a local-cache-sdk module, and certainly, in a scene with a plurality of application servers, the cluster can collect key access events reported by local-cache-sdk modules of each application server, and periodically (for example, once in 3 seconds) analyze and calculate cache access data of all application servers in a service system to detect a hot key list of the service system, wherein the hot key list comprises one or more hot keys. For the detected hot key list, the local-cache-server cluster pushes the hot key list to a communication module of a local-cache-sdk module in each application server, so that the local-cache-sdk module in each application server performs local caching based on the hot key list.
Furthermore, the set ()/del ()/expire () interface of the snf-redis-client module is called to cause the corresponding key value to fail, and when the interface is called, the snf-redis-client module synchronously calls the invalid () interface of the local-cache-sdk module to inform the corresponding key value failure event. Specifically, the local-cache-sdk module may determine whether the key value corresponding to the key value failure event is a hot key, if so, the core module of the local-cache-sdk module may first fail the value corresponding to the key value in the local cache to achieve strong consistency of local data, and the communication module of the local-cache-sdk module asynchronously pushes the key value failure event corresponding to the key value to the local-cache-sdk modules in other application servers, and when the communication module of the local-cache-sdk module in other application servers receives the key value failure event of the key value, the communication module may call the core module to fail the value corresponding to the key in the local cache, so as to achieve final consistency of local cache data of all application servers.
Furthermore, if there are multiple service systems in the application scenario, in order to meet the requirements of different service systems, an SCM configuration center may be further deployed in the architecture shown in fig. 4, and the SCM configuration center may perform parameterized configuration according to the actual requirements of each service system to implement personalized functions. As shown in fig. 5, after the SCM configuration center is deployed, the local-cache-sdk module of the application server in each service system reads configuration information of its own node, such as start-up/shut-down configuration and black-and-white list configuration of hot spot data identifier, from the SCM configuration center in the start-up and operation processes, and the local-cache-server cluster reads configuration information of its own node, such as a service application list and a hot spot threshold, from the SCM configuration center in the start-up and operation processes.
In one embodiment, as shown in fig. 6, there is provided a data acquisition apparatus based on a local cache and a distributed cache, including the following modules:
the remote cache client module 110 is configured to respond to a call from the user client for obtaining cache data corresponding to the target data identifier, and call the local cache SDK module to obtain cache data corresponding to the target data identifier;
the local cache SDK module 120 is configured to, in response to the call of the remote cache client module, check whether the target data identifier belongs to the hot data identifier, and when it is determined that the target data identifier belongs to the hot data identifier, obtain cache data corresponding to the target data identifier from a local cache of the application server, or, when it is determined that the target data identifier does not belong to the hot data identifier, call back a cache data obtaining interface of the remote cache client module to obtain cache data corresponding to the target data identifier from the distributed cache server.
In an embodiment, the local cache SDK module is further configured to generate a data identifier access event according to the target data identifier, and send the data identifier access event to the local cache server, so that the local cache server executes hot spot data identifier discovery processing according to the data identifier access event.
In an embodiment, the local cache SDK module is further configured to receive a hot spot data identifier list sent by the local cache server, and cache data corresponding to the data identifiers included in the hot spot data identifier list in the local cache, where the hot spot data identifier list includes at least one data identifier, the hot spot data identifier list is obtained by the local cache server after performing hot spot data identifier discovery processing, and the hot spot data identifier list includes at least one data identifier.
In one embodiment, the remote cache client module is further configured with a first preset interface; the local cache SDK module is configured with a data identification failure interface;
the remote cache client module is further used for determining the associated data identifier corresponding to the calling behavior when the first preset interface is monitored to have the corresponding calling behavior, and calling the data identifier failure interface to send the associated data identifier to the local cache SDK module;
and the local cache SDK module is further used for performing invalidation processing on cache data corresponding to the associated data identifier in the local cache of the application server when the associated data identifier is received through the data identifier invalidation interface.
In an embodiment, the local cache SDK module is configured to, when the local cache of the application server is used to perform invalidation processing on cache data corresponding to the associated data identifier, specifically, check whether the associated data identifier belongs to a hot data identifier, when it is determined that the associated data identifier belongs to the hot data identifier, perform invalidation processing on the cache data corresponding to the associated data identifier in the local cache of the application server, generate a data identifier invalidation event according to the associated data identifier, and send the data identifier invalidation event to the local cache SDK module of the external application server, so that the local cache SDK module of the external application server performs invalidation processing on the cache data corresponding to the data identifier invalidation event in the local cache of the external application server.
For specific limitations of the data acquisition device based on the local cache and the distributed cache, reference may be made to the above limitations of the data acquisition method based on the local cache and the distributed cache, and details are not described here. The modules in the data acquisition device based on the local cache and the distributed cache can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A data acquisition method based on local cache and distributed cache is characterized in that the method is applied to an application server, wherein the application server comprises a remote cache client module and a local cache SDK module; the method comprises the following steps:
the remote cache client module responds to the call from the user client for obtaining the cache data corresponding to the target data identification, and calls the local cache SDK module to obtain the cache data corresponding to the target data identification;
and the local cache SDK module responds to the call of the remote cache client module, checks whether the target data identifier belongs to a hot data identifier, and acquires cache data corresponding to the target data identifier from a local cache of the application server when the target data identifier is determined to belong to the hot data identifier, or calls back a cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identifier from a distributed cache server when the target data identifier is determined not to belong to the hot data identifier.
2. The method of claim 1, wherein the method further comprises:
and the local cache SDK module generates a data identifier access event according to the target data identifier and sends the data identifier access event to a local cache server so that the local cache server executes hot spot data identifier discovery processing according to the data identifier access event.
3. The method of claim 2, wherein the method further comprises:
when receiving a hot spot data identifier list sent by the local caching server after hot spot data identifier discovery is performed, the local caching SDK module caches data corresponding to data identifiers included in the hot spot data identifier list into a local cache, where the hot spot data identifier list includes at least one data identifier.
4. The method of claim 1, wherein the remote cache client module is configured with a first preset interface; when monitoring that the first preset interface has a corresponding calling behavior, the remote cache client module determines a relevant data identifier corresponding to the calling behavior, and calls a data identifier failure interface of the local cache SDK module, so that the local cache SDK module performs failure processing on cache data corresponding to the relevant data identifier in a local cache of the application server.
5. The method of claim 4,
the local cache SDK module performs invalidation processing on cache data corresponding to the associated data identifier in the local cache of the application server, and the invalidation processing includes:
the local cache SDK module checks whether the associated data identification belongs to a hot data identification, when the associated data identification is determined to belong to the hot data identification, cache data corresponding to the associated data identification in a local cache are invalidated, a data identification invalidation event is generated according to the associated data identification, and the data identification invalidation event is sent to an external application server, so that the external application server invalidates the cache data corresponding to the data identification invalidation event in the local cache of the external application server.
6. An application server, comprising:
the remote cache client module is used for responding to the call from the user client for obtaining the cache data corresponding to the target data identification, and calling the local cache SDK module to obtain the cache data corresponding to the target data identification;
and the local cache SDK module is used for responding to the calling of the remote cache client module, checking whether the target data identifier belongs to a hot spot data identifier, and acquiring cache data corresponding to the target data identifier from a local cache of the application server when the target data identifier is determined to belong to the hot spot data identifier, or calling back a cache data acquisition interface of the remote cache client module to acquire the cache data corresponding to the target data identifier from a distributed cache server when the target data identifier is determined not to belong to the hot spot data identifier.
7. The application server of claim 6,
the local cache SDK module is further configured to generate a data identifier access event according to the target data identifier, and send the data identifier access event to a local cache server, so that the local cache server executes hot spot data identifier discovery processing according to the data identifier access event.
8. The application server of claim 7,
the local cache SDK module is further configured to receive a hot spot data identifier list sent by the local cache server, and cache data corresponding to the data identifiers included in the hot spot data identifier list in a local cache, where the hot spot data identifier list includes at least one data identifier, the hot spot data identifier list is obtained by the local cache server after performing hot spot data identifier discovery processing, and the hot spot data identifier list includes at least one data identifier.
9. The application server of claim 6, wherein the remote cache client module is further configured with a first preset interface; the local cache SDK module is configured with a data identification failure interface;
the remote cache client module is further configured to determine an associated data identifier corresponding to the call behavior when monitoring that the first preset interface has the corresponding call behavior, and call the data identifier failure interface to send the associated data identifier to the local cache SDK module;
and the local cache SDK module is further configured to perform invalidation processing on cache data corresponding to the associated data identifier in the local cache of the application server when the associated data identifier is received through the data identifier invalidation interface.
10. The application server of claim 9,
the local cache SDK module is configured to, when the local cache of the application server is configured to perform invalidation processing on cache data corresponding to the associated data identifier in the local cache of the application server, specifically, check whether the associated data identifier belongs to a hot data identifier, when it is determined that the associated data identifier belongs to the hot data identifier, perform invalidation processing on cache data corresponding to the associated data identifier in the local cache of the application server, generate a data identifier invalidation event according to the associated data identifier, and send the data identifier invalidation event to the local cache SDK module of the external application server, so that the local cache SDK module of the external application server performs invalidation processing on cache data corresponding to the data identifier invalidation event in the local cache of the external application server.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010739659.4A CN112100537A (en) | 2020-07-28 | 2020-07-28 | Data acquisition method and application server based on local cache and distributed cache |
| PCT/CN2021/099660 WO2022022105A1 (en) | 2020-07-28 | 2021-06-11 | Data acquisition method based on local cache and distributed cache, and application server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010739659.4A CN112100537A (en) | 2020-07-28 | 2020-07-28 | Data acquisition method and application server based on local cache and distributed cache |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112100537A true CN112100537A (en) | 2020-12-18 |
Family
ID=73749528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010739659.4A Pending CN112100537A (en) | 2020-07-28 | 2020-07-28 | Data acquisition method and application server based on local cache and distributed cache |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112100537A (en) |
| WO (1) | WO2022022105A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112783866A (en) * | 2021-01-29 | 2021-05-11 | 深圳追一科技有限公司 | Data reading method and device, computer equipment and storage medium |
| CN113242285A (en) * | 2021-04-30 | 2021-08-10 | 北京京东拓先科技有限公司 | Hotspot data processing method, device and system |
| CN113806399A (en) * | 2020-12-23 | 2021-12-17 | 京东科技控股股份有限公司 | Hotspot data processing method, device, equipment and storage medium |
| CN114003472A (en) * | 2022-01-04 | 2022-02-01 | 连连（杭州）信息技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| WO2022022105A1 (en) * | 2020-07-28 | 2022-02-03 | 苏宁易购集团股份有限公司 | Data acquisition method based on local cache and distributed cache, and application server |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114629883B (en) * | 2022-03-01 | 2023-12-29 | 北京奇艺世纪科技有限公司 | Service request processing method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110258424A1 (en) * | 2010-04-14 | 2011-10-20 | Chia-Hsin Huang | Distributive Cache Accessing Device and Method for Accelerating to Boot Remote Diskless Computers |
| CN106599152A (en) * | 2016-12-07 | 2017-04-26 | 北京华夏电通科技有限公司 | Data caching method and system |
| CN107231395A (en) * | 2016-03-25 | 2017-10-03 | 阿里巴巴集团控股有限公司 | Date storage method, device and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9286293B2 (en) * | 2008-07-30 | 2016-03-15 | Microsoft Technology Licensing, Llc | Populating and using caches in client-side caching |
| CN110753099B (en) * | 2019-10-12 | 2023-09-29 | 平安健康保险股份有限公司 | Distributed cache system and cache data updating method |
| CN111078147B (en) * | 2019-12-16 | 2022-06-28 | 南京领行科技股份有限公司 | Processing method, device and equipment for cache data and storage medium |
| CN112100537A (en) * | 2020-07-28 | 2020-12-18 | 苏宁云计算有限公司 | Data acquisition method and application server based on local cache and distributed cache |
-
2020
- 2020-07-28 CN CN202010739659.4A patent/CN112100537A/en active Pending
-
2021
- 2021-06-11 WO PCT/CN2021/099660 patent/WO2022022105A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110258424A1 (en) * | 2010-04-14 | 2011-10-20 | Chia-Hsin Huang | Distributive Cache Accessing Device and Method for Accelerating to Boot Remote Diskless Computers |
| CN107231395A (en) * | 2016-03-25 | 2017-10-03 | 阿里巴巴集团控股有限公司 | Date storage method, device and system |
| CN106599152A (en) * | 2016-12-07 | 2017-04-26 | 北京华夏电通科技有限公司 | Data caching method and system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022022105A1 (en) * | 2020-07-28 | 2022-02-03 | 苏宁易购集团股份有限公司 | Data acquisition method based on local cache and distributed cache, and application server |
| CN113806399A (en) * | 2020-12-23 | 2021-12-17 | 京东科技控股股份有限公司 | Hotspot data processing method, device, equipment and storage medium |
| CN112783866A (en) * | 2021-01-29 | 2021-05-11 | 深圳追一科技有限公司 | Data reading method and device, computer equipment and storage medium |
| CN113242285A (en) * | 2021-04-30 | 2021-08-10 | 北京京东拓先科技有限公司 | Hotspot data processing method, device and system |
| CN114003472A (en) * | 2022-01-04 | 2022-02-01 | 连连（杭州）信息技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN114003472B (en) * | 2022-01-04 | 2022-04-12 | 连连（杭州）信息技术有限公司 | Data processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022022105A1 (en) | 2022-02-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112100537A (en) | Data acquisition method and application server based on local cache and distributed cache | |
| CN112910945B (en) | Request link tracking method and service request processing method | |
| CN110602169B (en) | Service calling method and device, computer equipment and storage medium | |
| CN110113224B (en) | Capacity monitoring method and device, computer equipment and storage medium | |
| US20160197948A1 (en) | System for detecting abnormal behavior by analyzing personalized initial use behavior pattern | |
| CN112015674A (en) | Cache resource access method and device based on multiple hierarchies and computer equipment | |
| CN112153170B (en) | Method, device and equipment for accessing server and storage medium | |
| CN110555041A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN109120616B (en) | Identity authentication method, identity authentication device, proxy server and storage medium | |
| CN112115167B (en) | Cache system hot spot data access method, device, equipment and storage medium | |
| CN112527410A (en) | Full link tracking method, device, equipment and storage medium | |
| CN108595280B (en) | Interface adaptation method and device, computer equipment and storage medium | |
| CN112883119A (en) | Data synchronization method and device, computer equipment and computer readable storage medium | |
| CN114629883B (en) | Service request processing method and device, electronic equipment and storage medium | |
| CN112764948A (en) | Data transmission method, data transmission device, computer device, and storage medium | |
| CN111988280A (en) | Server and request processing method | |
| CN111858588A (en) | Distributed application index service platform and data processing method | |
| CN109831521B (en) | Cache instance management method and device, computer equipment and storage medium | |
| CN114697232A (en) | Skywalking probe index data acquisition system and method and electronic equipment | |
| CN109857344B (en) | Heartbeat state judgment method and device based on shared memory and computer equipment | |
| CN113064732A (en) | Distributed system and management method thereof | |
| CN111159233A (en) | Distributed caching method, system, computer device and storage medium | |
| CN111338848B (en) | Failure application copy processing method and device, computer equipment and storage medium | |
| CN115550441A (en) | Caching proxy method and device | |
| CN114064780A (en) | Session information processing method, system, device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201218 |
|
| RJ01 | Rejection of invention patent application after publication |