CN117473011A - Data synchronization method, device and hybrid cache system - Google Patents
Data synchronization method, device and hybrid cache system Download PDFInfo
- Publication number
- CN117473011A CN117473011A CN202311436045.9A CN202311436045A CN117473011A CN 117473011 A CN117473011 A CN 117473011A CN 202311436045 A CN202311436045 A CN 202311436045A CN 117473011 A CN117473011 A CN 117473011A
- Authority
- CN
- China
- Prior art keywords
- cache
- data
- target data
- level
- middleware
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004590 computer program Methods 0.000 claims description 7
- 239000012634 fragment Substances 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 230000004044 response Effects 0.000 abstract description 9
- 230000002776 aggregation Effects 0.000 abstract description 7
- 238000004220 aggregation Methods 0.000 abstract description 7
- 230000003139 buffering effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- RYYVLZVUVIJVGH-UHFFFAOYSA-N caffeine Chemical group CN1C(=O)N(C)C(=O)C2=C1N=CN2C RYYVLZVUVIJVGH-UHFFFAOYSA-N 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 2
- LPHGQDQBBGAPDZ-UHFFFAOYSA-N Isocaffeine Natural products CN1C(=O)N(C)C(=O)C2=C1N(C)C=N2 LPHGQDQBBGAPDZ-UHFFFAOYSA-N 0.000 description 1
- 241000508269 Psidium Species 0.000 description 1
- 229960001948 caffeine Drugs 0.000 description 1
- VJEONQKOZGKCAK-UHFFFAOYSA-N caffeine Natural products CN1C(=O)N(C)C(=O)C2=C1C=CN2C VJEONQKOZGKCAK-UHFFFAOYSA-N 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a data synchronization method, a data synchronization device and a hybrid cache system, wherein the method comprises the following steps: obtaining multi-level cache between terminal equipment and a database; the multi-level cache includes: caching middleware and local caching of the terminal equipment; setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache order is from the first-level cache to the second-level cache; according to the caching order, sequentially synchronizing the target data into the multi-level cache, and sequentially searching the target data in the local cache and the cache middleware in response to a data acquisition request; and when the target data is not found in the local cache and the cache middleware, acquiring the target data in a database. Therefore, aggregation and use of various caches are realized, the use flow of the various caches is greatly simplified, and the performance of the system is improved.
Description
Technical Field
The disclosure relates to the technical field of networks, and in particular relates to a data synchronization method, a data synchronization device and a hybrid cache system.
Background
Before the data in the database is sent to the terminal equipment, the data can be stored in the cache in advance, so that when the terminal equipment accesses the data again, the data can be directly read from the memory, the access to the database is reduced, and the problem that the database is easy to crash under a high concurrency scene is solved.
However, the caches are usually of multiple types, and the caches of different types are good and bad when applied, so that multiple caches can be mixed for solving the performance problem in daily work, but the use methods of the caches of different types are different, and when the service has the requirement of using different caches, the service is complicated to realize and cannot be suitable for the requirement of improving the performance in the current cache use process.
Disclosure of Invention
The embodiment of the disclosure at least provides a data synchronization method, a data synchronization device and a hybrid cache system.
In a first aspect, an embodiment of the present disclosure provides a data synchronization method, including:
obtaining multi-level cache between terminal equipment and a database; the multi-level cache includes: caching middleware and local caching of the terminal equipment;
setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache;
according to the caching order, sequentially synchronizing the target data into the multi-level cache;
responding to a data acquisition request, and sequentially searching the target data in the local cache and the cache middleware;
and when the target data is not found in the local cache and the cache middleware, acquiring the target number in a database.
In an alternative embodiment, the sequentially synchronizing the target data into the multi-level cache according to the cache order includes:
caching the target data in the cache middleware;
and when the data acquisition request meets a caching condition, caching the target data into the local cache.
In an alternative embodiment, the buffering condition includes: the request times of the data acquisition requests reach preset times, the request time of the data acquisition requests reach preset synchronous time, and the local cache does not contain the target data;
the method further comprises the steps of:
and when the data acquisition request meets any condition in the cache conditions, determining that the data acquisition request meets the cache conditions.
In an alternative embodiment, the local cache includes a plurality of cache slices therein;
the caching the target data in the local cache includes:
after the target data is acquired, determining the data type of the target data;
and caching the target data into a cache fragment matched with the data type.
In a second aspect, an embodiment of the present disclosure further provides a hybrid cache system, including: the configuration module and the mixed cache module;
the configuration module acquires the multi-level caches to be configured and respectively determines the calling parameters of the multi-level caches;
and the mixed cache module provides a corresponding data interface for the multi-level cache based on the calling parameter so as to call the multi-level cache through the data interface.
In a third aspect, an embodiment of the present disclosure further provides a data synchronization apparatus, including:
the acquisition unit is used for acquiring the multi-level cache between the terminal equipment and the database; the multi-level cache includes: caching middleware and local caching of the terminal equipment;
the cache setting unit is used for setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache;
the data synchronization unit is used for sequentially synchronizing the target data into the multi-level cache according to the cache sequence;
the data searching unit is used for responding to a data acquisition request and sequentially searching the target data in the local cache and the cache middleware;
the data acquisition unit is used for acquiring the target data in the database when the target data is not found in the local cache and the cache middleware.
In a fourth aspect, embodiments of the present disclosure further provide a computer device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory in communication via the bus when the computer device is running, the machine-readable instructions when executed by the processor performing the steps of the first aspect, or any of the possible implementations of the first aspect.
In a fifth aspect, the present application further provides a computer storage medium storing computer executable instructions for performing the data synchronization method according to any one of the above-mentioned aspects of the present application.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, a multi-level cache between the terminal equipment and the database is firstly obtained, wherein the multi-level cache comprises: and setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache, and then sequentially synchronizing the target data into the multi-level caches according to the cache sequence. And then, in response to the data acquisition request, searching target data in the local cache and the cache middleware in turn, and acquiring the target data in the database when the target data is not found in the local cache and the cache middleware, thereby realizing aggregation and use of various caches, greatly simplifying the use flow of various caches and improving the performance of the system.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for the embodiments are briefly described below, which are incorporated in and constitute a part of the specification, these drawings showing embodiments consistent with the present disclosure and together with the description serve to illustrate the technical solutions of the present disclosure. It is to be understood that the following drawings illustrate only certain embodiments of the present disclosure and are therefore not to be considered limiting of its scope, for the person of ordinary skill in the art may admit to other equally relevant drawings without inventive effort.
FIG. 1 illustrates a flow chart of a data synchronization method provided by an embodiment of the present disclosure;
FIG. 2 illustrates an architecture diagram of a hybrid cache system provided by embodiments of the present disclosure;
FIG. 3 is a schematic diagram illustrating an operation flow of a hybrid cache system according to an embodiment of the disclosure;
FIG. 4 is a schematic diagram of a data synchronization apparatus according to an embodiment of the disclosure;
fig. 5 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, but not all embodiments. The components of the embodiments of the present disclosure, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be made by those skilled in the art based on the embodiments of this disclosure without making any inventive effort, are intended to be within the scope of this disclosure.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The term "and/or" is used herein to describe only one relationship, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
According to research, the data in the database can be stored in the cache in advance before the data is sent to the terminal equipment, so that the data can be directly read from the memory when the terminal equipment accesses the data again, the access to the database is reduced, and the problem that the database is easy to crash under a high concurrency scene is solved.
However, the caches are usually of multiple types, and the caches of different types are good and bad when applied, so that multiple caches can be mixed for solving the performance problem in daily work, but the use methods of the caches of different types are different, and when the service has the requirement of using different caches, the service is complicated to realize and cannot be suitable for the requirement of improving the performance in the current cache use process.
Based on the above study, the present disclosure provides a data synchronization method, apparatus, and hybrid cache system. In the embodiment of the present disclosure, a multi-level cache between a terminal device and a database is first obtained, where the multi-level cache includes: and setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache, and then sequentially synchronizing the target data into the multi-level caches according to the cache sequence. And then, in response to the data acquisition request, searching target data in the local cache and the cache middleware in turn, and acquiring the target data in the database when the target data is not found in the local cache and the cache middleware, thereby realizing aggregation and use of various caches, greatly simplifying the use flow of various caches and improving the performance of the system.
For the sake of understanding the present embodiment, first, a detailed description will be given of a data synchronization method disclosed in an embodiment of the present disclosure, where an execution body of the data synchronization method provided in the embodiment of the present disclosure is generally a computer device with a certain computing capability. In some possible implementations, the data synchronization method may be implemented by way of a processor invoking computer readable instructions stored in a memory.
Referring to fig. 1, a flowchart of a data synchronization method according to an embodiment of the disclosure is shown, where the method includes steps S101 to S109, where:
s101: acquiring multi-level cache between terminal equipment and a database; the multi-level cache includes: caching middleware and local caching of the terminal equipment.
S103: setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache order is from the first-level cache to the second-level cache.
In the embodiment of the present disclosure, the foregoing caching middleware may be redis, where redis may be used to cache hot data with higher request frequency of a terminal device, and other caching middleware may also be used, which is not specifically limited in this disclosure. In addition, the local cache may be a part of a portion of the physical memory of the terminal device, which is divided into a cache space.
Here, when the above-described caching order is set, cache persistence may be realized in consideration of higher stability of the cache middleware, so that the cache middleware may be determined as a primary cache, and, at the same time, in consideration of lower consumption of the local cache network, the data acquisition speed may be faster, so that the local cache may be regarded as a secondary cache. It should be understood that more types of caches may be further configured in the multi-level cache by the above system, and the specific manner of configuring the multi-level cache is based on actual use requirements, which is not specifically limited in this disclosure.
S105: and according to the caching order, sequentially synchronizing the target data into the multi-level caches.
S107: and responding to the data acquisition request, and sequentially searching the target data in the local cache and the cache middleware.
In the embodiment of the present disclosure, the above-mentioned data acquisition request may be used to instruct sequentially searching for the target data in the multi-level cache, for example, the terminal device may request to acquire the target data from the database through the data acquisition request, and instruct to acquire the target data into the multi-level cache at the same time, so that the subsequent terminal device acquires the target data again based on the multi-level cache.
Here, taking video buffering as an example, when a user watches video, the terminal device can request video data from the database through a data acquisition request, and the database responds to the data acquisition request to buffer the video data into the multi-level buffer, so that the terminal device directly requests the video data from the multi-level buffer when watching video later, thereby improving data acquisition efficiency and user watching experience.
S109: and when the target data is not found in the local cache and the cache middleware, acquiring the target data in a database.
In the embodiment of the disclosure, when the target data is searched, the network occupation of the local cache is considered to be smaller, and the acquisition is more convenient, so the data acquisition order may be opposite to the cache order, for example, the data acquisition order may be the local cache and the cache middleware in turn. That is, the target data may be preferentially searched in the local cache, and if not, the target data may be searched in the cache middleware. Under the condition that the multi-level cache is traversed according to the data acquisition order but target data are found, the target data can be found in the total data stored in the database in response to the data acquisition request, so that the utilization efficiency of the hybrid cache is improved, the access to the database is reduced, and the problem that the database is easily down in a high concurrency scene is solved.
As can be seen from the foregoing description, in the embodiments of the present disclosure, a multi-level cache between a terminal device and a database may be acquired first, where the multi-level cache includes: and setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache, and then sequentially synchronizing the target data into the multi-level caches according to the cache sequence. And then, in response to the data acquisition request, searching target data in the local cache and the cache middleware in turn, and acquiring the target data in the database when the target data is not found in the local cache and the cache middleware, thereby realizing aggregation and use of various caches, greatly simplifying the use flow of various caches and improving the performance of the system.
In an optional embodiment, step S105, according to the buffering order, synchronizes the target data into the multi-level buffer sequentially, includes:
s1051: and caching the target data into the cache middleware.
S1052: and when the data acquisition request meets a caching condition, caching the target data into the local cache.
In the embodiment of the present disclosure, considering that the local cache has instability, for example, the terminal device may periodically clean the local cache, and the local cache is limited by the limited memory size and capacity of the terminal device, based on this, the target data may be screened, and the screened target data is stored in the local cache, so as to reduce the memory management pressure of the terminal device.
Specifically, the buffer condition may be preset, and the target data that meets the buffer condition may be buffered in the local buffer, where the manner of filtering the target data by the buffer condition is described below, which is not described herein.
In the embodiment of the disclosure, considering that the local cache has instability and limited capacity, the target data can be screened through the cache condition, and the screened target data is stored in the local cache, so that the memory management pressure of the terminal equipment is reduced.
In an alternative embodiment, the above-mentioned buffering condition includes: the step S105 further includes the following steps that the number of requests of the data acquisition request reaches a preset number, the request time of the data acquisition request reaches a preset synchronization time, and the local cache does not contain the target data:
and when the data acquisition request meets any condition in the cache conditions, determining that the data acquisition request meets the cache conditions.
In the embodiment of the present disclosure, the above data acquisition request may hit one or more cache conditions, and thus, when the data request hits any one of the cache conditions, it may be determined that the data acquisition request satisfies the cache conditions. In particular, it may be determined whether the data acquisition request satisfies the buffering condition in the following manner.
Mode one: and determining whether the request times of the data acquisition requests reach the preset times.
In the embodiment of the present disclosure, the data acquisition request may be used to indicate that the same target data is acquired, after the data acquisition request is received, the number of requests of the data acquisition request may be queried, and if the number of requests reaches a preset number, it is indicated that the frequency of access of the user to the target data is higher, so it may be determined that the data acquisition request satisfies the buffering condition.
Mode two: and determining whether the request time of the data acquisition request reaches the preset synchronous time.
In the embodiment of the present disclosure, the data in the caching middleware and the local cache may be synchronized according to a certain time interval, specifically, if a request interval between two adjacent data acquisition requests is greater than or equal to the time interval, it is determined that the request time of the acquisition request reaches a preset synchronization time.
Mode three: whether the local cache contains target data.
In the embodiment of the present disclosure, considering that the local cache capacity is limited, the target data may be cleared or covered by mistake, so when the local cache does not contain the target data, the target data may be acquired from the cache middleware through the data acquisition request, and the target data is synchronized into the local cache again.
Based on this, in the embodiment of the present disclosure, considering that the local cache has instability and limited capacity, the target data may be screened through a cache condition, and the screened target data is stored in the local cache, so as to reduce the memory management pressure of the terminal device.
In an optional embodiment, the local cache includes a plurality of cache slices, and step S1052 caches the target data in the local cache, which specifically includes the following steps:
(1) After the target data are acquired, determining the data type of the target data;
(2) And caching the target data into a cache fragment matched with the data type.
In an embodiment of the present disclosure, the local cache may include a plurality of cache slices, for example, cache tiles, where a data type of the data stored in each cache tile is different. For example, taking video buffering as an example here, the data types of the video include: movies, television shows, animation, variety, etc. After the target data is acquired, the type information of the target data can be acquired, the caffeine fragments matched with the type information are determined, and the target data is cached in the caffeine fragments.
It should be appreciated that when a call is made to target data in the local cache, the call request may be parsed to obtain a data type and a cache tile matching the data type may be determined. Here, the hot data stored in all caffeine slices may be the same as the hot data in the cache middleware.
In the embodiment of the disclosure, a plurality of cache fragments can be set in the local cache for storing cache data of different data types, so that the data searching efficiency of the terminal equipment when the target data is called is improved.
In summary, in the embodiment of the present disclosure, a multi-level cache between a terminal device and a database may be obtained first, where the multi-level cache includes: and setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache, and then sequentially synchronizing the target data into the multi-level caches according to the cache sequence. And then, in response to the data acquisition request, searching target data in the local cache and the cache middleware in turn, and acquiring the target data in the database when the target data is not found in the local cache and the cache middleware, thereby realizing aggregation and use of various caches, greatly simplifying the use flow of various caches and improving the performance of the system.
It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.
Based on the same inventive concept, the embodiment of the present disclosure further provides a hybrid cache system corresponding to the data synchronization method, and since the principle of the device in the embodiment of the present disclosure for solving the problem is similar to that of the data synchronization method in the embodiment of the present disclosure, the implementation of the device may refer to the implementation of the method, and the repetition is omitted.
Referring to fig. 2, an architecture diagram of a hybrid cache system according to an embodiment of the disclosure is shown, where the system includes: a configuration module 10 and a hybrid cache module 20, wherein:
the configuration module 10 obtains the multi-level cache to be configured, and determines the call parameters of the multi-level cache respectively.
The hybrid cache module 20 provides a corresponding data interface for the multi-level cache based on the call parameter, so as to call the multi-level cache through the data interface.
In the embodiment of the disclosure, the calling modes of various caches can be encapsulated in the calling parameters, when each level of caches are called based on the hybrid cache system, the corresponding caches can be called by the calling modes of the calling parameters, so that unification of different types of cache calling modes is realized, a user does not need to pay attention to accessed cache implementation details, and the same mode is used.
It should be understood that the multi-level cache management may be performed by modifying the above-mentioned call parameter, for example, when a cache type is newly added, the call mode of the cache may be encapsulated into the call parameter, or a call method of a certain cache may be deleted from the call parameter, so as to deactivate the cache, or the like. In addition, the cache sequence and the data acquisition sequence of the multi-level cache can be encapsulated into the call parameters, and the two can be adjusted by modifying the call parameters.
Referring to fig. 3, a schematic operation flow diagram of a hybrid cache system according to an embodiment of the present disclosure is shown, where a user may call an application program on the terminal device through an http interface to obtain corresponding data, for example, video data. Specifically, the application program may initiate a data acquisition request to the hybrid cache system, and after the buffer cache system receives the data acquisition request, the buffer cache system may invoke the multi-level cache and/or the database based on the invocation parameters to invoke the target data.
As can be seen from fig. 3, the multi-level cache includes a local cache (cache fragment cache and guava cache) and a cache middleware redis, and in addition, the hybrid cache module may also call data in a local memory and a database (mysql). Here, the above-mentioned call parameters may be configured by a configuration module, and a specific configuration manner is described in the corresponding embodiment of fig. 2, which is not described herein again.
To sum up, in the embodiment of the present disclosure, a multi-level cache between a terminal device and a database is first obtained, where the multi-level cache includes: and setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache, and then sequentially synchronizing the target data into the multi-level caches according to the cache sequence. And then, in response to the data acquisition request, searching target data in the local cache and the cache middleware in turn, and acquiring the target data in the database when the target data is not found in the local cache and the cache middleware, thereby realizing aggregation and use of various caches, greatly simplifying the use flow of various caches and improving the performance of the system.
It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.
Based on the same inventive concept, the embodiments of the present disclosure further provide a data synchronization device corresponding to the data synchronization method, and since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to that of the data synchronization method in the embodiments of the present disclosure, the implementation of the device may refer to the implementation of the method, and the repetition is omitted.
Referring to fig. 4, a schematic diagram of a data synchronization device according to an embodiment of the disclosure is shown, where the device includes: an acquisition unit 41, a cache setting unit 42, a data synchronization unit 43, a data search unit 44, a data acquisition unit 45; wherein,
an obtaining unit 41, configured to obtain a multi-level cache between the terminal device and the database, and determine a cache order corresponding to the multi-level cache;
a cache setting unit 42, configured to set the cache middleware as a first level cache and set the local cache as a second level cache, where a cache order is from the first level cache to the second level cache;
a data synchronization unit 43, configured to sequentially synchronize the target data into the multi-level cache according to the cache order;
a data searching unit 44, configured to sequentially search the local cache and the cache middleware for the target data in response to a data acquisition request;
the data obtaining unit 45 is configured to obtain the target data in a database when the target data is not found in the local cache and the cache middleware.
In the embodiment of the present disclosure, a multi-level cache between a terminal device and a database may be acquired first, where the multi-level cache includes: and setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache, and then sequentially synchronizing the target data into the multi-level caches according to the cache sequence. And then, in response to the data acquisition request, searching target data in the local cache and the cache middleware in turn, and acquiring the target data in the database when the target data is not found in the local cache and the cache middleware, thereby realizing aggregation and use of various caches, greatly simplifying the use flow of various caches and improving the performance of the system.
In a possible implementation, the data synchronization unit 43 is further configured to:
caching the target data in the cache middleware;
and when the data acquisition request meets a caching condition, caching the target data into the local cache.
In a possible implementation manner, the buffering condition includes: the request times of the data acquisition requests reach preset times, the request time of the data acquisition requests reach preset synchronous time, and the local cache does not contain the target data; the data synchronization unit 43 is further configured to:
and when the data acquisition request meets any condition in the cache conditions, determining that the data acquisition request meets the cache conditions.
In a possible implementation manner, the local cache includes a plurality of cache slices; the data synchronization unit 43 is further configured to:
after the target data is acquired, determining the data type of the target data;
and caching the target data into a cache fragment matched with the data type.
The process flow of each unit in the apparatus and the interaction flow between units may be described with reference to the related descriptions in the above method embodiments, which are not described in detail herein.
Corresponding to the data synchronization method in fig. 1, the embodiment of the present disclosure further provides a computer device 500, as shown in fig. 5, which is a schematic structural diagram of the computer device 500 provided in the embodiment of the present disclosure, including:
a processor 51, a memory 52, and a bus 53; memory 52 is used to store execution instructions, including memory 521 and external storage 522; the memory 521 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 51 and data exchanged with the external memory 522 such as a hard disk, and the processor 51 exchanges data with the external memory 522 through the memory 521, and when the computer device 500 is operated, the processor 51 and the memory 52 communicate with each other through the bus 53, so that the processor 51 executes the following instructions:
obtaining multi-level cache between terminal equipment and a database; the multi-level cache includes: caching middleware and local caching of the terminal equipment;
setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache order is from the first-level cache to the second-level cache;
according to the caching order, sequentially synchronizing the target data into the multi-level cache;
responding to a data acquisition request, and sequentially searching the target data in the local cache and the cache middleware;
and when the target data is not found in the local cache and the cache middleware, acquiring the target data in a database.
The disclosed embodiments also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the data synchronization method described in the method embodiments above. Wherein the storage medium may be a volatile or nonvolatile computer readable storage medium.
The embodiments of the present disclosure further provide a computer program product, where the computer program product carries a program code, where instructions included in the program code may be used to perform the steps of the data synchronization method described in the foregoing method embodiments, and specifically reference the foregoing method embodiments will not be described herein.
Wherein the above-mentioned computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in essence or a part contributing to the prior art or a part of the technical solution, or in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present disclosure. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the foregoing examples are merely specific embodiments of the present disclosure, and are not intended to limit the scope of the disclosure, but the present disclosure is not limited thereto, and those skilled in the art will appreciate that while the foregoing examples are described in detail, it is not limited to the disclosure: any person skilled in the art, within the technical scope of the disclosure of the present disclosure, may modify or easily conceive changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features thereof; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the disclosure, and are intended to be included within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (8)
1. A method of data synchronization, comprising:
obtaining multi-level cache between terminal equipment and a database; the multi-level cache includes: caching middleware and local caching of the terminal equipment;
setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache order is from the first-level cache to the second-level cache;
according to the caching order, sequentially synchronizing target data into the multi-level caches;
responding to a data acquisition request, and sequentially searching the target data in the local cache and the cache middleware;
and when the target data is not found in the local cache and the cache middleware, acquiring the target data in a database.
2. The method of claim 1, wherein sequentially synchronizing the target data into the multi-level cache according to the cache order comprises:
caching the target data in the cache middleware;
and when the data acquisition request meets a caching condition, caching the target data into the local cache.
3. The method of claim 2, wherein the cache condition comprises: the request times of the data acquisition requests reach preset times, the request time of the data acquisition requests reach preset synchronous time, and the local cache does not contain the target data;
the method further comprises the steps of:
and when the data acquisition request meets any condition in the cache conditions, determining that the data acquisition request meets the cache conditions.
4. The method of claim 2, wherein the local cache includes a plurality of cache slices therein;
the caching the target data in the local cache includes:
after the target data is acquired, determining the data type of the target data;
and caching the target data into a cache fragment matched with the data type.
5. A hybrid cache system, comprising: the configuration module and the mixed cache module;
the configuration module acquires the multi-level caches to be configured and respectively determines the calling parameters of the multi-level caches;
and the mixed cache module provides a corresponding data interface for the multi-level cache based on the calling parameter so as to call the multi-level cache through the data interface.
6. A data synchronization device, comprising:
the acquisition unit is used for acquiring the multi-level cache between the terminal equipment and the database; the multi-level cache includes: caching middleware and local caching of the terminal equipment;
the cache setting unit is used for setting the cache middleware as a first-level cache and setting the local cache as a second-level cache, wherein the cache sequence is from the first-level cache to the second-level cache;
the data synchronization unit is used for sequentially synchronizing the target data into the multi-level cache according to the cache sequence;
the data searching unit is used for responding to a data acquisition request and sequentially searching the target data in the local cache and the cache middleware;
the data acquisition unit is used for acquiring the target data in the database when the target data is not found in the local cache and the cache middleware.
7. A computer device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating over the bus when the computer device is running, said machine readable instructions when executed by said processor performing the steps of the data synchronization method according to any of claims 1 to 4.
8. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the data synchronization method according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311436045.9A CN117473011A (en) | 2023-10-31 | 2023-10-31 | Data synchronization method, device and hybrid cache system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311436045.9A CN117473011A (en) | 2023-10-31 | 2023-10-31 | Data synchronization method, device and hybrid cache system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117473011A true CN117473011A (en) | 2024-01-30 |
Family
ID=89634305
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311436045.9A Pending CN117473011A (en) | 2023-10-31 | 2023-10-31 | Data synchronization method, device and hybrid cache system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117473011A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118132598A (en) * | 2024-05-07 | 2024-06-04 | 深圳九有数据库有限公司 | Database data processing method and device based on multi-level cache |
-
2023
- 2023-10-31 CN CN202311436045.9A patent/CN117473011A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118132598A (en) * | 2024-05-07 | 2024-06-04 | 深圳九有数据库有限公司 | Database data processing method and device based on multi-level cache |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106911780B (en) | Service ID generation method, device and system | |
| CN108804234B (en) | Data storage system and method of operation thereof | |
| CN106649349B (en) | Data caching method, device and system for game application | |
| CN107943594B (en) | Data acquisition method and device | |
| CN110909025A (en) | Database query method, query device and terminal | |
| US20150143065A1 (en) | Data Processing Method and Apparatus, and Shared Storage Device | |
| US20170031948A1 (en) | File synchronization method, server, and terminal | |
| WO2017185616A1 (en) | File storage method and electronic equipment | |
| CN110162543A (en) | A kind of information query method and device | |
| CN105224255B (en) | A kind of storage file management method and device | |
| CN102023973A (en) | Method, device and system for synchronizing database with application cache server | |
| CN104333501B (en) | Contact list display method and device | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| US20180300250A1 (en) | Method and apparatus for storing data | |
| CN110674247A (en) | Barrage information intercepting method and device, storage medium and equipment | |
| CN101763433B (en) | Data storage system and method | |
| CN111416825A (en) | Inter-thread lock-free log management method and system, terminal and storage medium | |
| CN117473011A (en) | Data synchronization method, device and hybrid cache system | |
| CN102073733A (en) | Method and device for managing Hash table | |
| WO2016078388A1 (en) | Data aging method and apparatus | |
| CN108268497A (en) | The method of data synchronization and device of relevant database | |
| CN104168174A (en) | Method and apparatus for information transmission | |
| CN109165712A (en) | Distributed generation method, device and computer storage medium by stages number | |
| CN112579650A (en) | Data processing method and system based on Redis cache | |
| CN111913913A (en) | Access request processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |