CN112988708A - Version updating method and device, computer readable storage medium and processor - Google Patents
Version updating method and device, computer readable storage medium and processor Download PDFInfo
- Publication number
- CN112988708A CN112988708A CN202110267621.6A CN202110267621A CN112988708A CN 112988708 A CN112988708 A CN 112988708A CN 202110267621 A CN202110267621 A CN 202110267621A CN 112988708 A CN112988708 A CN 112988708A
- Authority
- CN
- China
- Prior art keywords
- service
- version number
- updating
- version
- memory cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000015654 memory Effects 0.000 claims abstract description 177
- 238000011084 recovery Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000001360 synchronised effect Effects 0.000 description 18
- 230000007246 mechanism Effects 0.000 description 17
- 238000012360 testing method Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 4
- 238000005859 coupling reaction Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009545 invasion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Abstract
The invention discloses a version updating method and device, a computer readable storage medium and a processor. The updating method comprises the following steps: acquiring all service table data recorded in a preset database, wherein a service detail table and a service general table are preset in the preset database, the service detail table is used for recording a service version number corresponding to each service table, the service general table records a latest version number, then updating the version numbers in the service detail table and the service general table based on all service table data to obtain an updating result, then judging whether the version number in the service general table is consistent with the total version number recorded by a memory cache end or not based on the updating result, and if the version number in the service general table is not consistent with the total version number recorded by the memory cache end, updating the total version number of the memory cache end based on the version number in the service general table. The invention solves the technical problem of inconsistent data in the database and the cache due to the separate updating of the data in the database and the cache in the related technology.
Description
Technical Field
The invention relates to the technical field of data updating, in particular to a version updating method and device, a computer readable storage medium and a processor.
Background
At present, the information age has entered, and how to keep the data in the memory cache synchronized with the data in the database is a problem that must be solved by using the memory cache technology. In a business system, the form of accessing a database can be divided into: read access and write access, wherein the number of read accesses is significantly higher than the number of write accesses.
In the related art, in order to make the database of the software system bear greater performance pressure, a layer of memory cache (e.g., Redis, etc.) is added, so that a large number of read requests are completed by the memory cache, and thus, the database only processes write requests, and the database pressure can be greatly reduced. In addition, the memory cache data is available only when the memory cache data and the database data keep consistent, and the existing update strategy for ensuring the consistency comprises the following steps: 1, updating a database and then updating a cache; 2, deleting the cache and then updating the database; and 3, updating the database and then deleting the cache. There are several drawbacks to these three strategies: in the first place, inconsistency between the two data may occur, for example, the database is updated first, and then the cache is updated. If the database in the first step is successfully updated and the cache in the second step is failed to be updated, the database and the cache are inconsistent. This inconsistency can lead to data corruption, where business logic relies on data to make decisions, and this cache inconsistency is often difficult to find in business systems.
The method has the following disadvantages: due to the high coupling with the business system, the intrusion of the code into the business system is high. These three strategies are typically to develop the updated cached code of the response in the business system. The mechanism directly coupled in the service system makes the service program too complex, and the service program is dispersed in different service functions, which is difficult to completely cover during testing and is easy to leave hidden quality troubles.
The disadvantages are three: it is difficult to accommodate the cache server horizontal expansion scenario. These three strategies typically require that the caching server be stand-alone. If the cache server is a plurality of machines and the number of machines can be dynamically expanded/contracted at any time, the cache updated service system code is more complex, and a large number of parameters and codes need to be maintained to support the real-time dynamic expansion of the cache server. At the same time, it is also difficult to ensure consistency among multiple cache servers (e.g., when there is a short jitter in a network of cache servers).
The defect is four: and a large amount of cache can not be hit for a long time in disaster recovery switching and the like. Production environments often employ a multi-center, multi-server disaster recovery deployment architecture. When disaster recovery switching occurs, the new cache server takes over the original failed cache server. Since the database in the memory of the new cache server is in an initial clean state, when the business system caches through the memory, a large amount of cache data cannot be hit, and the condition that the cache data cannot be hit lasts for a long time, so that the performance of the business system is obviously reduced. When the service system finds that the data can not be hit, the data is loaded from the database to the cache, and only when the service system finds that the data can not be hit, the data is loaded passively afterwards, so that the data can not be loaded actively and efficiently in batches in advance, and the phenomenon that a large amount of cache can not be hit for a long time can be avoided.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a version updating method and device, a computer readable storage medium and a processor, which are used for at least solving the technical problem that data in a database and data in a cache are inconsistent because the data in the database and the data in the cache are updated separately in the related art.
According to an aspect of the embodiments of the present invention, there is provided a version updating method applied to a memory cache end of a service system, where the memory cache end is connected to a preset database, the method including: acquiring all service table data recorded in the preset database, wherein a service detail table and a service summary table are preset in the preset database, the service detail table is used for recording a service version number corresponding to each service table, and the service summary table records a latest version number; updating the version numbers in the service detail table and the service summary table based on all the service table data to obtain an updating result; judging whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not based on the updating result; and if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end, updating the total version number of the memory cache end based on the version number in the service summary table.
Optionally, the step of updating the version numbers in the service detail table and the service summary table based on all the service table data to obtain an update result includes: analyzing whether a new service table appears in the preset database or not based on all the service table data; if a new service table appears in the preset database, adding a new version number in the service detail table, wherein the added new version number corresponds to the new service table; and if the new version number is higher than the version number recorded in the service summary table, updating the version number in the service summary table into the new version number to obtain the updating result.
Optionally, after determining whether the version number in the service summary table is consistent with the total version number recorded by the memory cache, the version updating method further includes: if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end, querying the service detail table by adopting a preset control thread to obtain the new service table corresponding to the new version number; reading data stored in the new service table; and updating the data in the new service table to the memory cache end.
Optionally, the step of determining, based on the update result, whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end includes: adopting a preset control thread to compare whether the version number in the business summary is consistent with the total version number recorded by the memory cache end or not; and if the version number in the service summary table is consistent with the total version number recorded by the memory cache end, continuing to execute version refreshing operation according to a version updating strategy.
Optionally, a total version number and a service detail version number are recorded in the memory cache end, where the total version number is consistent with a latest version number recorded in a service summary table in the preset database, and the service detail version number is consistent with a service version number corresponding to each service table recorded in a service detail table in the preset database.
Optionally, after updating the total version number of the memory cache terminal based on the version number in the service summary table, the version updating method further includes: updating the total version number of the memory cache end by adopting a preset control thread based on the version number in the service summary table; and updating the total version number of the memory cache terminal by adopting a thread control pool based on the version number recorded in the service detail table.
Optionally, the version updating method further includes: judging whether the memory cache end enters a disaster recovery switching mode or not; and if the memory cache end enters the disaster recovery switching mode, automatically loading all currently cached data of the memory cache end to a new server by adopting a preset control thread.
According to another aspect of the embodiments of the present invention, there is provided a version updating apparatus, applied to a memory cache end of a service system, where the memory cache end is connected to a preset database, the version updating apparatus including: an obtaining unit, configured to obtain all service table data recorded in the preset database, where a service detail table and a service summary table are preset in the preset database, the service detail table is used to record a service version number corresponding to each service table, and the service summary table records a latest version number; a first updating unit, configured to update the version number in the service detail table and the service summary table based on all the service table data, so as to obtain an update result; a judging unit, configured to judge whether a version number in the service summary table is consistent with a total version number recorded by the memory cache terminal based on the update result; and the second updating unit is used for updating the total version number of the memory cache end based on the version number in the service summary table if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end.
Optionally, the first updating unit includes: the first analysis module is used for analyzing whether a new business table appears in the preset database or not based on all the business table data; a first adding module, configured to add a new version number to the service detail table if a new service table appears in the preset database, where the added new version number corresponds to the new service table; and the first updating module is used for updating the version number in the business summary table into the new version number to obtain the updating result if the new version number is higher than the version number recorded in the business summary table.
Optionally, the version updating apparatus further includes: a first output module, configured to, after determining whether a version number in the service summary table is consistent with a total version number recorded by the memory cache end, query the service detail table by using a preset control thread if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end, so as to obtain the new service table corresponding to the new version number; the first reading module is used for reading the data stored in the new service table; and the second updating module is used for updating the data in the new service table to the memory cache end.
Optionally, the determining unit includes: the first comparison module is used for comparing whether the version number in the business summary table is consistent with the total version number recorded by the memory cache end or not by adopting a preset control thread; and the first execution module is used for continuing to execute version refreshing operation according to the version updating strategy if the version number in the service summary table is consistent with the total version number recorded by the memory cache terminal.
Optionally, a total version number and a service detail version number are recorded in the memory cache end, where the total version number is consistent with a latest version number recorded in a service summary table in the preset database, and the service detail version number is consistent with a service version number corresponding to each service table recorded in a service detail table in the preset database.
Optionally, the version updating apparatus further includes: a third updating module, configured to update the total version number of the memory cache terminal based on the version number in the service summary table by using a preset control thread after the total version number of the memory cache terminal is updated based on the version number in the service summary table; and the fourth updating module is used for updating the total version number of the memory cache terminal by adopting a thread control pool based on the version number recorded in the service detail table.
Optionally, the version updating apparatus further includes: the first judgment module is used for judging whether the memory cache end enters a disaster recovery switching mode or not; and the first loading module is used for automatically loading all currently cached data of the memory cache end to a new server by adopting a preset control thread when the memory cache end enters the disaster recovery switching mode.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, wherein the storage medium is used for storing a computer program, and when the computer program is executed by a processor, the computer-readable storage medium is controlled by a device to execute the version updating method.
According to another aspect of the embodiments of the present invention, there is further provided a processor, wherein the processor is configured to execute a program, and the program executes to perform the version update method described above.
In the embodiment of the present invention, all service table data recorded in a preset database is obtained, wherein a service detail table and a service summary table are preset in the preset database, the service detail table is used for recording a service version number corresponding to each service table, the service summary table records a latest version number, then version numbers in the service detail table and the service summary table are updated based on all service table data to obtain an update result, then based on the update result, whether a version number in the service summary table is consistent with a total version number recorded by a memory cache end is judged, and if the version number in the service summary table is not consistent with the total version number recorded by the memory cache end, the total version number of the memory cache end is updated based on the version number in the service summary table. In this embodiment, a cache synchronization mechanism based on version numbers is implemented, and two version tables are pre-designed in a database: the total version number and the detail version number are respectively recorded in the memory cache, and then synchronous updating is carried out through the version numbers, so that the problem of data inconsistency caused by two-step data updating of the database and the memory cache is solved, and the technical problem of data inconsistency caused by the fact that data in the database and the data in the cache are updated separately in the related technology is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow diagram of an alternative version update method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an alternative memory synchronization policy based on version numbers according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an alternative version updating apparatus according to an embodiment of the present invention;
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The following embodiments of the present invention can be applied to the memory cache end of each service system, and the related service systems include but are not limited to: a banking business system, a personnel management business system, a logistics management business system and the like.
Optionally, in the following embodiments of the present invention, a version number atom auto-increment mechanism and a database change real-time capture mechanism (i.e., a trigger) are used, and data is updated synchronously through independent threads, so that not only the problem of inconsistency between data in a database and data in a memory cache can be avoided, but also a problem of distributed development and missed test can be avoided without depending on a specific service system, unified development and test can be performed, and meanwhile, centralized maintenance can be performed, reliability of the service system is provided, maintenance difficulty is reduced, and cache servers can be flexibly added and deleted and disaster recovery switching scenarios can be automatically used.
The present invention will be described in detail with reference to examples.
Example one
In accordance with an embodiment of the present invention, there is provided a version update method embodiment, it is noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a flowchart of an alternative version updating method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, acquiring all service table data recorded in a preset database, wherein a service detail table and a service summary table are preset in the preset database, the service detail table is used for recording a service version number corresponding to each service table, and the service summary table records a latest version number;
step S104, updating the version numbers in the service detail table and the service summary table based on all the service table data to obtain an updating result;
step S106, judging whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not based on the updating result;
step S108, if the version number in the service summary table is not consistent with the total version number recorded by the memory cache end, the total version number of the memory cache end is updated based on the version number in the service summary table.
Through the steps, all service table data recorded in the preset database can be acquired, wherein the preset database is preset with a service detail table and a service summary table, the service detail table is used for recording the service version number corresponding to each service table, the service summary table records the latest version number, then the version numbers in the service detail table and the service summary table are updated based on all service table data to obtain an update result, then whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not is judged based on the update result, and if the version number in the service summary table is not consistent with the total version number recorded by the memory cache end, the total version number of the memory cache end is updated based on the version number in the service summary table. In this embodiment, two version tables are pre-designed in the database based on the cache synchronization mechanism of the version number: the total version number and the detail version number are respectively recorded in the memory cache, and then synchronous updating is carried out through the version numbers, so that the problem of data inconsistency caused by two-step data updating of the database and the memory cache is solved, and the technical problem of data inconsistency caused by the fact that data in the database and the data in the cache are updated separately in the related technology is solved.
In this embodiment, when a cache data synchronization policy based on a version number is implemented, a version number atom auto-increment mechanism and a database change real-time capture mechanism (i.e., a trigger) are used, and data is updated synchronously through independent threads.
The following describes embodiments of the present invention in detail with reference to the above-mentioned respective implementation steps.
The embodiment of the invention is applied to the memory cache end of the business system, wherein the memory cache end is connected with the preset database.
Step S102, obtaining all service table data recorded in a preset database, wherein a service detail table and a service summary table are preset in the preset database, the service detail table is used for recording a service version number corresponding to each service table, and the service summary table records a latest version number.
In this embodiment, the database includes, but is not limited to: an Oracle database, an SQL Server database, a DB2 database, and the like, in this embodiment, two version tables are designed in the database, including: the system comprises a service summary table and a service detail table, wherein the service summary table records the highest version number in a whole library, namely the highest version number is equal to the version numbers of all the detail tables, and only one record is usually provided, so that the refreshing comparison efficiency is higher, the performance is better, the service detail table usually has a plurality of records, each record corresponds to one version number and data to be synchronized, and the initial value of all the version numbers is set to be 0.
Similarly, in this embodiment, the total version number and the detail version number are also recorded at the memory cache end, respectively. The total version number represents the version number of the global bus in the current memory cache, the detail version number is the specific version number of each type of cache data (usually, one service table corresponds to one type of cache data), and the initial value of all the version numbers is 0.
And step S104, updating the version numbers in the service detail list and the service summary list based on all the service list data to obtain an updating result.
Optionally, the step of updating the version number in the service detail table and the service summary table based on all the service table data to obtain the update result includes: analyzing whether a new service table appears in a preset database or not based on all service table data; if a new service table appears in the preset database, adding a new version number in the service list, wherein the added new version number corresponds to the new service table; and if the new version number is higher than the version number recorded in the service summary table, updating the version number in the service summary table into the new version number to obtain an updating result.
In this embodiment, if there is a change in the service table data in the database, the database trigger mechanism is used to increase the version number of the service table in the service detail table in the database in real time after each change, where when a new version number is added to the service detail table, the version number atomic self-increment mechanism is used, and if the updated version number of the service detail table is higher than the version number of the service summary table, the version number of the service summary table is also updated to the highest version number.
And step S106, judging whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not based on the updating result.
Optionally, after determining whether the version number in the service summary table is consistent with the total version number recorded in the memory cache, the version updating method further includes: if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end, querying the service detail table by adopting a preset control thread to obtain a new service table corresponding to the new version number; reading data stored in the new service table; and updating the data in the new service table to a memory cache end.
In this embodiment, the independent thread may compare whether the total version number of the service in the database is consistent with the total version number of the memory cache, where the independent thread may be scheduled by the database or may be scanned at regular time. If the version numbers are consistent, action is not needed, refreshing is continuously kept, and if the version numbers are not consistent, difference is judged, so that under the condition that the version numbers are consistent, only the total version number needs to be compared, performance loss caused by large-scale scanning of a database is avoided, comparison objects are simplified, and refreshing efficiency is improved.
In this embodiment, if the version number in the service summary table is not consistent with the total version number recorded by the memory cache end, the independent thread finds the corresponding database from the service detail table of the database according to the difference, caches the range of the difference of the version number into the memory, that is, reads out the data in the memory cache behind the range of the version number of the database, and then updates the latest database into the memory cache.
Optionally, the step of determining whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end based on the update result includes: comparing whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not by adopting a preset control thread; and if the version number in the business summary table is consistent with the total version number recorded by the memory cache end, continuing to execute version refreshing operation according to the version updating strategy.
In this embodiment, if the total version numbers are consistent, the refresh operation is maintained without other comparisons, which can avoid performance loss caused by a large amount of scanning from the database, simplify comparison objects, and improve refresh efficiency.
Step S108, if the version number in the service summary table is not consistent with the total version number recorded by the memory cache end, the total version number of the memory cache end is updated based on the version number in the service summary table.
In the strategy, even if occasional synchronous updating fails due to abnormal factors such as network jitter and the like, updating can be continued in next updating of the comparison version number, so that the strategy has the capability of automatically discovering data abnormity and repairing.
In this embodiment, after the memory cache is updated, the detail version number and the total version number in the memory cache are updated, where the total version number is equal to the version numbers of all the details.
In an optional implementation manner of the present invention, a total version number and a service detail version number are recorded in a memory cache, where the total version number is consistent with a latest version number recorded in a service summary table in a preset database, and the service detail version number is consistent with a service version number corresponding to each service table recorded in a service detail table in the preset database.
That is, in the memory cache, the current latest total version number and detail version number in the memory environment are also maintained.
In another optional implementation manner of the present invention, after updating the total version number of the memory cache terminal based on the version number in the service summary table, the version updating method further includes: updating the total version number of the memory cache end by adopting a preset control thread based on the version number in the service summary table; and updating the total version number of the cache end of the memory by adopting a thread control pool based on the version number recorded in the service detail table.
In the embodiment, the data is updated by presetting the control thread and the thread control pool, so that the data updating is independent of a specific service system, the service system is not invasive, the synchronous updating action is completed by the independent thread or the thread pool, the unified development and test are realized, the distributed development missing test is avoided, the centralized maintenance is realized, the reliability of the service system is improved, the maintenance difficulty is reduced, in addition, the machine is manufactured into a universal mechanism and can be used in a plurality of service systems, and the reusable popularization value is realized. The problem that the code of the service system is highly coupled and highly invaded in the prior art is solved.
Meanwhile, the implementation mode has flexible horizontal expansion capability of the cache server, if one or more cache servers need to be added, only a refreshing mechanism (thread or multithreading) needs to be started newly, the new cache server can be automatically loaded to the latest data according to the synchronous updating strategy, and the memory cache loading is not loading one by one, but loading in batches, so that the efficiency of the memory synchronous updating strategy is greatly improved.
Another optional version updating method further includes: judging whether a memory cache end enters a disaster recovery switching mode or not; and if the memory cache end enters the disaster recovery switching mode, automatically loading all currently cached data of the memory cache end to the new server by adopting a preset control thread.
In this embodiment, a disaster recovery switching scenario can be automatically used, and whether to update the cache is determined based on the difference of version numbers, so that in a multi-center multi-node disaster recovery deployment structure, when disaster recovery switching occurs, manual intervention and operation are not required, an independent thread can automatically load all cache data to a brand-new server (that is, the initial version number is 0) according to the difference of the version numbers, and the memory cache loading is not loading one by one, but loading in batch, thereby greatly improving the efficiency of the memory synchronous updating strategy, so that a new server memory after disaster recovery switching can be synchronized into the latest data within a short time, and the problem that a large amount of memories cannot be hit for a long time is avoided.
Fig. 2 is a schematic diagram of another optional memory synchronization policy based on version numbers according to an embodiment of the present invention, as shown in fig. 2, a cache synchronization mechanism based on version numbers is designed and defined with self-increment numbers as version numbers, and two version tables are designed in a database: the total table records the highest version number in the whole library, is equal to max (the version number of all details), and usually has only one record, so that the refreshing comparison efficiency is higher, the performance is better, the detail table usually has a plurality of records, each record corresponds to one version number and data to be synchronized, and the initial value of all the version numbers is 0. In the memory cache, a total version number and a detail version number are also recorded respectively, the total version number represents the version number of the global bus in the current memory cache, the detail version number is the specific version number of each type of cache data (usually, one service table corresponds to one type of cache data), and the initial value of all the version numbers is 0.
The specific memory synchronization strategy is as follows:
1. if the business table data in the database is changed, the version number of the business table in the detail table is added in the database in real time after each change through a database trigger mechanism, the version number is atomic self-increment, and if the updated detail version number is higher than the version number of the general table, the version of the general table is also updated to be the highest version number.
2. In the memory cache, the current latest total version number and detail version of the memory environment are also maintained.
3. The total version number in the database and the total version number of the memory cache are compared through independent threads (the database can be used for scheduling, and the timing scanning can be carried out), if the total version numbers are consistent, no action is carried out, the refreshing is continuously kept, and if the version numbers are inconsistent, the difference is judged, so that only the total version number needs to be compared under the condition that the version numbers are consistent, the performance loss caused by the large-scale scanning of the database is avoided, the comparison object is simplified, and the refreshing efficiency is improved.
4. And then the independent thread finds the corresponding database from the detail table in the database according to the difference, caches the difference range of the version number into the memory, namely, reads out the data which lags behind the range of the version number of the database in the memory cache, and then updates the latest database into the memory cache.
5. And after the memory cache is updated, updating the detail version number and the total version number in the memory cache. The total version number is equal to max (the version number of all details).
The memory cache synchronous updating strategy based on the version number has the following advantages:
1. the strategy is to compare a plurality of version numbers to determine whether to synchronize the data of the memory cache, thereby avoiding the problem of inconsistency caused by two-step writing in the prior art. In the mechanism, even if occasional synchronous updating fails due to abnormal factors such as network jitter and the like, updating can be continued in next updating of the comparison version number, so that the mechanism has the capability of automatically discovering data abnormity and repairing.
2. The strategy has independence, does not depend on a specific service system, has no invasion to the service system, is uniformly developed and tested by completing synchronous updating actions through independent threads or thread pools, avoids scattered development and missing test, can be maintained in a centralized manner, provides the reliability of the service system, reduces the maintenance difficulty, can be used in a plurality of service systems as a general strategy, and has reusable popularization value.
3. The strategy has flexible horizontal expansion capability of the cache server, if one or more cache servers are needed to be added, only a refreshing mechanism (thread or multithreading) needs to be started newly, the latest data can be automatically loaded on the new cache server according to the synchronous updating strategy, and the memory cache loading is not loading one by one, but loading in batch, so that the efficiency of the memory synchronous updating strategy is greatly improved.
4. The strategy can automatically use a disaster recovery switching scene, and whether the cache is updated or not is determined based on the difference of version numbers, so that in a multi-center multi-node disaster recovery deployment structure, when disaster recovery switching occurs, manual intervention and operation are not needed, an independent thread can automatically load all cache data to a brand-new server (the initial version number is 0) according to the difference of the version numbers, and the cache loading of the memory is not loading one by one but loading in batch, so that the efficiency of the strategy for synchronously updating the memory is greatly improved, the memory of the new server after disaster recovery switching can be synchronized into the latest data in a short time, and the defect that a large amount of the memory cannot be hit for a long time is avoided.
The invention is described below in connection with an alternative embodiment.
Example two
The embodiment of the present invention provides a version updating apparatus, and each implementation unit included in the updating apparatus corresponds to each implementation step in the first embodiment.
Fig. 3 is a schematic diagram of an alternative version updating apparatus according to an embodiment of the present invention, as shown in fig. 3, the version updating apparatus includes: an obtaining unit 30, a first updating unit 32, a judging unit 34, a second updating unit 36, wherein,
an obtaining unit 30, configured to obtain all service table data recorded in a preset database, where a service detail table and a service summary table are preset in the preset database, the service detail table is used to record a service version number corresponding to each service table, and the service summary table records a latest version number;
a first updating unit 32, configured to update the version numbers in the service detail table and the service summary table based on all service table data, so as to obtain an update result;
a determining unit 34, configured to determine, based on the update result, whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end;
the second updating unit 36 is configured to update the total version number of the memory cache end based on the version number in the service summary table if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end.
The version updating apparatus may obtain all service table data recorded in a preset database through the obtaining unit 30, where a service detail table and a service summary table are preset in the preset database, the service detail table is used to record a service version number corresponding to each service table, the service summary table records a latest version number, then, based on all service table data, the version numbers in the service detail table and the service summary table are updated through the first updating unit 32 to obtain an updating result, then, based on the updating result, the determining unit 34 determines whether the version number in the service summary table is consistent with the total version number recorded at the memory cache end, and if the version number in the service summary table is not consistent with the total version number recorded at the memory cache end, the total version number at the memory cache end is updated through the second updating unit 36 based on the version number in the service summary table. In this embodiment, a cache synchronization mechanism based on version numbers is designed and defined with self-increment numbers as version numbers, and two version tables are pre-designed in a database: the total version number and the detail version number are respectively recorded in the memory cache, and then synchronous updating is carried out through the version numbers, so that the problem of data inconsistency caused by two-step data updating of the database and the memory cache is solved, and the technical problem of data inconsistency caused by the fact that data in the database and the data in the cache are updated separately in the related technology is solved.
Optionally, the first updating unit includes: the first analysis module is used for analyzing whether a new business table appears in a preset database or not based on all the business table data; the first adding module is used for adding a new version number in the service detail list if a new service list appears in the preset database, wherein the added new version number corresponds to the new service list; and the first updating module is used for updating the version number in the business summary table into a new version number to obtain an updating result if the new version number is higher than the version number recorded in the business summary table.
Optionally, the version updating apparatus further includes: the first output module is used for inquiring the service detail table by adopting a preset control thread to obtain a new service table corresponding to the new version number if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end after judging whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end; the first reading module is used for reading the data stored in the new service table; and the second updating module is used for updating the data in the new service table to the memory cache end.
Optionally, the judging unit includes: the first comparison module is used for comparing whether the version number in the business summary table is consistent with the total version number recorded by the memory cache end by adopting a preset control thread; and the first execution module is used for continuing to execute version refreshing operation according to the version updating strategy if the version number in the service summary table is consistent with the total version number recorded by the memory cache end.
Optionally, a total version number and a service detail version number are recorded in the memory cache, where the total version number is consistent with a latest version number recorded in a service general table in the preset database, and the service detail version number is consistent with a service version number corresponding to each service table recorded in a service detail table in the preset database.
Optionally, the version updating apparatus further includes: the third updating module is used for updating the total version number of the memory cache end based on the version number in the service summary table by adopting a preset control thread after the total version number of the memory cache end is updated based on the version number in the service summary table; and the fourth updating module is used for updating the total version number of the memory cache end by adopting the thread control pool based on the version number recorded in the service detail table.
Optionally, the version updating apparatus further includes: the first judgment module is used for judging whether the memory cache end enters a disaster recovery switching mode or not; and the first loading module is used for automatically loading all currently cached data of the memory cache end to the new server by adopting a preset control thread if the memory cache end enters the disaster recovery switching mode.
The version updating apparatus may further include a processor and a memory, and the acquiring unit 30, the first updating unit 32, the determining unit 34, the second updating unit 36, and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.
The processor comprises a kernel, and the kernel calls a corresponding program unit from the memory. The kernel can be set to be one or more, whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not is judged by adjusting the kernel parameters, and if the version number in the service summary table is not consistent with the total version number recorded by the memory cache end, the total version number of the memory cache end is updated based on the version number in the service summary table.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, wherein the storage medium is used for storing a computer program, and the computer program, when executed by a processor, controls an apparatus where the computer-readable storage medium is located to execute the version updating method described above.
According to another aspect of the embodiments of the present invention, there is further provided a processor, wherein the processor is configured to execute a program, and the program executes the version update method.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: acquiring all service table data recorded in a preset database, updating the version numbers in the service detail table and the service summary table based on all the service table data to obtain an updating result, then judging whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end based on the updating result, and updating the total version number of the memory cache end based on the version number in the service summary table if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A version updating method is characterized in that the version updating method is applied to a memory cache end of a business system, the memory cache end is connected with a preset database, and the version updating method comprises the following steps:
acquiring all service table data recorded in the preset database, wherein a service detail table and a service summary table are preset in the preset database, the service detail table is used for recording a service version number corresponding to each service table, and the service summary table records a latest version number;
updating the version numbers in the service detail table and the service summary table based on all the service table data to obtain an updating result;
judging whether the version number in the service summary table is consistent with the total version number recorded by the memory cache end or not based on the updating result;
and if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end, updating the total version number of the memory cache end based on the version number in the service summary table.
2. The method according to claim 1, wherein the step of updating the version numbers in the service list and the service summary table based on all the service table data to obtain the updated result comprises:
analyzing whether a new service table appears in the preset database or not based on all the service table data;
if a new service table appears in the preset database, adding a new version number in the service detail table, wherein the added new version number corresponds to the new service table;
and if the new version number is higher than the version number recorded in the service summary table, updating the version number in the service summary table into the new version number to obtain the updating result.
3. The method according to claim 2, wherein after determining whether the version number in the service summary table is consistent with the total version number recorded in the memory cache, the version updating method further comprises:
if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end, querying the service detail table by adopting a preset control thread to obtain the new service table corresponding to the new version number;
reading data stored in the new service table;
and updating the data in the new service table to the memory cache end.
4. The method according to claim 1, wherein the step of determining whether the version number in the service summary table is consistent with the total version number recorded by the memory cache side based on the update result comprises:
adopting a preset control thread to compare whether the version number in the business summary is consistent with the total version number recorded by the memory cache end or not;
and if the version number in the service summary table is consistent with the total version number recorded by the memory cache end, continuing to execute version refreshing operation according to a version updating strategy.
5. The method according to claim 1, wherein a total version number and a service detail version number are recorded in the memory cache, wherein the total version number is consistent with a latest version number recorded in a service summary table in the preset database, and the service detail version number is consistent with a service version number corresponding to each service table recorded in a service detail table in the preset database.
6. The method according to claim 5, wherein after updating the total version number of the cache memory side based on the version number in the service summary table, the version updating method further comprises:
updating the total version number of the memory cache end by adopting a preset control thread based on the version number in the service summary table;
and updating the total version number of the memory cache terminal by adopting a thread control pool based on the version number recorded in the service detail table.
7. The method of claim 1, wherein the version update method further comprises:
judging whether the memory cache end enters a disaster recovery switching mode or not;
and if the memory cache end enters the disaster recovery switching mode, automatically loading all currently cached data of the memory cache end to a new server by adopting a preset control thread.
8. A version updating device is characterized in that the version updating device is applied to a memory cache end of a business system, the memory cache end is connected with a preset database, and the version updating device comprises:
an obtaining unit, configured to obtain all service table data recorded in the preset database, where a service detail table and a service summary table are preset in the preset database, the service detail table is used to record a service version number corresponding to each service table, and the service summary table records a latest version number;
a first updating unit, configured to update the version number in the service detail table and the service summary table based on all the service table data, so as to obtain an update result;
a judging unit, configured to judge whether a version number in the service summary table is consistent with a total version number recorded by the memory cache terminal based on the update result;
and the second updating unit is used for updating the total version number of the memory cache end based on the version number in the service summary table if the version number in the service summary table is inconsistent with the total version number recorded by the memory cache end.
9. A computer-readable storage medium, characterized in that the storage medium is used for storing a computer program, wherein the computer program, when executed by a processor, controls an apparatus in which the computer-readable storage medium is located to perform the version updating method according to any one of claims 1 to 7.
10. A processor, configured to run a program, wherein the program when running performs the version update method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110267621.6A CN112988708A (en) | 2021-03-11 | 2021-03-11 | Version updating method and device, computer readable storage medium and processor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110267621.6A CN112988708A (en) | 2021-03-11 | 2021-03-11 | Version updating method and device, computer readable storage medium and processor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112988708A true CN112988708A (en) | 2021-06-18 |
Family
ID=76335037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110267621.6A Pending CN112988708A (en) | 2021-03-11 | 2021-03-11 | Version updating method and device, computer readable storage medium and processor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988708A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024055343A1 (en) * | 2022-09-14 | 2024-03-21 | 武汉理工光科股份有限公司 | Data update method and apparatus, and electronic device and computer storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060219772A1 (en) * | 2005-04-01 | 2006-10-05 | Microsoft Corporation | Relaxed currency constraints |
| CN107133234A (en) * | 2016-02-29 | 2017-09-05 | 阿里巴巴集团控股有限公司 | The method of data cached renewal, apparatus and system |
| CN107301051A (en) * | 2017-06-27 | 2017-10-27 | 深圳市金立通信设备有限公司 | The caching of terminal dynamic data and exchange method, terminal, system and computer-readable recording medium |
| CN111475518A (en) * | 2020-03-26 | 2020-07-31 | 平安银行股份有限公司 | Parameter increment loading method and device, computer equipment and storage medium |
-
2021
- 2021-03-11 CN CN202110267621.6A patent/CN112988708A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060219772A1 (en) * | 2005-04-01 | 2006-10-05 | Microsoft Corporation | Relaxed currency constraints |
| CN107133234A (en) * | 2016-02-29 | 2017-09-05 | 阿里巴巴集团控股有限公司 | The method of data cached renewal, apparatus and system |
| CN107301051A (en) * | 2017-06-27 | 2017-10-27 | 深圳市金立通信设备有限公司 | The caching of terminal dynamic data and exchange method, terminal, system and computer-readable recording medium |
| CN111475518A (en) * | 2020-03-26 | 2020-07-31 | 平安银行股份有限公司 | Parameter increment loading method and device, computer equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024055343A1 (en) * | 2022-09-14 | 2024-03-21 | 武汉理工光科股份有限公司 | Data update method and apparatus, and electronic device and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8874515B2 (en) | Low level object version tracking using non-volatile memory write generations | |
| US9607001B2 (en) | Automated failover of a metadata node in a distributed file system | |
| CN108932295A (en) | Primary database method for handover control, device, computer equipment and storage medium | |
| CN111880956B (en) | Data synchronization method and device | |
| US20140019495A1 (en) | Processing a file system operation in a distributed file system | |
| CN110413685B (en) | Database service switching method, device, readable storage medium and computer equipment | |
| CN112468601B (en) | Data synchronization method, access method and system of distributed storage system | |
| CN110781214A (en) | Database reading and writing method and device, computer equipment and storage medium | |
| JP2007241486A (en) | Memory system | |
| CN111506253B (en) | Distributed storage system and storage method thereof | |
| US20210034477A1 (en) | Transaction recovery from a failure associated with a database server | |
| CN109460438A (en) | Message data storage method, device, computer equipment and storage medium | |
| CN104866388B (en) | Data processing method and device | |
| CN109726211B (en) | Distributed time sequence database | |
| CN112988708A (en) | Version updating method and device, computer readable storage medium and processor | |
| JP2019204527A (en) | Processing method and processing device of data position of storage equipment, computer equipment, and computer-readable storage medium | |
| CN111966531B (en) | Data snapshot method and device, computer equipment and storage medium | |
| CN110287164B (en) | Data recovery method and device and computer equipment | |
| CN112115166B (en) | Data caching method and device, computer equipment and storage medium | |
| US10168935B2 (en) | Maintaining access times in storage systems employing power saving techniques | |
| CN114461455A (en) | Method and device for repairing bad blocks of disk of stream replication cluster | |
| CN110297673B (en) | Method and storage medium for optimizing loading of memory data | |
| CN112699325A (en) | Method and system for guaranteeing data consistency through cache secondary elimination | |
| JPS62245348A (en) | Method and device for updating data base | |
| CN111176886A (en) | Database mode switching method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |