CN116561147A - Data updating method and system - Google Patents
Data updating method and system Download PDFInfo
- Publication number
- CN116561147A CN116561147A CN202310543876.XA CN202310543876A CN116561147A CN 116561147 A CN116561147 A CN 116561147A CN 202310543876 A CN202310543876 A CN 202310543876A CN 116561147 A CN116561147 A CN 116561147A
- Authority
- CN
- China
- Prior art keywords
- data
- update
- updating
- cache database
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004590 computer program Methods 0.000 claims description 11
- 230000003111 delayed effect Effects 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000003203 everyday effect Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to the field of data updating technologies, and in particular, to a data updating method and system. According to the method and the device, the updated data in the specified time is recorded through the data updating state table, the updating frequency threshold is set, the cache database is built, some data which are higher in updating frequency and are not expired can be stored in the cache database, and when the data are updated, the data only need to be updated to the cache database, but are not required to be updated to the service database frequently, so that the effects of reducing the updating frequency of the service database and improving the system efficiency are achieved.
Description
Technical Field
The present disclosure relates to the field of data updating technologies, and in particular, to a data updating method and system.
Background
At present, the internet enters a big data age, massive data are updated every day, and for some updated data, the updated data are generally searched and updated in a service database through indexes, and because the data volume in the service database is large and the updated data volume is also large, a great deal of time is required to process the massive data during updating, so that the updating speed is low, and the situation needs to be further improved.
Disclosure of Invention
In order to solve the problem that the existing updating method needs to take a large amount of time to process mass data, so that the updating speed is slow, the application provides a data updating method and system, and the following technical scheme is adopted:
a data updating method, comprising the steps of:
acquiring the update times and the expiration time of each update data in a data update state table, wherein the update data in the specified time is recorded in the data update state table;
acquiring an update frequency threshold;
judging whether the update times are larger than the update times threshold, if the update times are larger than the update times threshold and the update data are not expired, updating the update data into a cache database;
under the condition that a user inquires target data, searching the target data in a cache database; searching the target data from a service database under the condition that the target data cannot be searched in the cache database; wherein the data volume in the cache database is smaller than the data volume in the service database.
By adopting the technical scheme, the data updating method records the updated data in the specified time through the data updating state table, sets the updating times threshold and establishes the cache database, compares the updating times with the updating times threshold after acquiring the updating times and the expiration time of each updating data in the service database, and when the updating times are larger than the updating times threshold, the updating frequency of the data is higher, if the data is not expired at the same time, the data is updated into the cache database, then when the data is inquired, the data is searched in the cache database with smaller data quantity, and when the target data is not found in the cache database, the data which is higher in updating frequency and not expired yet is searched in the cache database, and only the cache database is required to be updated and is not required to be updated into the service database frequently when the data is updated, thereby achieving the effects of reducing the updating times of the service database and improving the system efficiency.
Optionally, the method further comprises:
and when the update data has not expired, adding 1 to the update times of the update data, and deferring the expiration time of the update data for 1 day.
By adopting the technical scheme, when the update data is not expired, the update times of the update data are increased by one due to the current update, and then the expiration time of the update data is delayed for one day, so that the expiration time of the data which are frequently updated can be delayed along with the increase of the update times, and the data can be in an unexpired state for a long time, so that the data can be cached in a cache database for a long time and waiting to be called at any time.
Optionally, the method further comprises:
when the update data has expired, setting the update times of the update data to 1, and resetting the expiration time to the next preset expiration time, wherein the next preset time period is that the current time is delayed for 1 day.
By adopting the technical scheme, when the update data is out of date, the update times of the update data are counted again, the update times are set to be 1, then the expiration time is reset, the expiration time is set to be one day later, if the accumulated update times of the data before the next expiration are more, the update data can be cached in the cache database when the update times threshold can be exceeded.
Optionally, the method further comprises:
and when the update times of the update data are not more than the update times threshold value or the update data are outdated, updating the update data in a service database.
By adopting the technical scheme, when the update times of the update data is not more than the update times threshold or the update data is expired, the update data is not updated into the cache database, but the update data is still updated in the service database, so that the data acquired from the service database next time is the latest data.
Optionally, the method further comprises:
acquiring the total data amount of the cache database;
when the total data amount of the cache database is larger than the preset maximum data amount, increasing the updating frequency threshold;
and when the total data amount of the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold.
By adopting the technical scheme, the data total amount of the cache database is obtained, when the data total amount in the cache database is larger than the preset maximum data total amount, the updating frequency threshold is increased, and a part of data with fewer updating frequencies is reduced to be updated in the cache database; when the total data amount in the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold value, and adding a part of data to update into the cache database; therefore, the data volume of the cache database is in a reasonable range, and the faster searching of the data is realized.
Optionally, the method further comprises:
and synchronizing the data in the cache database to a service database and clearing the data of the data update state table and the cache database when the update times threshold is increased, the update times threshold is reduced or the data volume of the data update state table exceeds a preset range.
By adopting the technical scheme, when the threshold value is updated each time, or the data quantity of the data updating state table exceeds the preset range, the data of the cache database is synchronized into the service database, so that the data information is synchronized timely, and the data updating state table and the cache database are emptied at the same time, so that the problem that the data searching is slower due to excessive data in the data updating state table and the cache database is prevented.
In a second aspect, the present application provides a data update system, comprising:
the updating data acquisition module is used for acquiring the updating times and the expiration time of each updating data in the data updating state table, wherein the updating data in the specified time is recorded in the data updating state table;
the updating frequency threshold value acquisition module is used for acquiring the updating frequency threshold value;
the cache database updating module is used for judging whether the updating times are larger than the updating times threshold, and if the updating times are larger than the updating times threshold and the updating data are not expired, updating the updating data into the cache database;
the searching module is used for searching the target data in the cache database under the condition that a user inquires the target data; searching the target data from a service database under the condition that the target data cannot be searched in the cache database; wherein the data volume in the cache database is smaller than the data volume in the service database.
Optionally, the data updating system further includes:
and the unexpired data parameter setting module is used for adding 1 to the update times of the update data when the update data are not expired, and deferring the expiration time of the update data by 1 day.
Optionally, the data updating system further includes:
and the expiration data parameter setting module is used for setting the update times of the update data to be 1 when the update data are expired, resetting the expiration time to be the next preset expiration time, and deferring the current time for 1 day later.
In a third aspect, the present application provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the data updating method described above when executing the computer program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data update method described above.
In summary, the present application includes at least one of the following beneficial technical effects:
1. according to the method, updated data in a specified time is recorded through the data updating state table, and the updating frequency threshold value is set and the cache database is built, so that data which are higher in updating frequency and are not expired can be stored in the cache database, and only the cache database is required to be updated when the data are updated, but the data are not required to be updated into the service database frequently, and therefore the effects of reducing the updating frequency of the service database and improving the system efficiency are achieved;
2. when the update data is not expired, the update times of the update data are increased by one, and then the expiration time of the update data is delayed for one day, so that the expiration time of the data which are frequently updated can be delayed along with the increase of the update times, and the data are in an unexpired state for a long time, so that the data can be cached in a cache database for a long time and waiting to be called at any time;
3. according to the method, the data total amount of the cache database is obtained, when the data total amount in the cache database is larger than the preset maximum data total amount, the updating frequency threshold is increased, and a part of data with fewer updating frequencies is reduced to be updated in the cache database; when the total data amount in the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold value, and adding a part of data to update into the cache database; therefore, the data volume of the cache database is in a reasonable range, and the faster searching of the data is realized.
Drawings
FIG. 1 is an explanatory view of an understanding of a data warehouse and a business database in the related art;
FIG. 2 is an understanding schematic diagram of a data update method in an embodiment of the present application;
FIG. 3 is an exemplary flow chart of a method for data update according to an embodiment of the present application;
FIG. 4 is an understanding of the number of updates and expiration time updates of data in an embodiment of the present application;
FIG. 5 is another exemplary flow chart of a method for data update according to an embodiment of the present application;
FIG. 6 is a schematic block diagram of a data update system according to an embodiment of the present application;
fig. 7 is an internal structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The terminology used in the following embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates to the contrary. It should also be understood that the term "and/or" as used in this application is intended to encompass any or all possible combinations of one or more of the listed items.
The terms "first," "second," and the like, are used below for descriptive purposes only and are not to be construed as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature, and in the description of embodiments of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.
The Internet enters a big data age, massive data are updated every day, and for some updated data, the updated data are generally searched and updated through indexes or a brand new data table is generated, and the two modes take a great deal of time to search the original data in the massive data in the data warehouse.
In the related art, a service database is built, and recently updated data is synchronized from a data warehouse to the service database and stored in the service database.
Referring to FIG. 1, a schematic diagram of an understanding of a data warehouse and a business database in the related art is shown in FIG. 1. There may be more than one billion or even one billion data in a data warehouse, where at least more than 100 tens of thousands of data updates are made per day, and in the related art, a business database is established for storing the most recently updated data.
However, even if the service database is established to store the latest updated data, the data amount in the service database is large, for example, the service database stores updated data within one week, and if the data amount updated every day is more than 100 ten thousand, more than 700 ten thousand data are stored in the service database. Thus, more time is wasted when the data needs to be searched and updated.
Therefore, the application provides a data updating method and system, update data in a specified time is recorded through a data updating state table, and data which is higher in updating frequency and is not expired can be stored in a cache database through setting an updating frequency threshold value and establishing the cache database, and when the data is updated, only the cache database is required to be updated, and the data is not required to be updated into a service database frequently, so that the effects of reducing the updating frequency of the service database and improving the system efficiency are achieved.
Referring to fig. 2, an understanding schematic diagram of a data updating method in an embodiment of the present application is shown.
As shown in fig. 2, after the service database is established, update data within a specified time is recorded through a data update status table in which an update number and an expiration time are set in each piece of data. For example, id in the figure indicates the number of data, name indicates data, num indicates the number of updates of data, and expire_time indicates the expiration time of data.
In one embodiment, there are 4 pieces of data in the data update status table, and the system updates the data into the cache database redis when the conditions are met and updates the data into the service database when the conditions are not met by judging whether the update times and the expiration time are met. And then when the user searches data, searching in the redis first, and if the data does not exist in the redis, searching in the service database, thereby updating the service database for times and improving the system efficiency.
Referring to fig. 3, an exemplary flow chart of a data updating method according to an embodiment of the present application is shown.
A data updating method, comprising the steps of:
s310, acquiring the update times and the expiration time of each update data in the data update state table.
Wherein the data update status table records updated data within a prescribed time, stores recently updated data, and records an expiration time of each piece of data, and the number of updates per piece of data before the expiration time, wherein the prescribed time may be 1 month.
S320, acquiring an update times threshold.
The system is preset with an updating frequency threshold value for comparing with the updating frequency of each data, so as to carry out a subsequent corresponding data processing mode according to the comparison result. In the present embodiment, the update number threshold is set to 5 times.
S330, judging whether the update times are larger than an update times threshold.
S340, judging whether the update data is expired.
The system judges whether the update times of the data are larger than an update times threshold value or not, namely whether the update times of the data are larger than 5 times or not; and determining the data processing mode by determining whether the updated data has expired, i.e. whether the current date exceeds the expiration time recorded in the service database.
S350, when the update times are larger than the update times threshold value and the update data are not expired, updating the update data into the cache database.
The method comprises the steps of establishing a cache database in advance, wherein the cache database is specially used for storing data which is updated frequently recently, comparing the update times of the updated data with an update times threshold value, and updating the data into the cache database when the update times are greater than 5 and the current data does not reach the expiration time.
S360, under the condition that a user inquires target data, searching the target data in the cache database, and under the condition that the target data cannot be searched in the cache database, searching the target data from the service database.
Wherein, because only the data which is updated frequently recently and is not expired is stored in the cache database, the data volume in the cache database is far smaller than the data volume in the service database. In this embodiment, when data is queried, the data is first searched in a cache database with smaller data size, and when the target data cannot be found in the cache database, the data is searched in a service database. When the data is found in the cache database, the data is directly updated in the cache database, and the data is not required to be updated in the service database, so that the updating times of the service database can be greatly reduced, and the data with higher updating frequency can be found in the cache database, so that the searching speed is improved, and the effect of improving the system efficiency is achieved.
And when the update data cannot meet the condition that the update times are larger than the update times threshold value at the same time and the update data are not expired, adopting other data processing modes.
And S370, when the update times of the update data are not more than the update times threshold value, or the update data are expired, updating the update data in the service database.
For data with infrequent updating times or data with fewer updating times in a near-period time, in order to not occupy the space of the cache database, the updating data is not updated into the cache database, but the updating data still needs to be updated in the service database, so that the data acquired from the service database next time is the latest data.
In the above embodiment, the number of updates and the expiration time of the data are not kept unchanged, but change in real time as the data update proceeds, and the change manner is different for the expired data and the unexpired data.
Referring to fig. 4, a schematic diagram is shown for an understanding of the update times and expiration time of data in the embodiment of the present application.
The data updating method further comprises the following steps:
when the update data has not expired, adding 1 to the update times of the update data, and delaying the expiration time of the update data for one day;
when the update data has expired, the update times of the update data are set to 1, and the expiration time is reset to the next preset expiration time, and the next preset time period is delayed by 1 day after the current time.
As shown in FIG. 4, there are pieces of data as in FIG. 4 in the data update status table, A, B, AAAOR, RRRTTT, A expiration time 2022/12/28, B expiration time 2022/12/28, AAAOR expiration time 2022/12/28, RRRTTT expiration time 2022/12/28. If 4 data are updated on day 25 of 11 in 2022, for A and RRRTTT, the current time does not exceed the expiration time, so the data are not expired, the number of times of updating the data is added with 1, and the expiration time is postponed for 1 day; for B and AAAOR, since the current time has exceeded the expiration time, the data has expired, and therefore the number of times the data is updated is reset to 1, and the expiration time is set to 2022/11/26.
Through the arrangement, as for the data A and the RRRTTT, the expiration time can be delayed along with the increase of the update times, so that the data A and the RRRTTT are in an unexpired state for a long time, and therefore, the data can be cached in a cache database for a long time and waiting to be called at any time;
for the data B and the AAAOR, when the update data is out of date, the update times of the update data are counted again, the update times are set to be 1, then the expiration time is reset, the expiration time is set to be one day later, if the accumulated update times of the data before the next expiration are more, the update data can be cached in a cache database when the update times threshold can be exceeded, otherwise, the storage resources of the cache database are not necessarily occupied.
In addition, since a satisfies the number of updates greater than 5 and has not expired, a is updated into the cache database and other data is updated in the traffic database.
In the above embodiment, by setting the update times threshold and establishing the cache database, some data which has higher update frequency and has not expired yet is stored in the cache database, and only needs to be updated to the cache database when the data is updated, but does not need to be updated to the service database frequently, thereby achieving the effects of reducing the update times of the service database and improving the system efficiency. In some embodiments, it is desirable to keep the data size of the cache database within a suitable range, so that the data is retrieved from the cache database more quickly, and more quickly.
Referring to fig. 5, another exemplary flow chart of a data updating method according to an embodiment of the present application is shown.
Optionally, in some embodiments, the data updating method further includes:
s380, acquiring the total data amount of the cache database.
And S390, when the total data amount of the cache database is larger than the preset maximum data amount, increasing the updating frequency threshold.
And S400, when the total data amount of the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold.
The method comprises the steps of obtaining the total data amount of a cache database, changing an update frequency threshold value based on the total data amount, increasing the update frequency threshold value when the total data amount in the cache database is larger than a preset maximum total data amount, and reducing a part of data with fewer update times to be updated in the cache database; when the total data amount in the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold value, and adding a part of data to update into the cache database; therefore, the data volume of the cache database is in a reasonable range, and the faster searching of the data is realized.
Optionally, in some embodiments, the method further comprises: when the updating times threshold value is increased, the updating times threshold value is reduced or the data quantity of the data updating state table exceeds a preset range, synchronizing the data in the cache database into the service database, and clearing the data of the data updating state table and the cache database; therefore, the data information is synchronized in time, and the data update state table and the data of the cache database are emptied at the same time, so that the problem that the data search is slower due to excessive data in the data update state table and the cache database is prevented.
In a second aspect, the present application provides a data updating system, and the data updating system of the present application is described below in conjunction with the above data updating method. Referring to fig. 6, fig. 6 is a schematic block diagram of a data updating system according to an embodiment of the present application.
The data updating system includes:
an update data obtaining module 610, configured to obtain the update times and expiration times of each update data in a data update status table, where the update data in a specified time is recorded in the data update status table;
an update times threshold obtaining module 620, configured to obtain an update times threshold;
the cache database updating module 630 is configured to determine whether the update times is greater than an update times threshold, and update the update data to the cache database if the update times is greater than the update times threshold and the update data has not expired;
the searching module 640 is configured to, when a user queries the target data, first search the cache database for the target data; searching target data from the service database when the target data cannot be searched in the cache database; wherein the data volume in the cache database is smaller than the data volume in the service database.
Optionally, in some embodiments, the data updating system further comprises:
the unexpired data parameter setting module 650 is configured to add 1 to the update times of the update data and delay the expiration time of the update data by 1 day when the update data has not expired.
Optionally, in some embodiments, the data updating system further comprises:
the expiration data parameter setting module 660 is configured to set the update number of times of the update data to 1 when the update data has expired, and reset the expiration time to a next preset expiration time, where the next preset time period is that the current time is postponed for 1 day.
Optionally, in some embodiments, the data updating system further comprises:
the service database updating module 670 is configured to update the update data in the service database when the update number of times of the update data is not greater than the update number threshold, or the update data has expired.
Optionally, in some embodiments, the data updating system further comprises:
a data total amount obtaining module 680, configured to obtain a data total amount of the cache database;
the update times threshold adjustment module 690 is configured to increase the update times threshold when the total data amount of the cache database is greater than the preset maximum data amount; and when the total data amount of the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold.
In one embodiment, the present application provides a computer device, which may be a server, whose internal structure may be as shown in fig. 7. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data updating method.
It will be appreciated by those skilled in the art that the structure shown in fig. 7 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The foregoing are all preferred embodiments of the present application, and are not intended to limit the scope of the present application in any way, therefore: all equivalent changes in structure, shape and principle of this application should be covered in the protection scope of this application.
Claims (10)
1. A data updating method, comprising the steps of:
acquiring the update times and the expiration time of each update data in a data update state table, wherein the update data in the specified time is recorded in the data update state table;
acquiring an update frequency threshold;
judging whether the update times are larger than the update times threshold, if the update times are larger than the update times threshold and the update data are not expired, updating the update data into a cache database;
under the condition that a user inquires target data, searching the target data in a cache database; searching the target data from a service database under the condition that the target data cannot be searched in the cache database; wherein the data volume in the cache database is smaller than the data volume in the service database.
2. The data updating method according to claim 1, characterized in that the method further comprises:
and when the update data has not expired, adding 1 to the update times of the update data, and deferring the expiration time of the update data for 1 day.
3. The data updating method according to claim 2, characterized in that the method further comprises:
when the update data has expired, setting the update times of the update data to 1, and resetting the expiration time to the next preset expiration time, wherein the next preset time period is that the current time is delayed for 1 day.
4. The data updating method according to claim 1, characterized in that the method further comprises:
when the update times of the update data are not more than the update times threshold value or the update data are out of date, the update data are updated in a service database, and the corresponding update data are deleted in a cache database.
5. The data updating method according to claim 1, characterized in that the method further comprises:
acquiring the total data amount of the cache database;
when the total data amount of the cache database is larger than the preset maximum data amount, increasing the updating frequency threshold;
and when the total data amount of the cache database is smaller than the preset minimum data amount, reducing the updating frequency threshold.
6. The data updating method according to claim 5, characterized in that the method further comprises:
and synchronizing the data in the cache database to a service database and clearing the data of the data update state table and the cache database when the update times threshold is increased, the update times threshold is reduced or the data volume of the data update state table exceeds a preset range.
7. A data updating system, comprising:
the updating data acquisition module is used for acquiring the updating times and the expiration time of each updating data in the data updating state table, wherein the updating data in the specified time is recorded in the data updating state table;
the updating frequency threshold value acquisition module is used for acquiring the updating frequency threshold value;
the cache database updating module is used for judging whether the updating times are larger than the updating times threshold, and if the updating times are larger than the updating times threshold and the updating data are not expired, updating the updating data into the cache database;
the searching module is used for searching the target data in the cache database under the condition that a user inquires the target data; searching the target data from a service database under the condition that the target data cannot be searched in the cache database; wherein the data volume in the cache database is smaller than the data volume in the service database.
8. The data update system of claim 7, wherein the data update system further comprises:
and the unexpired data parameter setting module is used for adding 1 to the update times of the update data when the update data are not expired, and deferring the expiration time of the update data by 1 day.
9. The data update system of claim 7, wherein the data update system further comprises:
and the expiration data parameter setting module is used for setting the update times of the update data to be 1 when the update data are expired, resetting the expiration time to be the next preset expiration time, and deferring the current time for 1 day later.
10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized by: the processor, when executing the computer program, implements the steps of the data updating method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310543876.XA CN116561147A (en) | 2023-05-15 | 2023-05-15 | Data updating method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310543876.XA CN116561147A (en) | 2023-05-15 | 2023-05-15 | Data updating method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116561147A true CN116561147A (en) | 2023-08-08 |
Family
ID=87501514
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310543876.XA Pending CN116561147A (en) | 2023-05-15 | 2023-05-15 | Data updating method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116561147A (en) |
-
2023
- 2023-05-15 CN CN202310543876.XA patent/CN116561147A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107943718B (en) | Method and device for cleaning cache file | |
| CN107491523B (en) | Method and device for storing data object | |
| US10489476B2 (en) | Methods and devices for preloading webpages | |
| EP3229142A1 (en) | Read cache management method and device based on solid state drive | |
| US8738593B2 (en) | Method and apparatus for reducing index sizes and increasing performance of non-relational databases | |
| EP2541423A1 (en) | Replacement policy for resource container | |
| CN110471939A (en) | Data access method, device, computer equipment and storage medium | |
| CN111339200A (en) | Data verification method and data verification system based on database synchronization | |
| CN111309650A (en) | Cache control method, device, storage medium and equipment | |
| CN110232074A (en) | Flow data and dimension table correlating method and stream calculation device | |
| US9094225B1 (en) | Discovery of short-term and emerging trends in computer network traffic | |
| CN108874930A (en) | File attribute information statistical method, device, system, equipment and storage medium | |
| CN107786668B (en) | Weight caching website method based on CDN (content delivery network) | |
| JPH11102366A (en) | Retrieval method and retrieval device | |
| CN110647542A (en) | Data acquisition method and device | |
| CN110955639A (en) | Data processing method and device | |
| CN117130792B (en) | Processing method, device, equipment and storage medium for cache object | |
| CN111913913B (en) | Access request processing method and device | |
| CN116561147A (en) | Data updating method and system | |
| CN109992708B (en) | Method, device, equipment and storage medium for metadata query | |
| US8028011B1 (en) | Global UNIX file system cylinder group cache | |
| CN113885801A (en) | Memory data processing method and device | |
| CN114493875A (en) | Transaction execution method, computer device, and storage medium | |
| CN114895850A (en) | Method for optimizing writing of data lake | |
| CN109948098B (en) | Data filtering method, and method and device for recording cache information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |