CN112131260A - Data query method and device - Google Patents
Data query method and device Download PDFInfo
- Publication number
- CN112131260A CN112131260A CN202011061410.9A CN202011061410A CN112131260A CN 112131260 A CN112131260 A CN 112131260A CN 202011061410 A CN202011061410 A CN 202011061410A CN 112131260 A CN112131260 A CN 112131260A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- cache
- cache region
- change rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000008859 change Effects 0.000 claims description 121
- 230000008569 process Effects 0.000 claims description 9
- 238000004904 shortening Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000010006 flight Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003139 buffering effect Effects 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/40—Business processes related to the transportation industry
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Business, Economics & Management (AREA)
- Computer Security & Cryptography (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Probability & Statistics with Applications (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
According to the data query method, the local cache of the client comprises the first cache region and the second cache region, wherein the data updating time of the second cache region is shorter than the updating time of the first cache region, and the data of the first cache region is updated according to the data of the second cache region. After receiving the query request, the client firstly judges whether the data cached locally is valid or not and then further determines which cache region has valid data when querying the data. When the data which is not matched in the local cache is determined or the data of the local cache is invalid, the data is directly read from the data server, and the accuracy of the query result is ensured.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a data query method and device.
Background
Clients of an airline company or a third party (e.g., a channel side) need to regularly access a data server of a data center, and whether data in the data server and data in the clients are consistent or not is a key factor influencing a flight status inquiry system to provide quick, convenient and reliable service for customers.
In the related art, for example, flight information query is taken as an example, an airline or a client of a third party caches data read from a data server locally, and when a client queries flight information, the client can directly query from the local cache. The cached data has a certain validity period, which is usually measured by taking a reference time as a standard, for example, if the reference time is 5 minutes, the validity period of the cached data is 5 minutes, and the cached data expires after 5 minutes. If the cache data is out of date, the client is required to acquire a new flight cache from the data server to the local, and the process is repeated to continuously update the local cache data, so that the cache data of the client is coordinated and consistent with the data in the data server.
In this way, by updating the local cache data with the reference time, the consistency between the cache data and the data in the data server is poor, which may result in low accuracy of the query result.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a data query method and apparatus, so as to solve the technical problems in the related art that the consistency between the cache data of the client and the data in the data server is poor, and the accuracy of the query result is low, and the disclosed technical solution is as follows:
in a first aspect, the present application provides a data query method, which is applied to a client side, where a local cache of the client side includes a first cache region and a second cache region, a data update time of the second cache region is less than a data update time of the first cache region, and data of the first cache region is updated according to data of the second cache region; the method comprises the following steps:
receiving a query request, and analyzing to obtain a query condition corresponding to the query request;
when it is determined that data matched with the query condition are stored in the local cache, judging whether the query time interval between the current query and the last query is smaller than a preset reference time;
if the query time interval is smaller than the preset reference time, judging whether the data change rate corresponding to the query is smaller than a reference change rate or not;
if the data change rate is smaller than the reference change rate, inquiring target data matched with the inquiry condition from the first cache region;
if the data change rate is larger than or equal to the reference change rate, inquiring target data matched with the inquiry condition from the second cache region;
and if the local cache does not have data matched with the query condition, or the query time interval is greater than or equal to the preset reference time, directly querying the target data matched with the query condition from a data server.
Optionally, after querying the target data matching the query condition from the first cache region, the method further includes:
querying target data matched with the query condition from the second cache region;
calculating cache data variation existing between the target data of the second cache region and the target data of the first cache region;
and when the cache data variation is larger than or equal to a preset threshold value, directly returning the target data inquired from the second cache region, and updating the data in the first cache region according to the data in the second cache region.
Optionally, after querying the target data matching the query condition from the data server, the method further includes:
and acquiring the latest data from the data server, and updating the latest data to the second cache region.
Optionally, after directly querying, if the query time interval is greater than or equal to the preset reference time, target data matching the query condition from a data server, the method further includes:
calculating to obtain the current data change rate according to the target data inquired from the data server and the data result inquired last time;
if the current data change rate is greater than or equal to the reference change rate, shortening the preset reference time;
and if the current data change rate is smaller than the reference change rate, increasing the preset reference time.
Optionally, after querying, if the data change rate is greater than or equal to the reference change rate, target data matching the query condition from the second cache region, the method further includes: and directly shortening the preset reference time.
Optionally, after querying, if the data change rate is smaller than the reference change rate, the target data matching the query condition from the first cache region, the method further includes: and directly increasing the preset reference time.
Optionally, the process of obtaining the data change rate of the query includes:
acquiring a first data sum which accords with the query condition of the query at this time and a second data sum which accords with the query condition of the query at the last time;
and calculating the difference value between the first data sum and the second data sum, and calculating the ratio of the difference value to the second data sum to obtain the data change rate corresponding to the query.
In a second aspect, the present application further provides a data query device, which is applied to a client, where a local cache of the client includes a first cache region and a second cache region, a data update time of the second cache region is less than a data update time of the first cache region, and data of the first cache region is updated according to data of the second cache region; the device comprises:
the query condition determining module is used for receiving the query request and analyzing to obtain a query condition corresponding to the query request;
the query interval judging module is used for judging whether the query time interval between the current query and the last query is less than the preset reference time or not when the data matched with the query condition is determined to be stored in the local cache;
the data change rate judging module is used for judging whether the data change rate corresponding to the query is smaller than the reference change rate or not when the query time interval is smaller than the preset reference time;
the first query module is used for querying target data matched with the query condition from the first cache region when the data change rate is smaller than the reference change rate;
the second query module is used for querying target data matched with the query condition from the second cache region when the data change rate is greater than or equal to the reference change rate;
and the third query module is used for directly querying the target data matched with the query condition from the data server when the local cache does not have the data matched with the query condition or the query time interval is greater than or equal to the preset reference time.
Optionally, after the first query module, the apparatus further comprises:
the fourth query module is used for querying the target data matched with the query condition from the second cache region;
a cache data variation calculation module, configured to calculate a cache data variation existing between the target data of the second cache region and the target data of the first cache region;
and the cache updating module is used for directly returning the target data inquired from the second cache region when the cache data variation is larger than or equal to a preset threshold value, and updating the data in the first cache region according to the data of the second cache region.
In a third aspect, the present application further provides a client, including a memory and a processor;
the memory stores program instructions, and the processor calls the program instructions stored in the memory to execute the data query method of any one of the first aspect.
According to the data query method, the local cache of the client comprises the first cache region and the second cache region, wherein the data updating time of the second cache region is shorter than the updating time of the first cache region, and the data of the first cache region is updated according to the data of the second cache region. After receiving a query request, a client firstly determines whether data matched with the query condition exists in a local cache; if the matched data is stored in the local cache, judging whether the data in the local cache is valid or not by judging whether the query time interval is smaller than preset reference time or not, if the data in the local cache is determined to be valid, continuously judging which cache region has valid data by judging whether the data change rate is smaller than the reference change rate or not, and if the data change rate is smaller than the reference change rate, directly querying the first cache region; and if the reference change rate is not less than the reference change rate, directly querying the second cache region. And if no matched data exists in the local cache or the query time interval is greater than the preset reference time, directly reading the data from the data server. According to the process, when the scheme is used for inquiring data, whether the data cached locally is valid or not is judged, and if the data cached locally is valid, the data cached in a cache region is further determined to be valid. When the data which is not matched in the local cache is determined or the data of the local cache is invalid, the data is directly read from the data server, and the accuracy of the query result is ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of a method of data query provided in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram of another method of data querying provided in accordance with an exemplary embodiment;
FIG. 3 is a schematic diagram of a data query device according to an exemplary embodiment;
FIG. 4 is a schematic block diagram of another data query device provided in accordance with an exemplary embodiment;
FIG. 5 is a schematic block diagram of another data query device provided in accordance with an illustrative embodiment;
fig. 6 is a schematic structural diagram of an electronic device according to an exemplary embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart of a data query method applied in a client according to an exemplary embodiment is provided.
The local cache of the client is provided with a first cache region and a second cache region, wherein the data updating time of the second cache region is less than the updating time of the first cache region, and the data of the first cache region is updated according to the data in the second cache region. That is, the data in the second cache region is updated according to the data in the data server, and the data in the first cache region is updated according to the data in the second cache region.
As shown in fig. 1, the method may include the steps of:
s110, receiving the query request, and analyzing to obtain the query condition corresponding to the query request.
The query request can be initiated by a user after corresponding operation is performed through an operation interface of the client.
After receiving a query request initiated by a user, the client further analyzes the query condition corresponding to the query request.
Taking the Flight Status inquiry System (AV) as an example, the user inputs inquiry conditions, such as a route number, an airline company, a trigger date, and a trigger time, when the client inquires Flight information locally.
The flight status query system is a core system which forms a complete journey for a single flight and queries the available status of flight slots, and is the only entry for AV query. The AV system is an open platform system based entirely on an open platform, and a data server in the AV system is also referred to as an AV server.
S120, judging whether data matched with the query condition is stored in the local cache; if yes, go to S130; if not, S170 is performed.
After the query condition of the query request is determined, whether the local cache stores the matched data or not is judged, if yes, the subsequent judging step is continued, and if not, the data server is accessed to query the data matched with the query condition.
S130, judging whether the query time interval between the current query and the last query is less than preset reference time or not; if so, go to S140; if not, S170 is performed.
If the local cache stores data matched with the query at this time, whether the data in the local cache is valid needs to be further judged, and specifically, whether the time interval between the local query and the last query exceeds the preset reference time is judged.
The system records the time corresponding to each query request, and the query time interval is the time difference between the query time and the last query.
The preset reference time is a time interval for updating the local cache according to the data server, and the preset reference time can be linearly and correspondingly set according to the data change condition of the system within a longer time (e.g., 3 years or 5 years). The data change rate and the reference time are in a reverse change trend, namely, the larger the data change rate is, the shorter the reference time is; the smaller the data change rate, the longer the reference time.
If the query time interval is smaller than the preset reference time, the data matched with the query condition in the local cache is considered to be valid, and further judgment is needed to be made as to which cache region the data is queried from.
S140, judging whether the data change rate corresponding to the query is smaller than a reference change rate or not; if yes, executing S150; if not, S160 is performed.
The data change rate corresponding to the query is a ratio of an absolute value of a difference between total data amounts of the query results obtained twice and the total data amount of the query result obtained last time, that is, the data change rate corresponding to the query, specifically, the data change rate corresponding to the local query is as shown in formula 1:
data change rate | (∑ this query- Σ last query) |/∑ last query (equation 1)
In formula 1, Σ this query represents the total data of this query, that is, the total amount of data that meets the query condition and is queried from the second cache region.
Σ the last query represents the total amount of data that meets the last query condition queried by the last query request, and the system records the query result of each query request, and the data source of the last query result may be different for different situations, and the query result may come from the data server, may also come from the second cache region, and may also come from the first cache region.
For example, taking the query for the bookable seats of the eligible flights as an example, Σ this query in this step is the sum of the numbers of bookable seats of the various slots of the flights eligible for this query in the local cache. Σ last query is the sum of the number of bookable seats in each bay of the flight that meets the last query during the last query.
The baseline rate of change may be a change in the overall data in the data server, for example, a change in all flight data in the flight status query system. Or, a baseline change of a certain type of data, for example, a change of data of a certain route in the flight status query system.
The larger the data change rate corresponding to the query is, the faster the data change is, and the data needs to be queried from the local cache with the newer data. On the contrary, if the change rate of the data corresponding to the query is smaller, the query precision can be satisfied by querying the data from the local cache with slower update.
S150, target data matched with the query condition is queried from the first cache region.
When the data change rate corresponding to the query is smaller than the reference change rate, the data cached in the first cache region with slower data updating can also meet the query requirement.
And S160, inquiring the target data matched with the inquiry condition from the second cache region.
When the data change rate corresponding to the current query is greater than the reference change rate, the data needs to be queried from the second cache region with faster data update.
And S170, inquiring the target data matched with the inquiry condition from the data server.
In an application scenario, when the local cache does not have data meeting the query condition, the data meeting the query condition is directly queried from the data server.
In another application scenario, if data meeting the query condition of this time originally exists in the local cache, but the query time interval is greater than or equal to the preset reference time, the data meeting the query condition of this time is also directly queried from the data server.
In the flight status query system, the data server is an AV server.
In the data query method provided in this embodiment, two cache regions are locally set at the client, which are a first cache region and a second cache region, respectively, data update time of the second cache region is less than data update time of the first cache region, and data of the first cache region is updated according to data of the second cache region. Through the cooperation of the two caches, the data queried by the user from the local cache can be closer to the data in the data server, so that the accuracy of the data queried by the local cache is improved. And when the local cache does not have data matched with the query condition, directly querying the data meeting the query condition from the data server. Thereby improving the accuracy of the query result.
Referring to fig. 2, a flow chart of another data query method according to an exemplary embodiment is provided. In this embodiment, after querying data from the first cache region, it is determined whether the data in the first cache region needs to be updated.
As shown in fig. 2, the method further includes the following steps after S150:
and S210, inquiring the target data matched with the inquiry condition from the second cache region.
After the data meeting the query condition of this time is queried from the first cache region, the data meeting the query condition of this time, namely the target data, is continuously queried from the second cache region.
S220, calculating the cache data variation between the target data of the second cache region and the target data of the first cache region.
Calculating to obtain a cache data variation according to the data queried from the first cache region and the data queried from the second cache region, where the specific cache data variation can be calculated by using the following formula:
the variation of the cache data is | (∑ second cache query- Σ first cache query) | (equation 2)
The larger the cache data variation indicates that the data in the first cache region is more different from the data in the second cache region, i.e. the data in the first cache region is older and may have failed, and the data in the first cache region needs to be updated.
And S230, when the cache data variation is larger than or equal to a preset threshold value, directly returning the target data inquired from the second cache region, and updating the data in the first cache region according to the data in the second cache region.
If the difference value between the total amount of data queried from the second cache region and the total amount of data queried from the first cache region is greater than or equal to the preset threshold value, it indicates that the data in the first cache region may have failed.
If the difference between the total amount of the data inquired by the two cache regions is smaller than the preset threshold value, the data in the first cache region is the newer data, and the data in the first cache region does not need to be updated.
In the data query method provided in this embodiment, after querying corresponding data from the first cache region, the corresponding data is compared with data queried from the second cache region, whether the data in the first cache region needs to be updated is determined, when a difference between total amounts of data queried by the two cache regions is greater than or equal to a preset threshold, it is determined that the data in the first cache region may have failed, and the data in the first cache region is updated according to the data in the second cache region, so as to improve accuracy of subsequent data query from the first cache region.
In addition, in another embodiment of the present application, the data queried from the data server may be updated into the second cache region. As shown in fig. 2, after S170, the method further includes the following steps:
and S240, updating the data in the second cache region according to the data in the data server.
This step includes the following two cases: one is that, the local cache does not have the data that basically meets the query condition, in this case, the latest data is obtained from the data server and updated to the second cache region, that is, the latest data read from the data server is added to the second cache region.
The other is that the local cache originally has data meeting the query condition, but the query time interval is greater than or equal to the preset reference time, the data in the local cache is considered invalid at this time, the latest data is obtained from the data server and updated into the second cache region, that is, the latest data read from the data server is used for replacing the original data in the second cache region.
In the data query method provided in this embodiment, after the data that meets the query condition is queried from the data server, the data in the local second cache region is updated according to the latest data in the data server, so that the data in the local cache is closer to the data in the data server, that is, the data in the local cache is more accurate, and the accuracy of the data that is subsequently queried from the local cache is further improved.
In another embodiment of the present application, after the data meeting the query condition is queried from the data server or the second cache region, the cache time interval of the local cache is adjusted. As shown in fig. 2, after S170, the method further includes the following steps:
adjusting the buffering time interval of the local buffer includes the following two cases:
if the query time interval is greater than the preset reference time, the data change rate of the query is further calculated, and whether the cache time interval of the local cache needs to be adjusted is determined according to the data change rate. In this case, as shown in fig. 2, after S170, the following steps may be further included:
s250, calculating to obtain the data change rate of the current query according to the target data queried from the data server and the data result queried last time;
s260, judging whether the data change rate of the current query is greater than or equal to the reference change rate, if so, executing S270; if not, S280 is performed.
The larger the data change rate is, the faster the data change in the system is, that is, the shorter the update interval of the local cache data is, therefore, it is necessary to shorten the preset reference time, that is, the update interval of the local cache data is shortened, and the update frequency of the local cache data is accelerated.
S270, the preset reference time is shortened, that is, the buffering time interval of the second buffer area is shortened.
Alternatively, after S160, that is, the query time interval is smaller than the preset time interval and the data change rate of the query is greater than the reference change rate, S270 is directly performed.
And S280, increasing the preset reference time.
If the data change rate is smaller than the reference change rate, the data change is slow, the data updating time interval of the local cache can be properly increased, the number of times of accessing the data server is reduced, and therefore the access pressure of the data server is reduced.
For example, in the spring festival or the five-one or eleven holidays, if each route is busy and flight information of each route changes faster, the preset reference time needs to be shortened. At ordinary times, the passengers on the flights are fewer, the change of flight information is relatively reduced, and the preset reference time can be prolonged.
In the data query method provided in this embodiment, after the query result of the current query is queried from the data server or the second cache region, the magnitude relationship between the data change rate and the reference change rate is continuously compared, and the preset reference time is dynamically adjusted according to the comparison result.
Corresponding to the data query method embodiment, the application also provides a data query device embodiment.
Fig. 3 is a schematic structural diagram of a data query device provided in an embodiment of the present application, where the data query device is applied to a client.
The local cache of the client comprises a first cache region and a second cache, the data updating time of the second cache region is shorter than that of the first cache region, and the data of the first cache region is updated according to the data of the second cache region.
As shown in fig. 3, the apparatus includes:
the query condition determining module 110 is configured to receive the query request and analyze the query request to obtain a query condition corresponding to the query request.
The query interval determining module 120 is configured to determine whether a query time interval between the current query and the previous query is smaller than a preset reference time when it is determined that data matching the query condition is stored in the local cache.
A data change rate determining module 130, configured to determine whether the data change rate corresponding to the current query is smaller than a reference change rate when the query time interval is smaller than the preset reference time.
In an embodiment of the present application, the process of obtaining the data change rate of the query includes:
acquiring a first data sum which accords with the query condition of the query at this time and a second data sum which accords with the query condition of the query at the last time;
and calculating the difference value between the first data sum and the second data sum, and calculating the ratio of the difference value to the second data sum to obtain the data change rate corresponding to the query.
And the first query module 140 is configured to query the target data matching the query condition from the first cache region when the data change rate is smaller than the reference change rate.
And the second query module 150 is configured to query, when the data change rate is greater than or equal to the reference change rate, target data matching the query condition from the second cache region.
And a third query module 160, configured to query, directly from the data server, target data that matches the query condition when there is no data that matches the query condition in the local cache, or the query time interval is greater than or equal to the preset reference time.
In the data query apparatus provided in this embodiment, the local cache of the client includes a first cache region and a second cache region, where the data update time of the second cache region is less than the update time of the first cache region, and the data in the first cache region is updated according to the data in the second cache region. After receiving a query request, a client firstly determines whether data matched with the query condition exists in a local cache; if the matched data is stored in the local cache, judging whether the data in the local cache is valid or not by judging whether the query time interval is smaller than preset reference time or not, if the data in the local cache is determined to be valid, continuously judging which cache region has valid data by judging whether the data change rate is smaller than the reference change rate or not, and if the data change rate is smaller than the reference change rate, directly querying the first cache region; and if the reference change rate is not less than the reference change rate, directly querying the second cache region. And if no matched data exists in the local cache or the query time interval is greater than the preset reference time, directly reading the data from the data server. According to the process, when the scheme is used for inquiring data, whether the data cached locally is valid or not is judged, and if the data cached locally is valid, the data cached in a cache region is further determined to be valid. When the data which is not matched in the local cache is determined or the data of the local cache is invalid, the data is directly read from the data server, and the accuracy of the query result is ensured.
Referring to fig. 4, a schematic structural diagram of another data query apparatus provided in the embodiment of the present application is shown, where the apparatus further includes, on the basis of the embodiment shown in fig. 3:
a fourth query module 210, configured to query the second cache for the target data matching the query condition.
The fourth query module 210 is connected to the first query module 140, and after the first query module 140 queries the target data matched with the query condition from the first cache region, the fourth query module is used to query the target data matched with the query condition from the second cache region.
A cache data variation calculating module 220, configured to calculate a cache data variation existing between the target data of the second cache region and the target data of the first cache region.
The cache updating module 230 is configured to, when the cache data variation is greater than or equal to a preset threshold, directly return the target data queried from the second cache region, and update the data in the first cache region according to the data in the second cache region.
The data query device provided in this embodiment, after querying corresponding data from the first cache region, compares the data with the data queried from the second cache region, determines whether the data in the first cache region needs to be updated, determines that the data in the first cache region may have failed when a difference between total amounts of data queried by the two cache regions is greater than or equal to a preset threshold, and updates the data in the first cache region according to the data in the second cache region, thereby improving accuracy of subsequent data query from the first cache region.
Referring to fig. 5, a schematic structural diagram of another data query apparatus provided in the embodiment of the present application is shown, where the apparatus further includes, on the basis of the embodiment shown in fig. 3:
and a second cache updating module 310, configured to obtain the latest data from the data server, and update the latest data to the second cache region.
And after the target data matched with the query condition is queried from the data server, the second cache updating module updates the second cache region according to the data queried from the data server.
In another embodiment of the present application, after the data meeting the query condition is queried from the data server or the second cache region, the cache time interval of the local cache is adjusted. As shown in figure 5 of the drawings,
a data change rate calculation module 410, configured to calculate a current data change rate according to target data queried from the data server and a data result queried last time;
a shortening module 420, configured to shorten the preset reference time when the current data change rate is greater than or equal to the reference change rate.
In another application scenario, if the data change rate judgment module judges that the data change rate is greater than or equal to the reference change rate, the shortening module is directly triggered to shorten the preset reference time.
An increasing module 430, configured to increase the preset reference time when the current data change rate is smaller than the reference change rate.
In another application scenario, if the data change rate judgment module judges that the data change rate is smaller than the reference change rate, the increase module is directly triggered to increase the preset reference time.
The data query device provided in this embodiment continues to compare the magnitude relationship between the data change rate and the reference change rate after querying the query result of this query from the data server or the second cache region, and dynamically adjusts the preset reference time according to the comparison result.
In another aspect, the present application also provides an electronic device, which may include a processor 510, a memory 520, a communication interface 530, an input unit 540, a display 550, and a communication bus 560, as shown in fig. 6.
The processor 510, the memory 520, the communication interface 530, the input unit 540, and the display 550 are all in communication with each other via a communication bus 560.
In the embodiment of the present application, the processor 510 may be a Central Processing Unit (CPU), an application specific integrated circuit, a digital signal processor, an off-the-shelf programmable gate array or other programmable logic device, etc.
The memory 520 is used to store one or more programs, which may include program code including computer operating instructions. Processor 510 may invoke a program stored in memory 520 to implement any of the data query methods described above.
In one possible implementation, the memory 520 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the computer, such as log data, user access data, and the like.
Further, the memory 520 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.
The communication interface 530 may be an interface of a communication module, such as an interface of a GSM module.
The present application may also include a display 540 and an input unit 550, and the like.
Of course, the structure of the electronic device shown in fig. 6 does not constitute a limitation to the application server in the embodiment of the present application, and in practical applications, the application server may include more or less components than those shown in fig. 6, or some components may be combined.
The electronic device in the embodiment of the present application may include, but is not limited to, a server, for example, a server cluster or an independent server. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
In another aspect, the present application further provides a storage medium executable by an electronic device, where the storage medium stores a program, and when the program is executed by the electronic device, the program causes the electronic device to execute any one of the data query methods described above.
While, for purposes of simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present invention is not limited by the illustrated ordering of acts, as some steps may occur in other orders or concurrently with other steps in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
It should be noted that technical features described in the embodiments in the present specification may be replaced or combined with each other, each embodiment is mainly described as a difference from the other embodiments, and the same and similar parts between the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The steps in the method of the embodiments of the present application may be sequentially adjusted, combined, and deleted according to actual needs.
The device and the modules and sub-modules in the terminal in the embodiments of the present application can be combined, divided and deleted according to actual needs.
In the several embodiments provided in the present application, it should be understood that the disclosed terminal, apparatus and method may be implemented in other manners. For example, the above-described terminal embodiments are merely illustrative, and for example, the division of a module or a sub-module is only one logical division, and there may be other divisions when the terminal is actually implemented, for example, a plurality of sub-modules or modules may be combined or integrated into another module, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules or sub-modules described as separate parts may or may not be physically separate, and parts that are modules or sub-modules may or may not be physical modules or sub-modules, may be located in one place, or may be distributed over a plurality of network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, each functional module or sub-module in the embodiments of the present application may be integrated into one processing module, or each module or sub-module may exist alone physically, or two or more modules or sub-modules may be integrated into one module. The integrated modules or sub-modules may be implemented in the form of hardware, or may be implemented in the form of software functional modules or sub-modules.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. The data query method is applied to a client side, wherein a local cache of the client side comprises a first cache region and a second cache region, the data update time of the second cache region is less than that of the first cache region, and the data of the first cache region is updated according to the data of the second cache region; the method comprises the following steps:
receiving a query request, and analyzing to obtain a query condition corresponding to the query request;
when it is determined that data matched with the query condition are stored in the local cache, judging whether the query time interval between the current query and the last query is smaller than a preset reference time;
if the query time interval is smaller than the preset reference time, judging whether the data change rate corresponding to the query is smaller than a reference change rate or not;
if the data change rate is smaller than the reference change rate, inquiring target data matched with the inquiry condition from the first cache region;
if the data change rate is larger than or equal to the reference change rate, inquiring target data matched with the inquiry condition from the second cache region;
and if the local cache does not have data matched with the query condition, or the query time interval is greater than or equal to the preset reference time, directly querying the target data matched with the query condition from a data server.
2. The method of claim 1, wherein after querying the first cache for target data matching the query condition, the method further comprises:
querying target data matched with the query condition from the second cache region;
calculating cache data variation existing between the target data of the second cache region and the target data of the first cache region;
and when the cache data variation is larger than or equal to a preset threshold value, directly returning the target data inquired from the second cache region, and updating the data in the first cache region according to the data in the second cache region.
3. The method according to claim 1, wherein after querying the target data matching the query condition from the data server, the method further comprises:
and acquiring the latest data from the data server, and updating the latest data to the second cache region.
4. The method according to claim 1, wherein after directly querying the target data matching the query condition from the data server if the query time interval is greater than or equal to the preset reference time, the method further comprises:
calculating to obtain the current data change rate according to the target data inquired from the data server and the data result inquired last time;
if the current data change rate is greater than or equal to the reference change rate, shortening the preset reference time;
and if the current data change rate is smaller than the reference change rate, increasing the preset reference time.
5. The method of claim 1, wherein after querying the second buffer for target data matching the query condition if the data change rate is greater than or equal to the baseline change rate, the method further comprises: and directly shortening the preset reference time.
6. The method of claim 1, wherein after querying the first buffer for target data matching the query condition if the data change rate is less than the baseline change rate, the method further comprises: and directly increasing the preset reference time.
7. The method according to any one of claims 1 to 6, wherein the process of obtaining the data change rate of the query comprises:
acquiring a first data sum which accords with the query condition of the query at this time and a second data sum which accords with the query condition of the query at the last time;
and calculating the difference value between the first data sum and the second data sum, and calculating the ratio of the difference value to the second data sum to obtain the data change rate corresponding to the query.
8. The data query device is applied to a client, wherein a local cache of the client comprises a first cache region and a second cache, the data update time of the second cache region is shorter than that of the first cache region, and the data of the first cache region is updated according to the data of the second cache region; the device comprises:
the query condition determining module is used for receiving the query request and analyzing to obtain a query condition corresponding to the query request;
the query interval judging module is used for judging whether the query time interval between the current query and the last query is less than the preset reference time or not when the data matched with the query condition is determined to be stored in the local cache;
the data change rate judging module is used for judging whether the data change rate corresponding to the query is smaller than the reference change rate or not when the query time interval is smaller than the preset reference time;
the first query module is used for querying target data matched with the query condition from the first cache region when the data change rate is smaller than the reference change rate;
the second query module is used for querying target data matched with the query condition from the second cache region when the data change rate is greater than or equal to the reference change rate;
and the third query module is used for directly querying the target data matched with the query condition from the data server when the local cache does not have the data matched with the query condition or the query time interval is greater than or equal to the preset reference time.
9. The apparatus of claim 8, wherein after the first query module, the apparatus further comprises:
the fourth query module is used for querying the target data matched with the query condition from the second cache region;
a cache data variation calculation module, configured to calculate a cache data variation existing between the target data of the second cache region and the target data of the first cache region;
and the cache updating module is used for directly returning the target data inquired from the second cache region when the cache data variation is larger than or equal to a preset threshold value, and updating the data in the first cache region according to the data of the second cache region.
10. A client comprising a memory and a processor;
the memory has stored therein program instructions that the processor invokes to perform the data query method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011061410.9A CN112131260A (en) | 2020-09-30 | 2020-09-30 | Data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011061410.9A CN112131260A (en) | 2020-09-30 | 2020-09-30 | Data query method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112131260A true CN112131260A (en) | 2020-12-25 |
Family
ID=73843465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011061410.9A Pending CN112131260A (en) | 2020-09-30 | 2020-09-30 | Data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112131260A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112783907A (en) * | 2020-12-31 | 2021-05-11 | 深圳云天励飞技术股份有限公司 | Data updating method, application end data updating method and related products |
| CN113779082A (en) * | 2021-01-05 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Method and device for updating data |
| CN116186109A (en) * | 2022-12-26 | 2023-05-30 | 中国长江电力股份有限公司 | Method for inquiring time sequence data with value changed by information system |
| CN116541420A (en) * | 2023-07-07 | 2023-08-04 | 上海爱可生信息技术股份有限公司 | Vector data query method |
| TWI831023B (en) * | 2021-06-28 | 2024-02-01 | 中國信託商業銀行股份有限公司 | Query system and its data update method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101458706A (en) * | 2008-11-11 | 2009-06-17 | 航旅信通（北京）信息技术有限公司 | Method and system for precisely enquiring GDS flight caching data |
| US20110055202A1 (en) * | 2009-08-31 | 2011-03-03 | Heimendinger Scott M | Predictive data caching |
| CN109101625A (en) * | 2018-08-13 | 2018-12-28 | 中国建设银行股份有限公司 | A kind of web cache method and device |
| CN109446448A (en) * | 2018-09-10 | 2019-03-08 | 平安科技（深圳）有限公司 | Data processing method and system |
| CN110362583A (en) * | 2018-04-08 | 2019-10-22 | 北京京东尚科信息技术有限公司 | A kind of data processing method and device for multi-data source |
| CN110598138A (en) * | 2018-06-12 | 2019-12-20 | 北京京东尚科信息技术有限公司 | Cache-based processing method and device |
| CN110989939A (en) * | 2019-12-16 | 2020-04-10 | 中国银行股份有限公司 | Data cache processing method, device and equipment and cache component |
-
2020
- 2020-09-30 CN CN202011061410.9A patent/CN112131260A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101458706A (en) * | 2008-11-11 | 2009-06-17 | 航旅信通（北京）信息技术有限公司 | Method and system for precisely enquiring GDS flight caching data |
| US20110055202A1 (en) * | 2009-08-31 | 2011-03-03 | Heimendinger Scott M | Predictive data caching |
| CN110362583A (en) * | 2018-04-08 | 2019-10-22 | 北京京东尚科信息技术有限公司 | A kind of data processing method and device for multi-data source |
| CN110598138A (en) * | 2018-06-12 | 2019-12-20 | 北京京东尚科信息技术有限公司 | Cache-based processing method and device |
| CN109101625A (en) * | 2018-08-13 | 2018-12-28 | 中国建设银行股份有限公司 | A kind of web cache method and device |
| CN109446448A (en) * | 2018-09-10 | 2019-03-08 | 平安科技（深圳）有限公司 | Data processing method and system |
| CN110989939A (en) * | 2019-12-16 | 2020-04-10 | 中国银行股份有限公司 | Data cache processing method, device and equipment and cache component |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112783907A (en) * | 2020-12-31 | 2021-05-11 | 深圳云天励飞技术股份有限公司 | Data updating method, application end data updating method and related products |
| CN113779082A (en) * | 2021-01-05 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Method and device for updating data |
| TWI831023B (en) * | 2021-06-28 | 2024-02-01 | 中國信託商業銀行股份有限公司 | Query system and its data update method |
| CN116186109A (en) * | 2022-12-26 | 2023-05-30 | 中国长江电力股份有限公司 | Method for inquiring time sequence data with value changed by information system |
| CN116186109B (en) * | 2022-12-26 | 2024-01-05 | 中国长江电力股份有限公司 | Method for inquiring time sequence data with value changed by information system |
| CN116541420A (en) * | 2023-07-07 | 2023-08-04 | 上海爱可生信息技术股份有限公司 | Vector data query method |
| CN116541420B (en) * | 2023-07-07 | 2023-09-15 | 上海爱可生信息技术股份有限公司 | Vector data query method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112131260A (en) | Data query method and device | |
| US11341097B2 (en) | Prefetching based on historical use and real-time signals | |
| CN113630329B (en) | Load balancing method and system based on user access amount and readable storage medium | |
| CN111475519B (en) | Data caching method and device | |
| EP3937022B1 (en) | Method and apparatus of monitoring interface performance of distributed application, device and storage medium | |
| CN112434015B (en) | Data storage method and device, electronic equipment and medium | |
| US10585807B2 (en) | Balanced cache for recently frequently used data | |
| CN110909022A (en) | Data query method and device | |
| US11636112B2 (en) | Updating cache data | |
| CN111078585B (en) | Memory cache management method, system, storage medium and electronic equipment | |
| CN114741335A (en) | Cache management method, device, medium and equipment | |
| CN112948498A (en) | Method and device for generating global identification of distributed system | |
| CN113111076A (en) | Data caching method, device, equipment and storage medium | |
| CN113485980A (en) | Data processing method and device, electronic equipment and storage medium | |
| US20230267119A1 (en) | Data retrieval systems and methods | |
| EP3550446A1 (en) | Updating cache data | |
| CN107526530B (en) | Data processing method and device | |
| CN113076067A (en) | Method and device for eliminating cache data | |
| CN109547552B (en) | API request processing method and device, storage medium and electronic equipment | |
| CN113760876A (en) | Data filtering method and device | |
| CN113037420A (en) | Reading time stamp obtaining method and device, electronic equipment and storage medium | |
| CN110727694A (en) | Data processing method and device, electronic equipment and storage medium | |
| US10979525B1 (en) | Selective preemptive cache population based on data quality for rapid result retrieval | |
| CN113360528A (en) | Data query method and device based on multi-level cache | |
| CN111309724A (en) | Method and system for processing big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |