CN112187849A - Cache data dynamic generation method and system - Google Patents
Cache data dynamic generation method and system Download PDFInfo
- Publication number
- CN112187849A CN112187849A CN202010784802.1A CN202010784802A CN112187849A CN 112187849 A CN112187849 A CN 112187849A CN 202010784802 A CN202010784802 A CN 202010784802A CN 112187849 A CN112187849 A CN 112187849A
- Authority
- CN
- China
- Prior art keywords
- data
- version number
- function
- acquiring
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000006870 function Effects 0.000 claims description 80
- 238000004364 calculation method Methods 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 16
- 230000008859 change Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 7
- 238000011160 research Methods 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000006855 networking Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 241000209140 Triticum Species 0.000 description 2
- 235000021307 Triticum Nutrition 0.000 description 2
- 235000013339 cereals Nutrition 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/10—Flow control between communication endpoints
- H04W28/14—Flow control between communication endpoints using intermediate storage
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
Abstract
The embodiment of the invention discloses a method and a system for dynamically generating cache data, wherein the method comprises the following steps: acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data; acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list; and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list. The functional version number in the embodiment of the invention is the version number of the source code level, is directly controlled by research personnel, and is simple and effective. The functional identification is easy to identify in the cache system, which is beneficial for research personnel to troubleshoot problems; the cached KEY can be automatically switched during function upgrading and data changing, so that the problems of data leakage, confusion of old data and new data and the like caused by manual intervention are avoided, data display is changed instantly, and all modules are unified.
Description
Technical Field
The invention relates to the technical field of computer data processing, in particular to a method and a system for dynamically generating cache data.
Background
With the development of the internet and information industry, a large amount of fragmentary data records, such as transaction flow, consumption browsing habits, equipment position information and temperature sampling information, are generated in the enterprise operation and production process. The number and growth rate far exceed the load-bearing capacity of the enterprise infrastructure. How to effectively utilize the valuable digital assets and provide reliable and time-efficient data support for strategic decisions of enterprises is a matter of concern at every enterprise moment.
Big data has four main aspects: the method for processing the data in the real world includes the following common methods of high-speed processing, such as Volume (Volume), high-speed (Velocity), diversity (Variety) and authenticity (Veracity): and the capacity of computing is improved, such as single-machine multi-process, multi-thread or multi-machine cloud computing. Or optimized calculation structure, such as sub-table sub-base, wide table storage, row-column (vertical and horizontal) record storage, time series, document type or relational database. Or deriving a calculation mode (less used), and deriving an approximate function by using a non-precise formula. Optimized physical storage, RAID array, SSD disk, front SSD cache and memory cache.
In summary, the memory cache is used as a supplement to the application service operation, and has the characteristics of low cost, easy implementation and good framework integration compared with other technologies.
The access amount of the application service is increased, the query requests for the same resource may come from different client entities, and if data caching is not performed, all the requests are independently and parallelly calculated, so that the competition of IO resources and the waste of CPU calculation are caused. The common caching method is as follows:
for WEB application, when a user requests data, a service interface searches for a cache through a preset cache KEY, if the cache exists, the data are directly returned, if the cache does not exist, calculation is immediately carried out, a calculation result is cached to preset time, and the calculation result is returned to the user.
For applications with more data records and high access number, data are often fragmented by ID and stored in different cache nodes in a split manner, so that the data area and capacity of the cache are increased.
The two common caching schemes can obviously improve the service response speed and reduce the resource consumption, but still have the defects that:
the cache KEY is preset. When the system function needs to be upgraded and the cache needs to be cleared, the Key needs to be manually cleared or replaced, so that errors in operation easily occur, and the communication cost between research personnel and deployment personnel is increased. For list pages and second-level detail pages with more query conditions, cache cannot be set.
If the cache does not exist, the calculation is immediately carried out. If the waiting time required by the calculation is long and the number of the same requests is increased at the moment, all the calculations are performed at the same time, so that the situations of resource shortage, overhigh load and system false death are caused, and the stability of the system is reduced.
And caching the calculation result to preset time. For the same data resource, there are multiple business system access situations, so when the data resource is changed, the cache data will be inconsistent, that is: the data values of different service systems in the same data set are different. Further, a setting error in the buffer time is likely to cause a problem of buffer breakdown or buffer avalanche.
The prior art is therefore still subject to further development.
Disclosure of Invention
In view of the above technical problems, embodiments of the present invention provide a method and a system for dynamically generating cache data, which can solve the technical problem that in the prior art, when a plurality of service systems access the same data resource, the cache data may be inconsistent when the data resource is changed.
A first aspect of an embodiment of the present invention provides a method for dynamically generating cache data, including:
acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data;
acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list;
and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list.
Optionally, the obtaining the version number of the dataset includes:
acquiring data of an external system connected with the Internet of vehicles operation platform, and storing the data of the external system in a wide table;
and acquiring the data version number of each wide table, and generating the data set version number according to the data version numbers of all the wide tables.
Optionally, the obtaining the data version number of each wide table and generating the data set version number according to the data version numbers of all the wide tables includes:
and acquiring the data version number of each wide table, and splicing the data version numbers of all the wide tables to generate a data set version number.
Optionally, after dynamically generating the cache KEY according to the function identifier, the function version number, the version number of the data set, and the hash value of the real parameter list, the method further includes:
and if the change of the function version number and/or the data set version number is detected, generating a cache KEY according to the changed function version number and/or the changed data set version number.
Optionally, if it is detected that the function version number and/or the dataset version number is changed, after generating the cache KEY according to the changed function version number and/or dataset version number, the method further includes:
when detecting that the computing resources are idle, the preheating system inquires the business parameters again, carries out business parameter calculation according to the pre-calculated caching KEY weight, and caches the calculation result.
A second aspect of the embodiments of the present invention provides a system for dynamically generating cache data, where the system includes: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of:
acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data;
acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list;
and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list.
Optionally, the computer program when executed by the processor further implements the steps of:
acquiring data of an external system connected with the Internet of vehicles operation platform, and storing the data of the external system in a wide table;
and acquiring the data version number of each wide table, and generating the data set version number according to the data version numbers of all the wide tables.
Optionally, the computer program when executed by the processor further implements the steps of:
and acquiring the data version number of each wide table, and splicing the data version numbers of all the wide tables to generate a data set version number.
Optionally, the computer program when executed by the processor further implements the steps of:
and if the change of the function version number and/or the data set version number is detected, generating a cache KEY according to the changed function version number and/or the changed data set version number.
A third aspect of the embodiments of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by one or more processors, the computer-executable instructions may cause the one or more processors to perform the above-mentioned method for dynamically generating cache data.
According to the technical scheme provided by the embodiment of the invention, the operation data of the Internet of vehicles operation platform is obtained, and the function identification, the function version number and the data set version number in the Internet of vehicles operation platform are obtained according to the operation data; acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list; and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list. Therefore, compared with the prior art, the embodiment of the invention can automatically switch the cached KEY during function upgrading and data changing, thereby avoiding the problems of data leakage, confusion of old data and new data and the like caused by manual intervention, and the data display also becomes instant change and all modules are unified.
Drawings
Fig. 1 is a schematic flowchart of an embodiment of a dynamic cache data generation method according to the present invention;
fig. 2 is a schematic diagram of a hardware structure of another embodiment of a cache data dynamic generation system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following detailed description of embodiments of the invention refers to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart illustrating an embodiment of a dynamic cache data generation method according to the present invention. As shown in fig. 1, includes:
step S100, acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data;
s200, acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list;
and step S300, dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list.
Specifically, the car networking means: the vehicle-mounted equipment on the vehicle effectively utilizes all vehicle dynamic information in the information network platform through a wireless communication technology, and provides different functional services in the running process of the vehicle. The internet of vehicles exhibits the following features: the Internet of vehicles can provide guarantee for the distance between the vehicles, and the probability of collision accidents of the vehicles is reduced; the Internet of vehicles can help the vehicle owner to navigate in real time, and the efficiency of traffic operation is improved through communication with other vehicles and a network system. The car networking operation platform in the embodiment of the invention is introduced by taking a wheat and grain car networking operation service platform as an example.
The cached KEY is changed to be dynamically generated. And adding the function identifier, the function version number and the data set version number into the calculation generation process of the cache KEY. Supplementary explanation: if the function is upgraded or patched, the function number needs to be changed.
In some other embodiments, when multiple external system data are used, the multiple external data set version numbers are concatenated to generate the calculated cache KEY.
Aiming at list pages with more query conditions and second-level detail pages, a cache cannot be set; according to the embodiment of the invention, redundant request parameters are removed according to the field names at the function entrance according to the parameter list of the interface, and key service fields are reserved; the preset in the system configuration is preset in the system configuration; and (4) sorting according to rules, generating a specific character string, and mapping the specific character string into a unique digital KEY, namely dynamically generating a cache KEY, wherein most service interfaces have corresponding parameter configuration, and redundant fields can be eliminated according to the configuration.
In some other embodiments, external system data may not be needed, and only this calculation is used: key-function version number + identification of function + hash value of request argument list. Thereby obtaining the value of the cache key.
In some other embodiments, the hash values of the argument list are computed differently. May be calculated by other means, such as sorting and splicing according to the lexicographic order of the parameter values to obtain V; ordering and connecting according to the dictionary order of the parameter names to obtain N; and finally, calculating the hash value after S is equal to V + N.
The cache clearing operation in the embodiment of the invention reduces manual intervention and communication cost; pages screened by multiple conditions and second-level detail pages can be cached, and meanwhile, the waiting time of a data first page is greatly reduced.
Further, obtaining the version number of the dataset comprises:
acquiring data of an external system connected with the Internet of vehicles operation platform, and storing the data of the external system in a wide table;
and acquiring the data version number of each wide table, and generating the data set version number according to the data version numbers of all the wide tables.
When the method is specifically implemented, the method is realized on the basis of a big data wide table based on a wheat and grain vehicle networking data operation service platform (OMS). Data of external systems connected to the OMS is stored in a wide table. The wide table data corresponds to a data version number, and if the wide table data is changed, the data version number is changed accordingly. Each wide table corresponds to a data version number, and the set of data version numbers of all wide tables is the data set version number. The broad-form brief description is to put the related fields into the same form to be stored, and the related business contents are gathered together. The broad table has not met the three-normal model design specifications. The main disadvantage is the large redundancy of data, and the corresponding benefit is the improvement and convenience of query performance. The design of the wide table is widely applied to data preparation before training of the data mining model, and the efficiency of iterative computation in the training process of the data mining model can be greatly improved.
Further, acquiring the data version number of each wide table, and generating a data set version number according to the data version numbers of all the wide tables, including:
and acquiring the data version number of each wide table, and splicing the data version numbers of all the wide tables to generate a data set version number.
Specifically, each wide table has a corresponding data version number. And splicing the data version numbers of the used wide tables to obtain the version number of the data set.
Further, after dynamically generating the cache KEY according to the function identifier, the function version number, the version number of the dataset, and the hash value of the real parameter list, the method further includes:
and if the change of the function version number and/or the data set version number is detected, generating a cache KEY according to the changed function version number and/or the changed data set version number.
Specifically, when the function is upgraded, the function version number is changed; when the data set is operated and refreshed, the version number of the data set is changed.
Aiming at the problem that data values cached in web pages of different service systems in the same data set are inconsistent in the prior art, in the embodiment of the invention, after a data calculation task of a wide table is completed, the version number of the data set needs to be updated. When the version number of the data set is changed, the cache KEY is synchronously changed, the cache content is recalculated, and the cache is generated again. When the function version number changes, the cache KEY also changes synchronously.
Further, if it is detected that the function version number and/or the dataset version number is changed, after generating the cache KEY according to the changed function version number and/or dataset version number, the method further includes:
when detecting that the computing resources are idle, the preheating system inquires the business parameters again, carries out business parameter calculation according to the pre-calculated caching KEY weight, and caches the calculation result.
In specific implementation, after the processing of cache data consistency is completed, the cache KEY weight calculation method can be combined with a cache preheating system, and the cache KEY weight is calculated in advance according to factors such as user habits and access times. When the computing resources are idle, the preheating system inquires the business actual parameters again to perform computation and cache, so that the effects of reducing the system load in busy hours, accurately caching and improving the cache utilization rate are achieved.
Preheating a system: a system for simulating user access behavior; before the actual user flow is inrush (rush purchase and second kill), the conventional actions of the user are simulated, pages are accessed, data are screened, and the like, cold data are circulated in the whole system range, so that each subsystem is provided with accurate cache under the condition of low flow, the response rate of a back-end system to the burst flow is improved, and the system load is balanced.
The invention also provides a specific embodiment of a method for generating the cache KEY, which comprises the following steps:
acquiring user list data; acquiring a function identifier, wherein the function identifier is a full capital English name of an interface; acquiring a function version identifier, wherein the function version identifier is a source code level constant; requesting a list of version numbers for the data set { A, B }; splicing the character string S, namely a function version number-a data set list version number; h — calculated value of sha 1; splicing the cache KEY as function identification + H; the cached KEY is returned.
The invention also provides a specific embodiment of processing the cache KEY by the user request, and the method comprises the following steps:
the user requests the process to start; calculating a cache KEY; searching cache data according to the cache KEY; judging whether cache data exist or not; if yes, returning a data result; and if not, performing service calculation, acquiring a result of the service calculation, and returning the result serving as cache data according to the result of the service calculation.
Further, function development at the OMS side is responsible for version number change of the functional module; and the wide table calculation of the big data side calculates the business change version number. Therefore, the buffer memory among the data can realize the version linkage. Achieving the effect of immediate unification. The computing service comprises report statistics and self-defined label details in a time dimension (by day, month and week) or a channel dimension, an application dimension, a failure state dimension, a continuous N-day activity dimension and the like.
The process of changing the function version number is as follows:
initiating a demand; the logic of the function code is modified; modify the trim function version number (source code level); and (6) function release.
The process of changing the version number of the data set comprises the following steps:
associating the business form table and extracting data rows; extracting fields of the data lines and gathering the fields; calculating the service; and changing the version number of the data set and writing the version number into the list table. The single table is a data table in the wide table.
The embodiment of the invention also discloses a specific embodiment of a multi-screening condition cache KEY generation method, wherein the parameter list configures an interface and parameters; the interface refers to an interface English name which is open to the outside, a class name which is internally quoted by an interface HTTP method [ GET/POST ], and a method name; the parameters comprise a parameter English name, a parameter type and a parameter value range. The specific process is as follows:
acquiring service parameter list configuration; acquiring an actual request parameter; extracting the real ginseng list according to the configuration;
sorting according to the parameter names in a dictionary order; j is encoded in JSON; calculating the md5 value of J; calling a flow 1, and caching KEY generation; KEY + J (splice); the cached KEY is returned.
According to the method embodiment, the cached KEY can be automatically switched during function upgrading and data changing, so that the problems (such as data leakage and confusion of old data and new data) caused by manual intervention are avoided, the data display is instant, and all modules are unified.
The function version number is the version number of the source code level, is directly controlled by research personnel, and is simple and effective.
The identification of the function is directly using the interface name as the identification. The method is easy to identify in the cache system, and is beneficial to research personnel to troubleshoot problems.
Dataset version numbers use dataset version numbers for data references (broad tables) of external systems to achieve the same effect.
The calculation mode of the cache KEY is that KEY is the function version number + the identification of the function + the data set version number + the hash value of the request real parameter list.
And (4) calculating the actual parameter dispersion values under multiple screening conditions. And the common treatment mode is as follows: the dictionary ordering and the splicing character strings are different, and the json coding is more universal when being directly used. When external system data is quoted, the data is often inconsistent, and the multi-level data set version number can be used for calculation, so that the effect can be achieved.
The above describes the dynamic cache data generation method in the embodiment of the present invention, and the following describes the dynamic cache data generation system in the embodiment of the present invention, please refer to fig. 2, fig. 2 is a schematic hardware structure diagram of another embodiment of the dynamic cache data generation system in the embodiment of the present invention, and as shown in fig. 2, the system 10 includes: a memory 101, a processor 102 and a computer program stored on the memory and executable on the processor, the computer program realizing the following steps when executed by the processor 101:
acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data;
acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list;
and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
acquiring data of an external system connected with the Internet of vehicles operation platform, and storing the data of the external system in a wide table;
and acquiring the data version number of each wide table, and generating the data set version number according to the data version numbers of all the wide tables.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
and acquiring the data version number of each wide table, and splicing the data version numbers of all the wide tables to generate a data set version number.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
and if the change of the function version number and/or the data set version number is detected, generating a cache KEY according to the changed function version number and/or the changed data set version number.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
when detecting that the computing resources are idle, the preheating system inquires the business parameters again, carries out business parameter calculation according to the pre-calculated caching KEY weight, and caches the calculation result.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer-executable instructions for execution by one or more processors, for example, to perform method steps S100-S300 of fig. 1 described above.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for dynamically generating cache data is characterized by comprising the following steps:
acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data;
acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list;
and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list.
2. The method according to claim 1, wherein the obtaining the version number of the dataset comprises:
acquiring data of an external system connected with the Internet of vehicles operation platform, and storing the data of the external system in a wide table;
and acquiring the data version number of each wide table, and generating the data set version number according to the data version numbers of all the wide tables.
3. The method for dynamically generating cached data according to claim 2, wherein the obtaining the data version number of each wide table and generating the data set version number according to the data version numbers of all wide tables comprises:
and acquiring the data version number of each wide table, and splicing the data version numbers of all the wide tables to generate a data set version number.
4. The method for dynamically generating cached data as claimed in claim 3, wherein after dynamically generating the caching KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list, the method further comprises:
and if the change of the function version number and/or the data set version number is detected, generating a cache KEY according to the changed function version number and/or the changed data set version number.
5. The method according to claim 4, wherein if it is detected that the function version number and/or the dataset version number are changed, after generating the cache KEY according to the changed function version number and/or dataset version number, the method further comprises:
when detecting that the computing resources are idle, the preheating system inquires the business parameters again, carries out business parameter calculation according to the pre-calculated caching KEY weight, and caches the calculation result.
6. A system for dynamically generating cached data, the system comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of:
acquiring operation data of the Internet of vehicles operation platform, and acquiring a function identifier, a function version number and a data set version number in the Internet of vehicles operation platform according to the operation data;
acquiring an actual parameter list requested by a user, and calculating a hash value of the actual parameter list;
and dynamically generating a cache KEY according to the function identifier, the function version number, the data set version number and the hash value of the real parameter list.
7. The system for dynamically generating cached data as recited in claim 6, wherein the computer program when executed by the processor further performs the steps of:
acquiring data of an external system connected with the Internet of vehicles operation platform, and storing the data of the external system in a wide table;
and acquiring the data version number of each wide table, and generating the data set version number according to the data version numbers of all the wide tables.
8. The system for dynamically generating cached data as recited in claim 7, wherein the computer program when executed by the processor further performs the steps of:
and acquiring the data version number of each wide table, and splicing the data version numbers of all the wide tables to generate a data set version number.
9. The system for dynamically generating cached data as recited in claim 8, wherein the computer program when executed by the processor further performs the steps of:
and if the change of the function version number and/or the data set version number is detected, generating a cache KEY according to the changed function version number and/or the changed data set version number.
10. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the method of dynamically generating cached data as recited in any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784802.1A CN112187849A (en) | 2020-08-06 | 2020-08-06 | Cache data dynamic generation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784802.1A CN112187849A (en) | 2020-08-06 | 2020-08-06 | Cache data dynamic generation method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112187849A true CN112187849A (en) | 2021-01-05 |
Family
ID=73919737
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010784802.1A Pending CN112187849A (en) | 2020-08-06 | 2020-08-06 | Cache data dynamic generation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112187849A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115208893A (en) * | 2022-05-23 | 2022-10-18 | 中国银行股份有限公司 | Resource scheduling method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874465A (en) * | 2017-02-15 | 2017-06-20 | 浪潮软件集团有限公司 | Method for efficiently managing cache based on data version |
| CN110019350A (en) * | 2017-07-28 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data query method and apparatus based on configuration information |
| CN110737682A (en) * | 2019-10-17 | 2020-01-31 | 贝壳技术有限公司 | cache operation method, device, storage medium and electronic equipment |
| CN110928904A (en) * | 2019-10-31 | 2020-03-27 | 北京浪潮数据技术有限公司 | Data query method and device and related components |
| CN111339143A (en) * | 2020-02-27 | 2020-06-26 | 郑州阿帕斯数云信息科技有限公司 | Data caching method and device and cloud server |
-
2020
- 2020-08-06 CN CN202010784802.1A patent/CN112187849A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874465A (en) * | 2017-02-15 | 2017-06-20 | 浪潮软件集团有限公司 | Method for efficiently managing cache based on data version |
| CN110019350A (en) * | 2017-07-28 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data query method and apparatus based on configuration information |
| CN110737682A (en) * | 2019-10-17 | 2020-01-31 | 贝壳技术有限公司 | cache operation method, device, storage medium and electronic equipment |
| CN110928904A (en) * | 2019-10-31 | 2020-03-27 | 北京浪潮数据技术有限公司 | Data query method and device and related components |
| CN111339143A (en) * | 2020-02-27 | 2020-06-26 | 郑州阿帕斯数云信息科技有限公司 | Data caching method and device and cloud server |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115208893A (en) * | 2022-05-23 | 2022-10-18 | 中国银行股份有限公司 | Resource scheduling method and device |
| CN115208893B (en) * | 2022-05-23 | 2024-04-16 | 中国银行股份有限公司 | Resource scheduling method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11949759B2 (en) | Adaptive computation and faster computer operation | |
| US10715598B1 (en) | Implementation of a web-scale data fabric | |
| US20200097218A1 (en) | File system block-level tiering and co-allocation | |
| US9576024B2 (en) | Hierarchy of servers for query processing of column chunks in a distributed column chunk data store | |
| CN109947668B (en) | Method and device for storing data | |
| WO2020164276A1 (en) | Webpage data crawling method, apparatus and system, and computer-readable storage medium | |
| CN111709527A (en) | Operation and maintenance knowledge map library establishing method, device, equipment and storage medium | |
| US11093496B1 (en) | Performance-based query plan caching | |
| US11068456B2 (en) | Level-based hierarchies | |
| JP2017505936A (en) | System and method for hosting an in-memory database | |
| CN112148217B (en) | Method, device and medium for caching deduplication metadata of full flash memory system | |
| US10909091B1 (en) | On-demand data schema modifications | |
| CN111966938B (en) | Configuration method and system for realizing loading speed improvement of front-end page of cloud platform | |
| US20230244659A1 (en) | Query execution utilizing negation of a logical connective | |
| WO2021191702A1 (en) | Offloading statistics collection | |
| CN112187849A (en) | Cache data dynamic generation method and system | |
| US11455305B1 (en) | Selecting alternate portions of a query plan for processing partial results generated separate from a query engine | |
| EP4216076A1 (en) | Method and apparatus of processing an observation information, electronic device and storage medium | |
| CN115993932A (en) | Data processing method, device, storage medium and electronic equipment | |
| US11727003B2 (en) | Scaling query processing resources for efficient utilization and performance | |
| US11886439B1 (en) | Asynchronous change data capture for direct external transmission | |
| US20230132173A1 (en) | Data reading method, device and storage medium | |
| US20220107949A1 (en) | Method of optimizing search system | |
| US20220004531A1 (en) | In-memory data structure for data access | |
| US20240078221A1 (en) | Systems and methods of modeling and querying dynamic temporal graph on massive parallel graph processing and storage engine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210105 |
|
| RJ01 | Rejection of invention patent application after publication |