CN109656956B - Method and device for realizing centralized caching of service system data - Google Patents
Method and device for realizing centralized caching of service system data Download PDFInfo
- Publication number
- CN109656956B CN109656956B CN201811531069.1A CN201811531069A CN109656956B CN 109656956 B CN109656956 B CN 109656956B CN 201811531069 A CN201811531069 A CN 201811531069A CN 109656956 B CN109656956 B CN 109656956B
- Authority
- CN
- China
- Prior art keywords
- data
- hot spot
- centralized
- module
- service system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000000007 visual effect Effects 0.000 claims abstract description 6
- 238000007726 management method Methods 0.000 claims description 35
- 238000013500 data storage Methods 0.000 claims description 18
- 230000003139 buffering effect Effects 0.000 claims 1
- 230000004044 response Effects 0.000 description 6
- 238000010276 construction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for realizing centralized caching of service system data, which belong to the technical field of data access, and the technical problem to be solved by the invention is how to realize caching of hot spot data and centralized storage and management of cached data, and the adopted technical scheme is as follows: a method for realizing centralized caching of service system data includes adding centralized data caching module and API interface on server, utilizing Key-Value database of Redis open source to store cached hot spot data by centralized data caching module, enabling API interface provided by server to meet access of HTTP and NETTY protocol, carrying out operation of querying, updating and deleting hot spot data stored by Key-Value database of Redis open source by server and realizing visual management of hot spot data in centralized data caching module by Web management end of server. The invention also discloses a method for realizing the centralized caching of the service system data.
Description
Technical Field
The invention relates to the technical field of data access, in particular to a method and a device for realizing centralized caching of service system data.
Background
Data caching, which refers to a high-speed memory inside a hard disk, temporarily stores some data in a computer for reading and re-reading just like a block of buffer. Hard disks for large data caches have great advantages when accessing scattered files. The current cache of hard disk is generally 512KB-2MB, the current data cache of the mainstream ATA hard disk is 2MB, and the highest data cache in SCSI hard disk is already 16MB.
With the development of information technology, the data volume of a business system is increased, the access volume of users is increased, and a scene that the QPS (flow processed by a query server in a specified time) is particularly high exists, so that the database cannot directly support the pressure of the high QPS; in order to relieve the pressure of a database and an application server, improve the performance and response speed of a system and improve user experience, how to realize caching of hot spot data and centralized storage and management of the cached data is a technical problem which needs to be solved at present.
Patent number CN107256235A discloses a cache system hot spot data access method, which comprises the following steps: s1: adding a HotKey module on the DS of redis, starting the HotKey module after the default DS is started, controlling a start-stop hot spot statistics function at any time by using an interface, and performing related initialization after the hot spot statistics function is started; s2: adding HotKey logic for identifying hot spots of a server into a worker thread, competing a try lock for each read request reaching the server, and after the lock is successfully obtained, carrying out three stages, and counting the requests successfully passing through the three stages; s3: when the server feeds back that the hot spot is generated, the client firstly writes the key in the feedback packet into the local-cache, and simultaneously starts a hot-running mode. However, the technical scheme cannot relieve the pressure of the database and the application server, improve the performance and response speed of the system, improve the user experience, and cannot solve the problem of how to realize the caching of hot spot data and the centralized storage and management of the cached data.
Patent number CN105681438A discloses a centralized caching decision strategy for content-centric networks. Under the content center network architecture, the research of basic functions such as data naming, content forwarding, intranet caching and the like of an information center network is realized by utilizing the existing framework adopting SDN, and a centralized caching decision strategy is provided. A centralized controller is placed in the content-centric network, and generates a cache list for each node based on collecting all node request information, deterministically "directing" each node which content to cache, to minimize traffic in the network, and to reduce user request latency. However, the technical scheme cannot relieve the pressure of the database and the application server, improve the performance and response speed of the system, improve the user experience, and cannot solve the problem of how to realize the caching of hot spot data and the centralized storage and management of the cached data.
Disclosure of Invention
The technical task of the invention is to provide a method and a device for realizing centralized caching of service system data, which solve the problems of how to realize caching of hot spot data and centralized storage and management of cached data.
The invention discloses a method for realizing centralized caching of service system data, which is realized in the following way, wherein a centralized data caching module and an API (application program interface) are added on a server, the centralized data caching module adopts a Key-Value database of Redis open source to store cached hot spot data, meanwhile, the server can access HTTP and NETTY protocols for the API interface provided by the outside, query, update and delete the hot spot data stored in the Key-Value database of Redis open source through the server, and the hot spot data is stored in the centralized data caching module in a centralized way, so that unified management of the hot spot data in the centralized data caching module can be realized, and the Web management end of the server can realize visual management of the hot spot data in the centralized data caching module.
Preferably, the method comprises the following specific steps:
s1, when a service system caches hot spot data, generating a key value according to a key value generation rule;
s2, the service system calls an API interface provided by the centralized data caching module to store hot spot data to be cached;
s3, setting effective time of caching for the hot spot data in the step S2 through a server; wherein the effective time is in seconds;
s4, the service system provides data types of cache data for the centralized data cache module;
s5, inquiring, updating and deleting the hot spot data stored in the centralized data cache module through the Web management end. The service system in the invention refers to an application scene and service requirements of the service system participating in construction with cache hot spot data. The business system participating in construction of the invention has clear knowledge on centralized cache, the centralized cache can indeed improve the interaction speed of data to a certain extent, but the cache space cannot be infinitely large in consideration of factors such as performance, cost, instantaneity, network transmission and the like, the centralized cache server cannot be infinitely increased, all data cannot be cached, and a balance point between database interaction and cache interaction needs to be sought.
Preferably, the API interface is developed in Java language.
More preferably, the data types of the cache data include strings, hashes, lists, sets and ordered sets.
More preferably, the key value generation rule is specifically as follows:
(1) application code (appCode): the distributed data is distributed by a centralized data cache module, if an office system is OA and an approval system is ECGAP, the distributed data cannot be repeated;
(2) business data primary key (bizKey): provided by a business system;
(3) data to be stored (VALUE): provided by the business system.
More preferably, in the step S5, the Web management end queries the hot spot data stored in the centralized data cache module, which specifically includes the following steps: inquiring whether the cached data stored in the centralized data caching module has hot spot data required by a service system or not by calling an API interface:
(1) if yes, invoking hot spot data required by the service system;
(2) if not, the server queries the service data.
More preferably, in step S5, the Web management end updates the hot spot data stored in the centralized data cache module, which specifically includes the following steps: the service system firstly inquires the latest hot spot data to be cached, updates and caches the hot spot data by calling the API interface, and updates the effective time.
More preferably, in the step S5, the Web management end deletes the hot spot data stored in the centralized data cache module, which specifically includes the following steps: the service system deletes the hot spot data stored in the centralized data cache module by calling the API interface.
The device comprises a server, a display screen and a plurality of service systems, wherein the server comprises a centralized data storage module, an API (application program interface) module and a Web management module, the service systems are connected with the centralized data storage module through the API module, and the display screen is connected with the centralized data storage module through the Web management module;
the centralized data storage module is used for storing cached hot spot data in a centralized way;
the API interface module is used for realizing the operation of inquiring, updating and deleting the cache data stored in the centralized data storage module by the service system;
the Web management module is used for centralized visual management of the hot spot data stored by the centralized data storage module;
the display screen is used for displaying hot spot data visually managed through the Web management module.
Preferably, the centralized data caching module stores cached hot spot data by adopting a Key-Value database of Redis open source.
The method and the device for realizing the centralized caching of the service system data have the following advantages:
firstly, in order to relieve the pressure of a database and an application server, improve the performance and the response speed of a system, improve the user experience, adopt a cache mode, cache hot spot data (data which is frequently inquired and not frequently modified or deleted) is divided into a centralized cache and a local cache, and compared with the local cache, the centralized cache and the local cache are adopted, the data volume of the cache is larger, meanwhile, the problem of cache data synchronization among multiple instance applications is solved, the unified management and maintenance are convenient, a cache integration development interface is externally provided, cache management is provided for the inside, the service pain point is exactly solved, and the service response speed is improved;
secondly, when the service system participating in construction needs to cache hot spot data, the hot spot data is cached by calling an API interface, and the effective time is set, so that the effective period strategy of the centralized cache data is completed;
and thirdly, the data centralized caching method stores the data according to the service system codes, the service main keys and the data to be stored, only provides the service codes, the service main keys and the data to be stored when the data is queried and cached, can improve the query speed, and can query the data from the database if the cached data is not queried, cache the data into the centralized cache according to the service caching strategy and simultaneously return response data. When the same data is requested next time, the data can be returned from the centralized cache without inquiring the database, so that the data inquiring efficiency is greatly improved;
the visual maintenance interface is provided, the operations such as inquiring, updating and deleting the cache data can be performed, the convenience and the high efficiency are realized, and the maintenance cost is low.
Drawings
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a block diagram of a structure for implementing a centralized caching apparatus for service system data.
Detailed Description
The method and the device for realizing the centralized caching of the business system data are described in detail below with reference to the attached drawings and the specific embodiments of the specification.
Example 1:
the method for realizing the centralized caching of the service system data is characterized in that a centralized data caching module and an API (application program interface) are added on a server, the centralized data caching module adopts a Key-Value database of a Redis open source to store cached hot spot data, meanwhile, the server can access HTTP and NETTY protocols for the API interface provided by the outside, query, update and delete the hot spot data stored in the Key-Value database of the Redis open source through the server, the hot spot data is stored in the centralized data caching module in a centralized manner, unified management of the hot spot data is realized, and the hot spot data in the centralized data caching module can be visually managed through a Web management end of the server. The method comprises the following specific steps:
s1, when a service system caches hot spot data, generating a key value according to a key value generation rule; the key value generation rule is specifically as follows:
(1) application code (appCode): the distributed data is distributed by a centralized data cache module, if an office system is OA and an approval system is ECGAP, the distributed data cannot be repeated;
(2) business data primary key (bizKey): provided by a business system;
(3) data to be stored (VALUE): provided by the business system.
S2, the service system calls an API interface provided by the centralized data caching module to store hot spot data to be cached;
s3, setting effective time of caching for the hot spot data in the step S2 through a server; wherein the effective time is in seconds;
s4, the service system provides data types of cache data for the centralized data cache module; the data types of the cache data include strings, hashes, lists, sets, and ordered sets.
S5, inquiring, updating and deleting the hot spot data stored in the centralized data cache module through the Web management end. The hot spot data is queried, and the method is as follows: inquiring whether the cached data stored in the centralized data caching module has hot spot data required by a service system or not by calling an API interface:
(1) if yes, invoking hot spot data required by the service system;
(2) if not, the server queries the service data.
The hot spot data is updated as follows: the service system firstly inquires the latest hot spot data to be cached, updates and caches the hot spot data by calling the API interface, and updates the effective time.
The hot spot data is deleted, and the method specifically comprises the following steps: the service system deletes the hot spot data stored in the centralized data cache module by calling the API interface.
Wherein, the API interface is developed by adopting Java language. The main code for API interface development is as follows:
example 2:
as shown in fig. 1, the device for realizing the centralized caching of the service system data comprises a server, a display screen and a plurality of service systems, wherein the server comprises a centralized data storage module, an API (application program interface) module and a Web management module, the service systems are connected with the centralized data storage module through the API module, and the display screen is connected with the centralized data storage module through the Web management module;
the centralized data storage module is used for storing cached hot spot data in a centralized way; the centralized data caching module stores cached hot spot data by adopting a Key-Value database of Redis open source.
The API interface module is used for realizing the operation of inquiring, updating and deleting the cache data stored in the centralized data storage module by the service system;
the Web management module is used for centralized visual management of the hot spot data stored by the centralized data storage module;
the display screen is used for displaying hot spot data visually managed through the Web management module.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (5)
1. A method for realizing centralized caching of service system data is characterized in that a centralized data caching module and an API interface are added on a server, the centralized data caching module stores cached hot spot data by adopting a Key-Value database of Redis open source, meanwhile, the server can access HTTP and NETTY protocols to the externally provided API interface, query, update and delete the hot spot data stored in the Key-Value database of Redis open source through the server, the hot spot data is stored in the centralized data caching module in a centralized manner, unified management of the hot spot data is realized, and the hot spot data in the centralized data caching module can be visually managed through a Web management end of the server; the method comprises the following specific steps:
s1, when a service system caches hot spot data, generating a key value according to a key value generation rule; the key value generation rule is specifically as follows:
(1) application coding: distributed by a centralized data cache module;
(2) business data primary key: provided by a business system;
(3) data to be stored: provided by a business system;
s2, the service system calls an API interface provided by the centralized data caching module to store hot spot data to be cached;
s3, setting effective time of caching for the hot spot data in the step S2 through a server;
s4, the service system provides data types of cache data for the centralized data cache module;
s5, inquiring, updating and deleting the hot spot data stored in the centralized data cache module through the Web management end;
the hot spot data is queried, and the method specifically comprises the following steps: inquiring whether the cached data stored in the centralized data caching module has hot spot data required by a service system or not by calling an API interface:
(1) if yes, invoking hot spot data required by the service system;
(2) if not, inquiring the service data through the server;
the hot spot data is updated as follows: firstly, a service system inquires the latest hot spot data to be cached, updates and caches the hot spot data by calling an API interface, and updates the effective time at the same time;
the hot spot data is deleted, and the method specifically comprises the following steps: the service system deletes the hot spot data stored in the centralized data cache module by calling the API interface.
2. The method for implementing centralized buffering of service system data according to claim 1, wherein the API is developed in Java.
3. A method of implementing centralized caching of business system data according to claim 1 or 2, wherein the data types of the cached data include strings, hashes, lists, sets and ordered sets.
4. The device is characterized by comprising a server, a display screen and a plurality of service systems, wherein the server comprises a centralized data storage module, an API (application program interface) module and a Web management module, the service systems are connected with the centralized data storage module through the API module, and the display screen is connected with the centralized data storage module through the Web management module;
the centralized data storage module is used for storing cached hot spot data in a centralized way;
the API interface module is used for realizing the operation of inquiring, updating and deleting the cache data stored in the centralized data storage module by the service system;
the Web management module is used for centralized visual management of the hot spot data stored by the centralized data storage module;
the display screen is used for displaying hot spot data visually managed by the Web management module;
the specific steps of the working process of the device are as follows:
s1, when a service system caches hot spot data, generating a key value according to a key value generation rule; the key value generation rule is specifically as follows:
(1) application coding: distributed by a centralized data cache module;
(2) business data primary key: provided by a business system;
(3) data to be stored: provided by a business system;
s2, the service system calls an API interface provided by the centralized data caching module to store hot spot data to be cached;
s3, setting effective time of caching for the hot spot data in the step S2 through a server;
s4, the service system provides data types of cache data for the centralized data cache module;
s5, inquiring, updating and deleting the hot spot data stored in the centralized data cache module through the Web management end;
the hot spot data is queried, and the method specifically comprises the following steps: inquiring whether the cached data stored in the centralized data caching module has hot spot data required by a service system or not by calling an API interface:
(1) if yes, invoking hot spot data required by the service system;
(2) if not, inquiring the service data through the server;
the hot spot data is updated as follows: firstly, a service system inquires the latest hot spot data to be cached, updates and caches the hot spot data by calling an API interface, and updates the effective time at the same time;
the hot spot data is deleted, and the method specifically comprises the following steps: the service system deletes the hot spot data stored in the centralized data cache module by calling the API interface.
5. The centralized caching device for implementing service system data according to claim 4, wherein the centralized data caching module uses a Key-Value database of a dis open source to store cached hotspot data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811531069.1A CN109656956B (en) | 2018-12-14 | 2018-12-14 | Method and device for realizing centralized caching of service system data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811531069.1A CN109656956B (en) | 2018-12-14 | 2018-12-14 | Method and device for realizing centralized caching of service system data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109656956A CN109656956A (en) | 2019-04-19 |
| CN109656956B true CN109656956B (en) | 2023-06-09 |
Family
ID=66114116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811531069.1A Active CN109656956B (en) | 2018-12-14 | 2018-12-14 | Method and device for realizing centralized caching of service system data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109656956B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11429611B2 (en) | 2019-09-24 | 2022-08-30 | International Business Machines Corporation | Processing data of a database system |
| CN111046070B (en) * | 2019-11-21 | 2023-08-18 | 深圳前海环融联易信息科技服务有限公司 | Intelligent data caching method and device, computer equipment and storage medium |
| CN110941618A (en) * | 2019-11-27 | 2020-03-31 | 河钢数字技术股份有限公司 | Mass heterogeneous data storage method and system |
| CN112256709B (en) * | 2020-09-28 | 2023-06-20 | 中孚安全技术有限公司 | Data reading method, system and equipment under special machine system |
| CN112529737A (en) * | 2020-12-28 | 2021-03-19 | 山东鲁能软件技术有限公司 | Intelligent cache management method and system suitable for power information system |
| CN112506961A (en) * | 2020-12-28 | 2021-03-16 | 浪潮云信息技术股份公司 | Method for improving data query efficiency |
| KR102476119B1 (en) * | 2022-08-02 | 2022-12-09 | 주식회사 레픽스 | Api and socket non-disruptive system using Netty and Redis |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102244685A (en) * | 2011-08-11 | 2011-11-16 | 中国科学院软件研究所 | Distributed type dynamic cache expanding method and system supporting load balancing |
| CN103116627A (en) * | 2013-01-31 | 2013-05-22 | 汉柏科技有限公司 | Database access method with high concurrency service-oriented architecture (SOA) technology and system |
| WO2014131000A2 (en) * | 2013-02-25 | 2014-08-28 | Interdigital Patent Holdings, Inc. | Centralized content enablement service for managed caching in wireless networks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4306152B2 (en) * | 2001-06-26 | 2009-07-29 | 株式会社日立製作所 | Web system with clustered application server and database structure |
-
2018
- 2018-12-14 CN CN201811531069.1A patent/CN109656956B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102244685A (en) * | 2011-08-11 | 2011-11-16 | 中国科学院软件研究所 | Distributed type dynamic cache expanding method and system supporting load balancing |
| CN103116627A (en) * | 2013-01-31 | 2013-05-22 | 汉柏科技有限公司 | Database access method with high concurrency service-oriented architecture (SOA) technology and system |
| WO2014131000A2 (en) * | 2013-02-25 | 2014-08-28 | Interdigital Patent Holdings, Inc. | Centralized content enablement service for managed caching in wireless networks |
Non-Patent Citations (1)
| Title |
|---|
| 大规模游戏平台云缓存技术研究与实践;陈杰等;《移动通信》;20160315(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109656956A (en) | 2019-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109656956B (en) | Method and device for realizing centralized caching of service system data | |
| US11882054B2 (en) | Terminating data server nodes | |
| US6182111B1 (en) | Method and system for managing distributed data | |
| US6820085B2 (en) | Web system having clustered application servers and clustered databases | |
| US20080147974A1 (en) | Multi-level caching system | |
| US10534776B2 (en) | Proximity grids for an in-memory data grid | |
| CN104915319B (en) | The system and method for cache information | |
| EP2369494A1 (en) | Web application based database system and data management method therof | |
| US20080133830A1 (en) | Efficient utilization of cache servers in mobile communication system | |
| EP2541423A1 (en) | Replacement policy for resource container | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| US20200242118A1 (en) | Managing persistent database result sets | |
| US10635650B1 (en) | Auto-partitioning secondary index for database tables | |
| WO2021139431A1 (en) | Data synchronization method and apparatus for microservice, electronic device and storage medium | |
| CN107766343A (en) | A kind of date storage method, device and storage server | |
| CN107786668B (en) | Weight caching website method based on CDN (content delivery network) | |
| US10705978B2 (en) | Asynchronous tracking for high-frequency and high-volume storage | |
| CN109844723B (en) | Method and system for master control establishment using service-based statistics | |
| WO2010031297A1 (en) | Method of wireless application protocol (wap) gateway pull service and system thereof | |
| KR20090025940A (en) | Distribute file system and method for replacing cache data thereof | |
| US11714566B2 (en) | Customizable progressive data-tiering service | |
| WO2002061586A2 (en) | Smart-caching system and method | |
| CN113138943B (en) | Method and device for processing request | |
| US11055266B2 (en) | Efficient key data store entry traversal and result generation | |
| CN108628540A (en) | Data storage device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |