CN110581779A - Configuration issuing platform system and configuration issuing method based on local cache - Google Patents
Configuration issuing platform system and configuration issuing method based on local cache Download PDFInfo
- Publication number
- CN110581779A CN110581779A CN201910757171.1A CN201910757171A CN110581779A CN 110581779 A CN110581779 A CN 110581779A CN 201910757171 A CN201910757171 A CN 201910757171A CN 110581779 A CN110581779 A CN 110581779A
- Authority
- CN
- China
- Prior art keywords
- configuration
- client
- local cache
- configuration information
- issuing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Abstract
The invention discloses a configuration issuing platform system based on local cache and a corresponding configuration issuing method. The system comprises a platform and a plurality of different client end sets, wherein the platform consists of a plurality of servers, and each client end set consists of one or more client ends. And a unique mark character string used for indicating the identity information of the client is recorded in each client. Each of the servers includes: the local cache module is used for storing and recording the configuration information of the client for the client to use when requesting configuration; and the configuration service module is in communication connection with the local cache module and is used for storing and recording the configuration issuing strategy, and when the client requests configuration, the configuration service module is used for acquiring the configuration issuing content of the client through all the configuration issuing strategies. The invention obviously improves the integral processing capacity of the platform, improves the configuration timeliness, also solves the problem of system expansibility and has very wide application prospect.
Description
Technical Field
The invention relates to a configuration issuing system and a corresponding configuration issuing method, in particular to a configuration issuing platform system and a configuration issuing method based on local cache, and belongs to the technical field of network communication.
Background
The C/S architecture (Client/Server architecture), which is a common software system architecture, reduces the overall communication overhead of the system by reasonably distributing tasks to the Client and the Server. In this architecture, the programs in the client and the server are different, the programs used by the user are mainly concentrated in the client, the client program mainly completes the specific services of the user, and the server is mainly responsible for providing data management, data sharing, data and system maintenance and concurrency control, and the like. In terms of the application level, the system under the framework is convenient and easy to develop, and the operation process is very simple, but once the upgrading of the application program and the maintenance of the client program are involved, the operation is inconvenient.
Specifically, in the current configuration issuing platform system using a typical C/S architecture, the number of clients managed by the platform exceeds 5000 ten thousand, different configurations can be issued for different sets of clients according to business needs, and the clients can also be configured from the platform synchronously periodically. In the actual application process, version differences exist among clients, and configuration information acquired by clients of different versions is different. Since real-time is not an indicator of the client requesting the configuration, the client is allowed to delay for a period of time and resynchronize to the correct configuration when the platform changes configuration. The types of the configuration contents issued by the platform are limited, generally not more than 100, that is, the configurations obtained by most clients are the same or similar, and relatively, the length of the configuration contents issued by the platform is long and can reach about 1 KByte.
In one type of existing technical solution, a platform stores a configuration of a client using a database, and when a synchronization request of the client comes, the platform reads a latest configuration from the database and returns the latest configuration to the client, which is the most direct processing manner. The greatest limitation of this processing method is disk I/O (Input/Output), and in the test environment, after a QPS (Query Per Second) of a client exceeds 1kpps, the pressure of a database becomes very large, and the time delay for issuing configuration by the entire platform becomes uncontrollable. Even if the technical scheme is added, the performance cannot be improved, and the bottleneck lies in the reading and writing pressure of the database.
In another type of existing technical solution, a platform stores the configuration of a client using a database, and synchronizes configuration information to a redis memory at the same time. When a synchronization request of a client comes, the server interacts with the redis to obtain the latest configuration of the current client. The biggest limitation of the scheme is the processing pressure of the redis, and the configuration content of a certain client needs to be read and written for many times, so that the redis is found to be very busy in actual test, the packet loss of the single-machine server in a test environment is serious after the QPS of the client exceeds 3kpps, and the time delay is uncontrollable.
Therefore, how to provide a brand-new configuration issuing platform system and a corresponding configuration issuing method to solve the above-mentioned problems of system performance, configuration timeliness, system expansibility and the like becomes a problem to be solved by the technical personnel in the industry at present.
disclosure of Invention
in view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a configuration issuing platform system and a configuration issuing method based on a local cache, which are as follows.
a configuration issuing platform system based on local cache comprises a platform and a plurality of different client end sets, wherein the platform is in communication connection with the client end sets, the platform consists of a plurality of servers, each server corresponds to one or more client end sets, and each client end set consists of one or more client ends;
Each client records a unique mark character string for indicating the identity information of the client;
each of the servers includes therein a server that includes,
The local cache module is used for storing and recording the configuration information of the client for the client to use when requesting configuration;
And the configuration service module is in communication connection with the local cache module and is used for storing and recording the configuration issuing strategy, and when the client requests configuration, the configuration service module is used for acquiring the configuration issuing content of the client through all the configuration issuing strategies.
preferably, the local cache module is configured to store and record configuration information of the client, and when a certain client requests configuration, find and acquire corresponding configuration information from configuration information recorded therein according to a unique identifier string of the client.
preferably, the configuration information has a structure of,
the configuration is started to send down the content, then is the configuration integer synchronously acquired with the configuration service, then is the current configuration timeout time, and finally is the client version number.
Preferably, in the local cache module, the configuration information is gathered and stored in an index manner.
Preferably, the local cache module comprises three mapping tables, respectively,
A first mapping table, namely a mapping table from index to configuration information, for cache reading when the client requests configuration;
a second mapping table, namely a mapping table from configuration information to indexes, for adding new configuration information into the local cache module;
A third mapping table, i.e. a mapping table from the unique marker string of the client to an index.
preferably, in the configuration service module, the configuration issuing policy and the configuration information are aggregated and stored in a redis database manner.
A configuration issuing method based on local cache uses the configuration issuing platform system based on local cache, which comprises the following steps:
S1, when the client requests the configuration, reading the local cache module, according to the unique mark character string of the client, searching and obtaining the corresponding configuration information from the configuration information recorded in the local cache module;
s2, judging whether the configuration information acquired in S1 exists and is valid,
if yes, directly feeding back the configuration information to the client,
if not, connection is initiated to the configuration service module, the latest configuration information is obtained, and the obtained latest configuration information is fed back to the client and is added to the local cache module.
preferably, the method further comprises the following steps:
And S3, periodically synchronizing the configuration information by the local cache module and the configuration service module, wherein after the configuration information is synchronized each time, the local cache module updates the configuration information and the previous related cache fails.
Preferably, S1 includes a cache read operation, where the cache read operation is triggered when the client requests configuration; in the cache reading operation, the input is the unique mark character string of the client, and the output is the configuration information and can be null.
Preferably, S2 includes a cache write operation, where the cache write operation is triggered when the client acquires the latest configuration information; and in the cache writing operation, inputting the unique mark character string or the latest configuration information of the client, if the latest configuration information exists in the local cache module and is consistent with the corresponding index, not performing any processing, otherwise, deleting the content corresponding to the unique mark character string stored in the local cache module, and then adding the new unique mark character string and the latest configuration information.
compared with the prior art, the invention has the advantages that:
the configuration issuing platform system based on local cache and the corresponding configuration issuing method remarkably improve the overall processing capacity of the platform, and enable the memory and CPU resources used at the same time to be controllable.
moreover, the system and the method of the invention also furthest improve the timeliness of the configuration, and when the platform configuration is changed, the corresponding client can quickly acquire the latest configuration; when the client version is updated, the latest configuration can be acquired quickly.
meanwhile, the system of the invention also considers the problem of system expansibility, when the number of clients in the system is increased, the existing platform architecture is not required to be changed, and the overall performance of the platform system can be improved only by adding corresponding hardware, thereby further expanding the application range of the invention.
in addition, the invention also provides reference for other related schemes in the same field, can be expanded and extended based on the reference, is applied to other technical schemes related to the caching technology and the configuration issuing technology in the same field, and has very wide application prospect.
The following detailed description of the embodiments of the present invention is provided in connection with the accompanying drawings for the purpose of facilitating understanding and understanding of the technical solutions of the present invention.
drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Detailed Description
The invention provides a configuration issuing platform system and a configuration issuing method based on local cache, which are concretely provided as follows.
A configuration issuing platform system based on local cache comprises a platform and a plurality of different client end sets, wherein the platform is in communication connection with the client end sets, the platform is composed of a plurality of servers, each server corresponds to one or more client end sets, and each client end set is composed of one or more client ends.
And a unique mark character string key used for indicating the identity information of the client is recorded in each client.
Each of the servers includes therein a server that includes,
The local cache module is used for storing and recording the configuration information value of the client for the client to use when requesting configuration;
And the configuration service module is in communication connection with the local cache module and is used for storing and recording the configuration issuing strategy, and when the client requests configuration, all the configuration issuing strategies are performed to obtain the real configuration issuing content of the client.
Generally speaking, in the using process of the system of the present invention, when a certain client requests configuration, firstly searching in the local cache module to check whether the cached configuration information is valid, and if so, acquiring the configuration information and directly issuing the configuration information to the client; otherwise, new configuration information needs to be acquired from the configuration service module, and the configuration information is added to the local cache module.
the local cache module records therein a configuration information value, which includes a configuration issuing content, a variable of whether the current configuration is changed, and a current configuration timeout time (because some configurations are only valid for a certain period of time).
The local cache module is used for storing and recording the configuration information of the client, and when a certain client requests configuration, the corresponding configuration information value is searched and obtained from the configuration information value recorded in the local cache module according to the unique mark character string key of the client.
since the value of the configuration information value may be greater than 1Kbyte, it cannot be fully stored in memory. Therefore, in this embodiment, the local cache module aggregates the configuration information value into an index of 8 bytes for storage, so as to reduce the memory usage. From the actual online situation, the memory occupied by 500 ten thousand clients finally is below 2G.
Specifically, the configuration information value has the following structure,
The method comprises the steps that firstly, configuration issued content (conf info string) is used for representing the real content of configuration issued;
then, the configuration integer (confseed) which is 8 bytes long and is synchronously acquired with the configuration service represents the configuration state of the current cache;
Then, the current configuration timeout time (exception time) with the length of 8 bytes is followed, the configuration policy may have timeout time, and here, if the true timeout time is adopted, values of almost most clients will be different, which results in large memory occupation, in this embodiment, to ensure the convergence of the value, the exception time converges according to 2 hours, that is, there are 12 timeout times in 1 day at most;
the end is the client version number (version) with length of 4 bytes, and when the version number in the cache is not consistent with the newly requested client version number, the current cache needs to be set as invalid.
The following supplementary descriptions are provided for several cases of cache invalidation in the system of the present invention, and the cache invalidation has the following conditions,
One is that the stored conf seed is inconsistent with the current global conf seed, which means that the cache storage time is up to this time, the "configuration service" has been updated, so the cache is invalid;
The other is that expiration time expires, which indicates that the current cache is overtime, so the cache is invalid;
And the other is client version number version, if the version number stored in the cache is inconsistent with the client request, the client is indicated to perform upgrading operation, the configuration of the client needs to be refreshed, and therefore the cache is invalid.
The local cache module comprises three mapping tables, respectively,
A first mapping table (index = > value), that is, a mapping table from the index to the configuration information value, for cache reading when the client requests configuration;
a second mapping table (value = > index), that is, a mapping table from the configuration information value to the index, configured to add new configuration information to the local cache module and quickly decide whether the current configuration information value already exists;
A third mapping table (key = > index), i.e. a mapping table from the unique flag string key of the client to the index.
In this embodiment, in the configuration service module, the configuration issuing policy and the configuration information are aggregated and stored in a manner of a redis database.
a configuration issuing method based on local cache, which uses the above configuration issuing platform system based on local cache, as shown in fig. 1, includes the following steps:
s1, when the client requests the configuration, reading the local cache module, according to the unique mark character string of the client, searching and obtaining the corresponding configuration information from the configuration information recorded in the local cache module;
s2, judging whether the configuration information acquired in S1 exists and is valid,
If yes, directly feeding back the configuration information to the client,
if not, connection is initiated to the configuration service module, the latest configuration information is obtained, and the obtained latest configuration information is fed back to the client and is added to the local cache module.
The method of the present invention further comprises the steps of:
S3, the local cache module is periodically connected to the configuration service module, and periodically synchronizes the configuration information, where the period in this embodiment is 10 minutes, and after synchronizing the configuration information each time, the local cache module updates the configuration information, and the previous related cache fails. In addition, when the version of the client is updated, the local cache invalidation is also triggered to ensure that the client acquires the configuration information of the latest version.
It should be noted that S1 includes a cache read operation, where the cache read operation is triggered when the client requests configuration; in the cache reading operation, the unique mark character string of the client is input, and the configuration information is output and can be null; the cache read operation does not change the content of the configuration information recorded inside the local cache module.
s2 includes a cache write operation, which is triggered when the client acquires the latest configuration information; inputting a unique mark character string or latest configuration information of the client in the cache writing operation, if the latest configuration information exists in the local cache module and is consistent with the corresponding index, not performing any processing, otherwise deleting the content corresponding to the unique mark character string stored in the local cache module, and then adding a new unique mark character string and the latest configuration information; and the cache writing operation changes the content of the configuration information recorded in the local cache module.
The configuration issuing platform system based on local cache and the corresponding configuration issuing method remarkably improve the overall processing capacity of the platform, and enable the memory and CPU resources used at the same time to be controllable. On a production platform applying the system, about 70-80% of client requests hit a local cache, and the single-machine QPS capacity is improved from 3kpps to over 7 kpps. The system of the invention also obviously reduces the memory occupation and the operation pressure of the system, the single server stores the configuration and issuing content of 500 ten thousand clients, the value generally generated is about 300, and the redundant occupation is not more than 2G memory.
the system and the method of the invention also furthest improve the timeliness of the configuration, and when the platform configuration is changed, the corresponding client can quickly acquire the latest configuration; when the client version is updated, the latest configuration can be acquired quickly. Aiming at the configuration issued service, the system of the invention can enable all configuration changes to take effect within 10 minutes, and the minimum interval of all time period strategies is 2 hours
meanwhile, the system of the invention also considers the problem of system expansibility, and as the local cache in the invention does not depend on the global state, when the number of the clients in the system is increased, the existing platform architecture does not need to be changed, and only corresponding hardware needs to be added, thereby effectively improving the overall performance of the platform system and further expanding the application range of the invention.
in addition, the invention also provides reference for other related schemes in the same field, can be expanded and extended based on the reference, is applied to other technical schemes related to the caching technology and the configuration issuing technology in the same field, and has very wide application prospect.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein, and any reference signs in the claims are not intended to be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (10)
1. A configuration issuing platform system based on local cache comprises a platform and a plurality of different client sets, wherein the platform is in communication connection with the client sets, the platform consists of a plurality of servers, each server corresponds to one or more client sets, each client set consists of one or more clients, and the configuration issuing platform system is characterized in that:
each client records a unique mark character string for indicating the identity information of the client;
Each of the servers includes therein a server that includes,
the local cache module is used for storing and recording the configuration information of the client for the client to use when requesting configuration;
And the configuration service module is in communication connection with the local cache module and is used for storing and recording the configuration issuing strategy, and when the client requests configuration, the configuration service module is used for acquiring the configuration issuing content of the client through all the configuration issuing strategies.
2. The local cache based configuration delivery platform system according to claim 1, wherein: the local cache module is used for storing and recording the configuration information of the client, and when a certain client requests configuration, the corresponding configuration information is searched and obtained from the configuration information recorded in the local cache module according to the unique mark character string of the client.
3. the local cache based configuration delivery platform system according to claim 1, wherein: the configuration information is structured such that,
The configuration is started to send down the content, then is the configuration integer synchronously acquired with the configuration service, then is the current configuration timeout time, and finally is the client version number.
4. The local cache based configuration delivery platform system according to claim 1, wherein: and in the local cache module, the configuration information is gathered and stored in an index mode.
5. The local cache based configuration delivery platform system according to claim 4, wherein: the local cache module comprises three mapping tables, respectively,
a first mapping table, namely a mapping table from index to configuration information, for cache reading when the client requests configuration;
a second mapping table, namely a mapping table from configuration information to indexes, for adding new configuration information into the local cache module;
A third mapping table, i.e. a mapping table from the unique marker string of the client to an index.
6. the local cache based configuration delivery platform system according to claim 1, wherein: in the configuration service module, the configuration issuing strategy and the configuration information are gathered and stored in a redis database mode.
7. A configuration delivery method based on local cache, which uses the configuration delivery platform system based on local cache according to any claim 1 ~ 6, characterized by comprising the following steps:
s1, when the client requests the configuration, reading the local cache module, according to the unique mark character string of the client, searching and obtaining the corresponding configuration information from the configuration information recorded in the local cache module;
S2, judging whether the configuration information acquired in S1 exists and is valid,
if yes, directly feeding back the configuration information to the client,
if not, connection is initiated to the configuration service module, the latest configuration information is obtained, and the obtained latest configuration information is fed back to the client and is added to the local cache module.
8. The configuration issuing method based on the local cache according to claim 7, further comprising the steps of:
And S3, periodically synchronizing the configuration information by the local cache module and the configuration service module, wherein after the configuration information is synchronized each time, the local cache module updates the configuration information and the previous related cache fails.
9. The configuration issuing method based on the local cache according to claim 7, characterized in that: s1 includes a cache read operation, which is triggered when the client requests configuration; in the cache reading operation, the input is the unique mark character string of the client, and the output is the configuration information and can be null.
10. the configuration issuing method based on the local cache according to claim 7, characterized in that: s2 includes a cache write operation, which is triggered when the client acquires the latest configuration information; and in the cache writing operation, inputting the unique mark character string or the latest configuration information of the client, if the latest configuration information exists in the local cache module and is consistent with the corresponding index, not performing any processing, otherwise, deleting the content corresponding to the unique mark character string stored in the local cache module, and then adding the new unique mark character string and the latest configuration information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910757171.1A CN110581779A (en) | 2019-08-16 | 2019-08-16 | Configuration issuing platform system and configuration issuing method based on local cache |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910757171.1A CN110581779A (en) | 2019-08-16 | 2019-08-16 | Configuration issuing platform system and configuration issuing method based on local cache |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110581779A true CN110581779A (en) | 2019-12-17 |
Family
ID=68811507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910757171.1A Pending CN110581779A (en) | 2019-08-16 | 2019-08-16 | Configuration issuing platform system and configuration issuing method based on local cache |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110581779A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671550A (en) * | 2020-09-24 | 2021-04-16 | 紫光云技术有限公司 | Method for preventing equipment configuration from being lost in cloud computing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7441007B1 (en) * | 2001-07-30 | 2008-10-21 | At&T Intellectual Property I, L.P. | System and method for allowing applications to retrieve properties and configuration information from a persistent store |
| CN102255752A (en) * | 2011-06-30 | 2011-11-23 | 北京新媒传信科技有限公司 | Configuration management system and method of server cluster |
-
2019
- 2019-08-16 CN CN201910757171.1A patent/CN110581779A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7441007B1 (en) * | 2001-07-30 | 2008-10-21 | At&T Intellectual Property I, L.P. | System and method for allowing applications to retrieve properties and configuration information from a persistent store |
| CN102255752A (en) * | 2011-06-30 | 2011-11-23 | 北京新媒传信科技有限公司 | Configuration management system and method of server cluster |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671550A (en) * | 2020-09-24 | 2021-04-16 | 紫光云技术有限公司 | Method for preventing equipment configuration from being lost in cloud computing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241072A (en) | Buffering updating method and system based on Canal | |
| CN105324770B (en) | Effectively read copy | |
| US10838935B2 (en) | Automating the logging of table changes in a database | |
| CN101493826B (en) | Database system based on WEB application and data management method thereof | |
| CN103678523B (en) | Distributed cache cache data access methods and device | |
| US8788458B2 (en) | Data caching for mobile applications | |
| US9699017B1 (en) | Dynamic utilization of bandwidth for a quorum-based distributed storage system | |
| US9317213B1 (en) | Efficient storage of variably-sized data objects in a data store | |
| CN101263494B (en) | Method and device for monitoring affair related with object of storage network | |
| CN102447736B (en) | Management and control method for user quota in multi-network storage system | |
| CN109783438A (en) | Distributed NFS system and its construction method based on librados | |
| EA002931B1 (en) | Database synchronization and organization system and method | |
| CN109656956B (en) | Method and device for realizing centralized caching of service system data | |
| CN109033292A (en) | A kind of database distributed caching system and method | |
| JP2001522074A (en) | Communication system and method | |
| CN103761102B (en) | A kind of uniform data service platform and its implementation | |
| CN107818111B (en) | Method for caching file data, server and terminal | |
| US20140229435A1 (en) | In-memory real-time synchronized database system and method | |
| CN107368583A (en) | A kind of method and system of more cluster information inquiries | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| CN111221828A (en) | Method and terminal for improving consistency of database data and cache data | |
| CN103200278A (en) | Source station file updating and releasing method and cache file updating method | |
| US10649964B2 (en) | Incorporating external data into a database schema | |
| CN103365987B (en) | Clustered database system and data processing method based on shared-disk framework | |
| CN112307119A (en) | Data synchronization method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191217 |
|
| RJ01 | Rejection of invention patent application after publication |