CN115794840A - Method and device for supporting memory data table monitoring - Google Patents
Method and device for supporting memory data table monitoring Download PDFInfo
- Publication number
- CN115794840A CN115794840A CN202211184680.8A CN202211184680A CN115794840A CN 115794840 A CN115794840 A CN 115794840A CN 202211184680 A CN202211184680 A CN 202211184680A CN 115794840 A CN115794840 A CN 115794840A
- Authority
- CN
- China
- Prior art keywords
- key
- client
- value
- fcdb
- prefix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a method and a device for supporting memory data table monitoring, wherein the method comprises the following steps: defining 256 tables of a memory database Fcdb, and using a kv database to store data; when data changes, the Fcdb detects whether a client monitors the table; if the table is not monitored by the client, the table is not checked; if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table and sending the key and the value to the corresponding client; if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode; if the key of the table matches the monitoring, notifying the key and the value to the corresponding monitoring client; if there is no match, the update is ignored. In this way, the memory database Fcdb stores data using a kv database, which has high concurrent processing capability; the memory database Fcdb checks the corresponding table for monitoring and notifies the change of each key.
Description
Technical Field
The embodiments of the present invention generally relate to the technical field of memory databases, and in particular, to a method and an apparatus for supporting memory data table snooping.
Background
As information technology is continuously developed, various databases are developed, and database providers provide database-based operation actions, such as Data Manipulation Language (DML) operations, which may include insert operations, delete operations, update operations, and the like, when developing databases to operate Data tables, data objects, and the like in the databases. When the database is stored in the operation action, the database records the operation log information generated by the operation. However, under the instant high-concurrency high-flow access pressure, the traditional relational database is overwhelmed, so that the system service is slowed down, and the concurrency capability of the system is greatly influenced. And the monitoring event of the Redis database can only return key and does not return value.
Such as the patent: "a Zookeeper-based distributed architecture data consistency method (application No.: CN 201910036845.9)": the invention includes the following steps, buffer data initialization: after each node of the distributed system is started, loading data in the Zookeeper into a memory of each node; deployment subscription listening: defining a subscription monitoring method, subscribing Zookeeper catalogues and data by each node, and deploying monitoring; data synchronization: and after monitoring the data change in the Zookeeper, updating the memory data of each node by a deployed monitoring method, and keeping the data synchronization among the nodes. According to the method, the Zookeeper component is used as a cache data synchronization manager in a distributed architecture system, the cache data of distributed multiple nodes are synchronized in real time by using a release and subscription mechanism of the Zookeeper, and meanwhile, the data access speed is greatly improved by a memory-based Zookeeper database, so that the performance and the data consistency are well improved in the distributed architecture system, but the problems that the system service is slowed down and the value is not returned under the instant high-concurrency high-flow access pressure are not solved.
Disclosure of Invention
In order to solve the above problems, the memory database Fcdb of the present invention stores data using a kv database, which has high concurrency processing capability; the memory database Fcdb checks the monitoring of the corresponding table for the increment, modification, deletion and overtime of each key, if yes, the notification is carried out, and key + value is returned; in addition, prefix matching can be carried out on the monitored key.
According to the embodiment of the invention, a method and a device for supporting memory data table snooping are provided.
In a first aspect of the invention, a method for supporting snooping of a memory data table is provided. The method comprises the following steps:
s01: defining 256 tables of a memory database Fcdb, and using a kv database to store data;
s02: when data changes, the Fcdb detects whether a client monitors the table;
s03: if the table is not monitored by the client, the table is not checked;
s04: if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table and sending the key and the value to the corresponding client;
s05: if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode;
s06: if the key of the table matches the monitoring, notifying the key and the value to the corresponding monitoring client; if there is no match, the update is ignored.
Furthermore, the memory in S01 has 256 arrays, each array has a number, and the number is 0 to 255.
Further, the number value corresponds to the id value of the table.
Further, the client in S02 stores a client structure in the database, and the client structure has a function of monitoring a plurality of tables.
Further, the database has a monitoring list, the monitoring list includes an id of the list and a list pointed by the list, and if the list is not null, it indicates that a client monitors the list.
Further, the prefix described in S02 is defined by the user.
Further, the Fcdb also has a monitoring function, and the specific steps are as follows:
s011: providing the number of the table needing monitoring;
s012: selecting whether to provide a key prefix;
s013: if the key prefix is provided, the Fcdb judges the data change according to the key prefix;
s014: if no key prefix is provided, the Fcdb listens for all keys.
In a second aspect of the present invention, an apparatus for supporting snooping of a memory data table is provided. The device includes:
a definition module: 256 tables for defining the memory database Fcdb, and using a kv database to store data;
the client side monitoring and judging module: when data is changed, the Fcdb detects whether a client side monitors the table, and if the table is not monitored by the client side, the Fcdb does not check the table;
a monitoring and judging module: if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table and sending the key and the value to the corresponding client; if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode;
a monitoring matching module: the monitoring client side is used for notifying the key and the value to the corresponding monitoring client side if the key of the table matches the monitoring; if there is no match, the update is ignored.
Furthermore, the memory in the definition module has 256 arrays, each array has a number, and the value of the number is 0 to 255.
Further, the number value corresponds to the id value of the table.
Furthermore, the client in the client monitoring and judging module stores a client structure in the database, and the client structure has the function of monitoring a plurality of tables.
Further, the database has a monitoring list, the monitoring list includes an id of the list and a list pointed by the list, and if the list is not null, it indicates that a client monitors the list.
Further, the prefix in the interception judging module is defined by the user.
Further, the Fcdb further comprises:
a selection module: the number of the table which needs to be monitored is provided;
a judging module: the Fcdb is used for selecting whether to provide the key prefix or not, and if the key prefix is provided, the Fcdb judges data change according to the key prefix; if no key prefix is provided, the Fcdb listens for all keys.
The above-mentioned english abbreviation:
fcdb: in-memory database
Key: key word
Value: key word
Id: identity document, identification number of Identity card
Null: is a value reserved in the calculation to indicate that the pointer does not reference a valid object
Kv: key-value database is a database for storing data by Key value pairs
Redis: remote Dictionary Server, which is an open source log-type and Key-Value database written in ANSI C language, supporting network, based on memory and endurable, and providing API of multiple languages
The memory database Fcdb stores data by using a kv database, and has high concurrent processing capacity; the memory database Fcdb checks the monitoring of the corresponding table when the key is added, modified, deleted and overtime, if yes, the notice is carried out, and the key + value is returned; in addition, prefix matching can be carried out on the monitored key.
It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of any embodiment of the invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
The above and other features, advantages and aspects of embodiments of the present invention will become more apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings. Wherein:
FIG. 1 is a flow diagram illustrating a method of supporting memory data table snooping according to an embodiment of the invention;
FIG. 2 is a block diagram of an apparatus that supports memory table snooping according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
According to the embodiment of the invention, a method and a device for supporting memory data table monitoring are provided.
The principles and spirit of the present invention are explained in detail below with reference to several exemplary embodiments of the present invention.
Fig. 1 is a flowchart illustrating a method for supporting memory data table snooping according to an embodiment of the present invention. The method comprises the following steps:
s01: defining 256 tables of a memory database Fcdb, and using a kv database to store data;
s02: when data changes, the Fcdb detects whether a client monitors the table;
s03: if the table is not monitored by the client, no check is performed;
s04: if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table and sending the key and the value to the corresponding client;
s05: if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode;
s06: if the key of the table matches the monitoring, notifying the key and the value to the corresponding monitoring client; if there is no match, the update is ignored.
It should be noted that although the operations of the method of the present invention have been described in the above embodiments and the accompanying drawings in a particular order, this does not require or imply that these operations must be performed in this particular order or that all of the illustrated operations must be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
For a clearer explanation of the method for supporting the snooping of the memory data table, the following description is made with reference to two specific embodiments, however, it should be noted that the embodiments are only for better explaining the present invention and should not be construed as limiting the present invention.
The method for supporting memory table snooping is further described in more detail in the following two embodiments:
example 1:
defining 256 tables of the memory database Fcdb, wherein the serial number is 0-255, corresponding to the id of the table, and using a kv database to store data;
each table is provided with two pointers, one pointer points to a hash table and the other pointer points to a monitoring list, the monitoring list points to a monitoring item of a client, each client stores a client structure in a database, and one client can monitor a plurality of tables, so that one monitoring list is provided. Each list entry has a table id to listen to and a list pointed to by the table.
Key with abc of table0 updated: the original value data 001 is updated to 100, fcdb detects whether a client side monitors on the table, namely, the value of the lists pointed by the table is checked, if the value of the lists is null, the table is monitored by the client side, then a key prefix listenKey is checked, if the value of the listenKey is null, the client side monitors the data of the whole table, the data is sent to a corresponding client side, and key + value is returned.
The key of ab updated with table 10: the original value data 111 is updated to 010, fcdb detects whether a client monitors on the table, that is, the value of the lists pointed by the table is checked, if the value of the lists is not null, the table is monitored by the client, then the key prefix listenKey is checked, if the value of the listenKey is not null, the listenKey and the key are compared in a prefix matching mode, if the listenKey is a, the key is prefix matching of ab, the data is sent to the corresponding client, and the key + value is returned.
Key with abc of table255 updated: updating the original value data 100 to 111, fcdb detecting whether a client monitors on the table, namely checking the value of listens pointed by the table, if the value of listens is null, monitoring the table by the client, then checking the key prefix listenKey, if the value of listenKey is not null, comparing the listenKey and the key in a prefix matching mode, if the listenKey is abd, and if the key is ab, the prefix matching is inconsistent, and ignoring the update.
Example 2:
the client is started, connected with Fcdb, and appointed to monitor the number 0 table, the value of key prefix listenKey is null, prefix monitoring is selected to be carried out on table255, and the value of listenKey is abd.
The key of ab updated with table 10: the original value data 111 is updated to 010, the value of listenKey is checked to null, key and value are sent to the corresponding client, and key + value is returned.
The key of table255 is updated: updating the original value data 100 to 111, checking that the value of the listenKey is not null, comparing the listenKey with the key in a prefix matching mode, sending the data to the corresponding client and returning key + value, wherein the listenKey is abd, and the key is prefix match of abd.
Based on the same invention concept, the invention also provides a device for supporting the monitoring of the memory data table. The implementation of the device can be referred to the implementation of the method, and repeated details are not repeated. As shown in fig. 2, the apparatus 100 includes:
the definition module 101: 256 tables for defining the memory database Fcdb, and using a kv database to store data;
the client side monitoring and judging module 102: when data changes, the Fcdb detects whether a client monitors the table, and if the table is not monitored by the client, the Fcdb does not check;
the monitoring and judging module 103: if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table and sending the key and the value to the corresponding client; if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode;
the snoop matching module 104: if the key of the table matches the monitoring, the corresponding monitoring client is informed of the key and the value; if there is no match, the update is ignored.
According to the device supporting the memory data table monitoring, the memory database Fcdb stores data by using a kv database, and has high concurrent processing capacity; the memory database Fcdb checks the monitoring of the corresponding table when the key is added, modified, deleted and overtime, if yes, the notice is carried out, and the key + value is returned; in addition, prefix matching can be carried out on the monitored key.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects cannot be combined to advantage. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The limitation of the protection scope of the present invention is understood by those skilled in the art, and various modifications or changes which can be made by those skilled in the art without inventive efforts based on the technical solution of the present invention are still within the protection scope of the present invention.
Claims (14)
1. A method for supporting memory data table snooping, the method comprising:
s01: defining 256 tables of a memory database Fcdb, and using a kv database to store data;
s02: when data changes, the Fcdb detects whether a client monitors the table;
s03: if the table is not monitored by the client, the table is not checked;
s04: if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table and sending the key and the value to the corresponding client;
s05: if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode;
s06: if the key of the table matches the monitoring, notifying the key and the value to the corresponding monitoring client; if there is no match, the update is ignored.
2. The method of claim 1, wherein there are 256 arrays in the memory in S01, each array having a number, and the number has a value of 0 to 255.
3. The method of claim 2, wherein the number value corresponds to an id value of the table.
4. The method as claimed in claim 1, wherein the client in S02 stores a client structure in the database, and the client structure has a function of monitoring multiple tables.
5. The method as claimed in claim 4, wherein the database has a listening list, the listening list includes table id and list pointed by the table, if the list is not null, it indicates that there is a client listening to the table.
6. The method of claim 1, wherein the prefix in S04 is defined by a user.
7. The method of claim 1, wherein the Fcdb further has a monitoring function, and the specific steps are as follows:
s011: providing the number of the table needing monitoring;
s012: selecting whether to provide a key prefix;
s013: if the key prefix is provided, the Fcdb judges the data change according to the key prefix;
s014: if no key prefix is provided, the Fcdb listens for all keys.
8. An apparatus for supporting memory data table snooping, the apparatus comprising:
the definition module: 256 tables for defining the memory database Fcdb, and using a kv database to store data;
the client monitoring and judging module: when data changes, the Fcdb detects whether a client monitors the table, and if the table is not monitored by the client, the Fcdb does not check;
a monitoring and judging module: if the table is monitored by the client, judging whether the value of the key prefix is null, if so, indicating that the client monitors the whole table, and sending the key and the value to the corresponding client; if the value of the key prefix is not null, judging whether the key of the table is monitored or not in a prefix matching mode;
a monitoring matching module: if the key of the table matches the monitoring, the corresponding monitoring client is informed of the key and the value; if there is no match, the update is ignored.
9. The apparatus of claim 8, wherein the memory of the definition module has 256 arrays, each array having a number, the number being 0-255.
10. The apparatus of claim 9, wherein the number value corresponds to an id value of the table.
11. The apparatus as claimed in claim 8, wherein the client side snooping determining module stores a client structure in the database, and the client structure has a function of snooping a plurality of tables.
12. The apparatus as claimed in claim 11, wherein the database has a snoop list, the snoop list includes table id and list pointed to by the table, if the list is not null, it indicates that there is a client listening to the table.
13. The method of claim 8, wherein the prefix in the snoop judgment module is user-defined.
14. The apparatus of claim 8, wherein the Fcdb further comprises:
a selection module: the number of the table which needs to be monitored is provided;
a judging module: the Fcdb is used for selecting whether to provide the key prefix or not, and if the key prefix is provided, the Fcdb judges data change according to the key prefix; if no key prefix is provided, the Fcdb listens for all keys.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211184680.8A CN115794840A (en) | 2022-09-27 | 2022-09-27 | Method and device for supporting memory data table monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211184680.8A CN115794840A (en) | 2022-09-27 | 2022-09-27 | Method and device for supporting memory data table monitoring |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115794840A true CN115794840A (en) | 2023-03-14 |
Family
ID=85432245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211184680.8A Pending CN115794840A (en) | 2022-09-27 | 2022-09-27 | Method and device for supporting memory data table monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115794840A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116521688A (en) * | 2023-07-04 | 2023-08-01 | 浩鲸云计算科技股份有限公司 | Key prefix operation KEY value method based on Redis cluster |
-
2022
- 2022-09-27 CN CN202211184680.8A patent/CN115794840A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116521688A (en) * | 2023-07-04 | 2023-08-01 | 浩鲸云计算科技股份有限公司 | Key prefix operation KEY value method based on Redis cluster |
| CN116521688B (en) * | 2023-07-04 | 2023-09-26 | 浩鲸云计算科技股份有限公司 | Key prefix operation KEY value method based on Redis cluster |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11397721B2 (en) | Merging conflict resolution for multi-master distributed databases | |
| CN107273455B (en) | Block chain data access method and device | |
| US6567809B2 (en) | Disabling and reloading enterprise java beans using database trigger programs | |
| US10877956B2 (en) | Transactional cache invalidation for inter-node caching | |
| US10353895B2 (en) | Atomic visibility switch for transactional cache invalidation | |
| US7734581B2 (en) | Vector reads for array updates | |
| JP4340226B2 (en) | Providing usable versions of data items | |
| CN110213207B (en) | Network security defense method and equipment based on log analysis | |
| CN111414403B (en) | Data access method and device and data storage method and device | |
| JP2005503606A (en) | Consistent reading in a distributed database environment | |
| CN112040429B (en) | Short message management system and method based on distributed storage | |
| KR102119258B1 (en) | Technique for implementing change data capture in database management system | |
| US20130060810A1 (en) | Smart database caching | |
| CN115794840A (en) | Method and device for supporting memory data table monitoring | |
| KR20200056357A (en) | Technique for implementing change data capture in database management system | |
| CN111127252A (en) | Data management method of water resource management decision support system | |
| CN108958969A (en) | Database disaster recovery method, device and disaster recovery and backup systems | |
| CN106603610A (en) | Data acquisition method and apparatus | |
| US11003540B2 (en) | Method, server, and computer readable medium for index recovery using index redo log | |
| CN114328749A (en) | Business data processing method and device and computer readable storage medium | |
| CN114490744B (en) | Data caching method, storage medium and electronic device | |
| CN112835762B (en) | Data processing method and device, storage medium and electronic equipment | |
| CN116821232A (en) | Data synchronization method and related device | |
| CN111209304B (en) | Data processing method, device and system | |
| US8589513B1 (en) | Remote mirroring of group name services |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |