CN107766355B - Hierarchical data management method, hierarchical data management system and instant messaging system - Google Patents
Hierarchical data management method, hierarchical data management system and instant messaging system Download PDFInfo
- Publication number
- CN107766355B CN107766355B CN201610680284.2A CN201610680284A CN107766355B CN 107766355 B CN107766355 B CN 107766355B CN 201610680284 A CN201610680284 A CN 201610680284A CN 107766355 B CN107766355 B CN 107766355B
- Authority
- CN
- China
- Prior art keywords
- hierarchical data
- data
- hierarchical
- level data
- data management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/282—Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a hierarchical data management method, a hierarchical data management system and an instant messaging system. The hierarchical data management method comprises the steps of storing first hierarchical data in a non-relational database in a key value pair mode, wherein the first hierarchical data are tree-structured data; reading the first-level data into a memory, and reorganizing the first-level data in the memory to generate second-level data; and creating a cache in the memory, storing the second-level data in the cache, and managing the second-level data by using a cache elimination algorithm. The invention uses the non-relational database with higher reading and writing speed to store the hierarchical data, and remarkably improves the reading and writing efficiency of the hierarchical data by reconstructing the hierarchical data and using a cache elimination mode. The hierarchical data management system interacts with the communication system based on the flat architecture, so that the communication system based on the flat architecture can also support communication of a multi-level architecture.
Description
Technical Field
The invention relates to the field of instant messaging, in particular to a hierarchical data management method, a hierarchical data management system and an instant messaging system.
Background
The field of instant messaging is mainly based on two architectures for communication: a flat architecture and a multi-level architecture. The flat architecture is used for communication among users without the superior-subordinate relation, and the QQ group and the WeChat group are communicated based on the flat architecture; the multi-level communication architecture is used for communication among users with superior and subordinate relationships, and the nailing and enterprise WeChat are communicated based on the multi-level architecture. Because the data model of the communication software based on the flat architecture cannot process hierarchical data in a tree structure for representing a multi-level architecture, the communication software based on the flat architecture is difficult to support communication between users with upper and lower level relationships, thereby limiting the application range of the communication software.
In addition, most of software based on multi-level architecture communication is based on a relational database MySql to organize the hierarchical data, the MySql uses a hard disk as a memory, the read-write efficiency is low, SQL statements are required to be used for recursive query when the hierarchical data is read and written, and the read-write efficiency of the MySql is further reduced, so that the read-write speed of the existing software based on multi-level architecture communication using the MySql is still to be improved.
Disclosure of Invention
In order to solve the above technical problems, the present invention provides a hierarchical data management method, a hierarchical data management system and an instant messaging system. The hierarchical data management method provided by the invention uses the non-relational database with higher reading and writing speed to store the hierarchical data, and obviously improves the reading and writing efficiency of the hierarchical data by reconstructing the hierarchical data and managing the reconstructed hierarchical data by using a cache elimination algorithm. The hierarchical data management system interacts with the communication system based on the flat architecture, so that the communication system based on the flat architecture can also support communication of a multi-level architecture.
The invention is realized by the following technical scheme, and a hierarchical data management method comprises the following steps:
storing first-level data in a non-relational database in a key value pair mode, wherein the first-level data are tree-structured data;
reading the first-level data into a memory, and reorganizing the first-level data in the memory to generate second-level data;
and creating a cache in the memory, storing the second-level data in the cache, and managing the second-level data by using a cache elimination algorithm.
A hierarchical data management system comprising:
the first-level data management module is used for storing the first-level data in a non-relational database in a key value pair mode;
the second level data construction module is used for reading the first level data into the memory and reorganizing the first level data in the memory to generate second level data;
and the cache management module is used for creating a cache in an internal memory, storing the second-level data in the cache, and managing the second-level data by using a cache elimination algorithm.
An instant messaging system, comprising the above hierarchical data management system:
the hierarchical data management system further comprises an interface module, wherein the interface module comprises a relevant interface for hierarchical data management;
further comprising:
the system comprises an instant messaging server and at least one instant messaging client;
the instant communication client is used for sending out an operation instruction;
and the instant communication server is used for receiving the operation instruction and interacting with the hierarchical data management system by calling a relevant interface of the interface module.
The invention has the beneficial effects that:
the invention provides a hierarchical data management method, a hierarchical data management system and an instant messaging system. The invention has the following advantages:
(1) the hierarchical data management method organizes hierarchical data in a tree structure by using a double-layer structure, stores first hierarchical data by using a non-relational database with higher reading and writing speed, and remarkably improves the reading and writing efficiency of the hierarchical data by reconstructing the hierarchical data and managing the reconstructed hierarchical data by using a cache elimination algorithm.
(2) The algorithm logic and the storage management related to the hierarchical data management in the hierarchical data management system are realized in a modularized mode, so that the interaction between the hierarchical data management system and other systems is simplified, and the adaptability and the portability of the hierarchical data management system are enhanced.
(3) The instant messaging system based on the flat architecture can also support communication based on a multi-level architecture by interacting with the level data management system, and the use of the level data management system can enable the response speed of operation instructions related to level data in the instant messaging system to be higher, so that the performance of the instant messaging system is improved.
Drawings
FIG. 1 is a flowchart of a hierarchical data management method provided in embodiment 1;
FIG. 2(a) is a logical view of hierarchical data in example 1;
FIG. 2(b) is a schematic diagram of storing hierarchical data in a manner that MySQL is matched with an adjacency table in embodiment 1;
FIG. 3 is a schematic diagram of interaction with first-level data provided in embodiment 2;
FIG. 4 is a diagram showing interaction of layer-level data in the presence of a cache as provided in embodiment 2;
FIG. 5 is a flowchart of a method of querying hierarchical data provided in embodiment 3;
FIG. 6 is a flowchart of a method of changing hierarchical data provided in embodiment 4;
FIG. 7 is a schematic view of a hierarchical data management system provided in embodiment 5;
fig. 8 is a schematic view of an instant messaging system provided in embodiment 6;
fig. 9 is a timing chart of an inserting operation provided in embodiment 6;
fig. 10 is a timing chart of the deletion operation provided in embodiment 6;
FIG. 11 is a timing chart of a query operation provided in embodiment 6;
FIG. 12 is a timing chart of an internal migration operation provided in embodiment 6;
fig. 13 is a timing chart of the external migration operation provided in embodiment 6.
Detailed description of the invention
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1:
a hierarchical data management method, as shown in fig. 1, comprising:
s101, storing first-level data in a non-relational database in a key value pair mode, wherein the first-level data are tree-type structure data.
The first-level data is stored in the non-relational database, the non-relational database can store data in a hard disk or store data in a mode of combining an internal memory and the hard disk, and the reading and writing speed of the data is higher than that of the relational database MySql.
S102, reading the first-level data into a memory, and reorganizing the first-level data in the memory to generate second-level data.
The hierarchical data organized according to the organization mode of the database is first hierarchical data, and the hierarchical data organized according to the organization mode suitable for memory query is second hierarchical data. The second level data is stored in the memory as a cache of the first level data. Different from the prior art of copying part of data in the hard disk to the memory as a cache, after the first-level data is read into the memory, the first-level data can be randomly reconstructed in the memory according to the actual situation to generate second-level data, and the reconstruction aims at improving the query efficiency of the level data.
S103, creating a cache in the memory, storing the second-level data in the cache, and managing the second-level data by using a cache elimination algorithm.
The cache elimination algorithm is used for cache management, so that the second-level data which are not frequently used can be eliminated in time, the probability that the target data to be inquired is inquired in the cache is improved when the data is inquired, and the read-write efficiency of the level data is further optimized from the perspective of cache management.
In the field of instant messaging supporting multi-level architecture, mainstream instant messaging software such as enterprise WeChat and nailing manages hierarchical data in a tree structure in a mode of matching an adjacent table by using a relational database MySql. The management mode has the following characteristics:
(1) the function pair ratios of the relational database MySql and the non-relational database NoSql are shown in table 1:
TABLE 1 functional comparison of MySQL and NoSql
| Functional characteristics | Relational database MySql | NoSql non-relational database |
| Length of stored data | Without limitation | 64kb |
| Complex query operations | Support for | Do not support |
| Data reading efficiency | Is lower than | Is higher than |
| Data storage mode | Hard disk | Memory and hard disk |
| Automatic backup | One master and multiple slaves | One master and multiple slaves |
| Off-site storage | Support for | Support for |
| Automatic capacity expansion | Do not support | Support for |
The results of the analysis for table 1 are as follows:
in the field of instant messaging, the unlimited meaning of the data length of MySql is not large, but the long-distance storage and automatic capacity expansion have practical significance for the management of hierarchical data. MySql does not support automatic capacity expansion, thereby bringing hidden danger to the use of MySql in instant messaging software.
MySql supports complex query operation, so that the MySql management hierarchy data has the advantage of simplicity in development, but the MySql data reading efficiency is low, more query actions need to be executed on hierarchy data in the field of instant messaging, and obviously, the performance of software development by using NoSql is better.
The NoSql relies on the memory and the hard disk for data storage, and obviously has higher running speed compared with the MySql which relies on the hard disk.
In summary, in the instant messaging domain, the non-relational database NoSql has a significant advantage in managing hierarchical data compared to MySql.
(2) The adjacency list is used as a storage structure of tree-shaped structure data, and has small occupied space, high modification efficiency and low query efficiency. The SQL sentences are used for carrying out recursive query on the hierarchical data of the tree structure represented by the adjacent table, different SQL sentences have great efficiency difference, and performance risks exist.
As shown in fig. 2, for the hierarchical data in the tree structure in fig. 2(a), each row of data in MySql in fig. 2(b) records one node and one associated node, and a recursive method is required to be adopted when querying the upper level and the child nodes, which results in performance loss, and the performance loss is more serious for an inexperienced DBA or developer.
In summary, in the prior art, the hierarchical data in the tree structure is managed by using the relational database MySql in cooperation with the adjacency list, and both the read-write efficiency and the query speed are difficult to guarantee.
In the embodiment, the hierarchical data is stored and managed based on the non-relational database, so that the method has the remarkable advantages of good read-write performance and high query efficiency; in order to further improve the query efficiency of the hierarchical data, the embodiment generates the second hierarchical data with the query efficiency higher than that of the adjacency list used in the database by reconstructing the hierarchical data in the memory, and can ensure higher query performance in the actual application scene of the hierarchical data with more queries and less modifications; the cache management further improves the hit rate of the second-level data in the actual query process in this embodiment. The embodiment improves the management efficiency of the hierarchical data from three levels of data storage, data organization and cache management.
Example 2:
a hierarchical data management method, comprising:
s201, storing first-level data in a non-relational database in a key-value key value pair mode, wherein the first-level data are tree-structure data.
In the non-relational database, a key-value key value pair represents a tree, the root node of the tree is used as a key (key), and all nodes of the tree and the hierarchical relationship among the nodes are stored in values (values) corresponding to the key in an adjacent table mode. For the hierarchical data in 2(a), the first hierarchical data is stored in the non-relational database in the manner shown in table 2.
Table 2 storage mode of first hierarchy data
| key | value |
| 1 | [1,0][2,1][3,1][4,2][5,2][6,2][7,3][8,4][9,5][10,9] |
The first-level data in the non-relational database is used as the target data for data interaction, and data of the whole tree where the target data is located still needs to be pulled in each data interaction, as shown in fig. 3, the performance loss of data management still results from the full-scale pulling of the data, and in order to reduce the probability of this situation, the first-level data is reconstructed and cached in this embodiment.
S202, reading the first-level data into a memory, and reorganizing the first-level data in the memory through an associated container to generate second-level data.
The second-level data is organized to improve the query efficiency, and is cached in the memory, and the operation flow of the second-level data is as shown in fig. 4 on the premise that the second-level data exists. The data interaction comprises two modes of inquiry and change, the change operation occurrence probability and frequency are lower in the application scene of the hierarchical data, and the inquiry operation occurrence probability and frequency are higher. In fig. 4, the probability of querying the first-level data can be significantly reduced through caching, so that the query efficiency is further improved.
The cache is to realize fast query, the adjacency table used in the prior art records the mapping from the child node to the parent node, ancestor nodes can be searched through recursive query, however, the operation of querying descendant nodes has to be processed through polling, and thus the query efficiency is significantly reduced. In this embodiment, in order to ensure that the query on the ancestor node and the descendant node can have higher efficiency, the hierarchical data is reconstructed using the unordered mapping table and the unordered multiple mapping table.
Specifically, after any tree in the first-level data is read into the memory, two mapping tables for representing the tree are generated, which are an unordered mapping table (unordered _ map) for querying an ancestor node and an unordered multiple mapping table (unordered _ multimap) for querying a descendant node. The unordered mapping table stores multiple mappings from the parent node to the child node, as shown in table 4.
Table 3 unordered mapping table
Table 4 unordered multiple mapping table
| Current | Child node | |
| 1 | 2、3 | |
| 2 | 4、5、6 | |
| 3 | 7 | |
| 4 | 8 | |
| 5 | 9 | |
| 9 | 10 |
The unordered mapping table and the unordered multiple mapping table bottom layer rely on the hash table to store data, so that the method has higher query efficiency. If the ancestor of a certain node needs to be inquired, the inquiry is carried out in the unordered mapping table, and if the descendant of the certain node needs to be inquired, the inquiry is carried out in the unordered multiple mapping table, so that the inquiry of the ancestor node and the descendant node can be ensured to have higher efficiency.
As another preferred mode of this embodiment, the mapping table and the multiple mapping table can also be used to reconstruct data at the first level, and the bottom layer of the mapping table and the multiple mapping table depends on the balanced tree to store data, which also has higher query efficiency. Specifically, after any tree in the first-level data is read into the memory, two mapping tables for representing the tree are generated, which are a mapping table (map) for querying an ancestor node and a multiple mapping table (multimap) for querying a descendant node. The mapping table stores the mapping relationship from the child node to the parent node, and the multiple mapping table stores the multiple mapping relationship from the parent node to the child node. If the ancestor of a certain node needs to be inquired, the inquiry is carried out in the mapping table, and if the descendant of the certain node needs to be inquired, the inquiry is carried out in the multiple mapping tables, so that the inquiry of the ancestor node and the descendant node can be ensured to have higher efficiency.
The unordered mapping table, the unordered multiple mapping table, the mapping table, and the multiple mapping table all occupy smaller storage space than the adjacency table stored in the non-relational database, and are therefore suitable for being stored in a memory with limited space.
Furthermore, as a further alternative, the unordered mapping table, the unordered multiple mapping table, the mapping table, and the multiple mapping table may each separately reconstruct the first-level data to generate the second-level data. If the unordered mapping table or the mapping table is used independently to reconstruct the hierarchical data, the query efficiency of ancestor nodes can be only improved, and the query of descendant nodes still depends on the first hierarchical data in the non-relational database; if the unordered multiple mapping table or the multiple mapping table is used for reconstructing the first-level data independently, the query efficiency of the descendant nodes can be only improved, and the query of the ancestor nodes still depends on the first-level data in the non-relational database.
S203, creating a cache in the memory, storing the second-level data in the cache, and managing the second-level data by using a least recently used algorithm.
In consideration of the application scenarios of low speed and hierarchical data of the hard disk, in order to increase the interaction speed with the hierarchical data, a high-quality cache management algorithm needs to be adopted to reduce the probability of reading the first-level data in the hard disk.
The least recently used algorithm (LRU), also known as the least recently used algorithm, operates on the principle of locality, and eliminates the least used portion of the second level data each time cache data is exchanged.
The embodiment provides a hierarchical data management method, and a mode of using a non-relational database to cooperate with an adjacency list and reconstructing hierarchical data in a memory and performing cache management is adopted, so that the problem that bidirectional query performance is difficult to guarantee by using MySql to cooperate with the adjacency list in the prior art is solved, and bidirectional query efficiency of hierarchical data is remarkably improved.
Example 3:
a method for querying hierarchical data managed by the hierarchical data management method described in embodiment 1 or embodiment 2, as shown in fig. 5, comprising:
s301, inquiring the second-level data:
s302, if a query result is obtained, outputting the query result;
s303, if the query result is not obtained, querying the first-level data:
s304, if a query result is obtained, outputting the query result, constructing second level data corresponding to the query result, and updating the second level data in the cache according to the constructed result;
s305, if the query result is not obtained, the query is failed.
Specifically, the updating the second-level data in the cache according to the constructed result in S304 includes: if the cache capacity is sufficient, storing the constructed result in the cache; and if the cache capacity is insufficient, eliminating the least used part of the second-level data, and storing the constructed result.
The embodiment provides a method for querying hierarchical data, which can significantly improve the efficiency of hierarchical data query by fully utilizing cache to reduce the number of times of querying first hierarchical data.
Example 4:
a method of changing hierarchical data managed by the hierarchical data management method described in embodiment 1 or embodiment 2, as shown in fig. 6, comprising:
s401, acquiring first-level data to be changed;
data is read before hierarchical data change, which is a prerequisite for data change of a distributed data system, and after the data is read successfully, the data is write-protected, and other programs cannot write the data in the process of changing the data.
S402, changing the first-level data, and writing the changed first-level data into the non-relational database;
s403, reorganizing the changed first-level data, and constructing second-level data corresponding to the changed first-level data;
and S404, updating the second-level data in the cache according to the constructed result.
Specifically, changes to hierarchical data in a tree structure include inserting new nodes and deleting existing nodes.
Specifically, the updating the second-level data in the cache according to the constructed result in S404 includes: if the cache capacity is sufficient, storing the constructed result in the memory; and if the cache capacity is insufficient, eliminating the least used part of the second-level data, and storing the constructed result.
The embodiment provides a method for changing hierarchical data, which directly changes the first hierarchical data and updates the change result into a cache, so that the purpose of changing is achieved, and the speed of inquiring the change result in subsequent operations is increased.
Example 5:
a hierarchical data management system, as shown in fig. 7, comprising:
a first-level data management module 501, configured to store first-level data in a non-relational database in the form of key-value key value pairs;
a second hierarchical data construction module 502, configured to read the first hierarchical data into a memory, and reorganize the first hierarchical data in the memory to generate second hierarchical data;
the cache management module 503 is configured to create a cache in an internal memory, store the second-level data in the cache, and manage the second-level data by using a cache elimination algorithm.
Specifically, the hierarchical data management system can manage hierarchical data independently and can accept calls of other systems, so as to support the other systems to manage the hierarchical data, and the other systems can be an instant messaging system based on a flattening mechanism or an instant messaging system based on a multi-hierarchy architecture. As a preferred implementation manner, in order to interact with other systems, the present embodiment further includes an interface module 504, which is configured to provide an interface related to hierarchical data management for the other systems.
Specifically, the interface module comprises an inquiry interface, a deletion interface, an insertion interface, an internal migration interface and an external migration interface.
The internal migration interface is used for moving one or more nodes to another position in the hierarchical data with the same root node, if the moved nodes are multiple, the hierarchical relationship between the moved nodes before and after the movement is kept unchanged,
the external migration interface to move one or more nodes in a first set of hierarchical data into a second set of hierarchical data, the first set of hierarchical data having a different root node than the second set of hierarchical data; if there are a plurality of moved nodes, the hierarchical relationship between the moved nodes before and after the movement remains unchanged.
The hierarchical data management system provided in this embodiment can be used to implement the hierarchical data management method provided in embodiment 1 or embodiment 2, the hierarchical data query method provided in embodiment 3, and the hierarchical data change method provided in embodiment 4.
Example 6:
an instant messaging system, as shown in fig. 8, comprises:
the hierarchical data management system 601, where the hierarchical data management system 601 is implemented by the hierarchical data management system provided in embodiment 5, includes a first hierarchical data management module 6011, a second hierarchical data construction module 6012, a cache management module 6013, and an interface module 6014.
An instant messaging server 602 and at least one instant messaging client 603,
the instant messaging client 603 is configured to issue an operation instruction;
the instant messaging server 602 is configured to receive the operation instruction, and interact with the hierarchical data management system 601 by invoking a relevant interface of the interface module 6014.
Specifically, the operation instruction comprises a query instruction, a delete instruction, an insert instruction, an internal migration instruction and an external migration instruction. Correspondingly, the interface module comprises an inquiry interface, a deletion interface, an insertion interface, an internal migration interface and an external migration interface.
The internal migration is that one or more members are moved to another position in the same organization mechanism in a multi-level architecture, and if a plurality of moved members exist, the hierarchical relationship between the moved members before and after the movement is kept unchanged; correspondingly, the internal migration interface is used for moving one or more nodes to another position in the hierarchical data with the same root node, and if the moved nodes are multiple, the hierarchical relationship between the moved nodes before and after the movement is kept unchanged
The external migration is to move one or more members of one organization in the multi-level architecture to another organization in the multi-level architecture, and if a plurality of moved members exist, the hierarchical relationship between the moved members before and after the movement is kept unchanged; accordingly, the external migration interface is to move one or more nodes in a first set of hierarchical data into a second set of hierarchical data, the first set of hierarchical data having a different root node than the second set of hierarchical data; if there are a plurality of moved nodes, the hierarchical relationship between the moved nodes before and after the movement remains unchanged.
The flat architecture is a structure with fewer levels, for example, in a company, all departments are flat, and no relationship between upper and lower levels exists, so that the company has the flat architecture; a multi-level architecture is a structure that has a sufficient hierarchical relationship, such as a 5-level company: an office/business group/department/team is a multi-level architecture.
The instant messaging server 602 is used for independently supporting communication of a flat architecture, and can support communication of a multi-level architecture through interaction with the hierarchical data management system 601. As another embodiment, the instant messaging server 602 can also be used for communication of independent multi-level architectures, and optimize the hierarchical data management through interaction with the hierarchical data management system 601, thereby improving the communication efficiency based on the multi-level architecture. In general, the instant messaging software supporting the flat architecture is difficult to support communication in the multi-level architecture, and the instant messaging server 602 in this embodiment can support communication in the multi-level architecture by interacting with the hierarchical data management system, so as to expand the functions of the instant messaging system supporting the flat architecture.
Specifically, as shown in fig. 9, in the timing chart of the inserting operation in the instant messaging system provided in this embodiment, the instant messaging client 603 issues an inserting instruction, and the communication server 602 invokes an inserting interface provided by the interface module 6014 to complete the inserting instruction. The run logic of the plug-in interface is consistent with the change method to the hierarchical data provided in example 4. Specifically, before performing an insertion action according to a parameter carried in an insertion instruction, the parameter also needs to be subjected to validity check. For example, after the insertion instruction is completed, the maximum number of nodes of the inserted tree cannot exceed the first preset value, and the number of child nodes of each node of the inserted tree cannot exceed the second preset value.
Specifically, the present embodiment provides a timing chart of the deletion operation in the instant messaging system, as shown in fig. 10. In the instant communication system, the hierarchical relationship data only represents membership between groups, and does not involve membership, and thus, the group deletion operation is performed in two steps:
(1) all members of the group are deleted by the instant messaging server 602 supporting the flat architecture.
(2) The communication server 602 calls the deletion interface provided by the interface module 6014 to complete the deletion instruction. The operating logic of the delete interface is consistent with the change method to the hierarchical data provided in example 4. Before the delete command executes the delete action, the group to be deleted is checked for validity. For example, the group to be deleted must be the group that actually exists and can only be a leaf node in the hierarchical data.
Specifically, as shown in fig. 11, in the timing chart of the query operation in the instant messaging system provided in this embodiment, the instant messaging client 603 issues a query instruction, and the communication server 602 invokes the query interface provided by the interface module 6014 to complete the query instruction. The operating logic of the query interface is consistent with the query method for hierarchical data provided in example 3.
Specifically, in the timing chart of the internal migration operation in the instant messaging system provided in this embodiment, as shown in fig. 12, the instant messaging client 603 issues an internal migration instruction, and the communication server 602 invokes the internal migration interface provided by the interface module 6014 to complete the internal migration instruction. The operating logic of the internal migration interface is consistent with the change method to the hierarchical data provided in embodiment 4.
Specifically, in the timing diagram of the external migration operation in the instant messaging system provided in this embodiment, as shown in fig. 13, the instant messaging client 603 issues an external migration instruction, and the communication server 602 invokes the external migration interface provided by the interface module 6014 to complete the external migration instruction. The running logic of the external migration interface includes a source architecture change operation and a destination architecture change operation, and both the source architecture change operation and the destination architecture change operation are consistent with the method for changing the hierarchical data provided in embodiment 4.
All the modules in the technical scheme of the invention can be realized by a computer terminal or other equipment. The computer terminal includes a processor and a memory. The memory is used for storing the program instructions/modules in the invention, and the processor realizes the corresponding functions of the invention by operating the program instructions/modules stored in the memory.
The technical solution of the present invention may be substantially implemented or a part of or all or part of the technical solution that contributes to the prior art may be implemented in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.
The division of the modules/units described in the present invention is only a logical function division, and other division manners may be available in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. Some or all of the modules/units can be selected according to actual needs to achieve the purpose of implementing the scheme of the invention.
In addition, each module/unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (14)
1. A hierarchical data management method, comprising:
storing first-level data in a non-relational database in a key value pair mode, wherein the first-level data are tree-structured data; in the non-relational database, a key value pair represents a tree, the root node of the tree is used as a key, and all nodes of the tree and the hierarchical relationship among the nodes are stored in values corresponding to the key in an adjacent table mode;
reading the first-level data into a memory, and reorganizing the first-level data in the memory to generate second-level data;
and creating a cache in the memory, storing the second-level data in the cache, and managing the second-level data by using a cache elimination algorithm.
2. The hierarchical data management method of claim 1, wherein the second hierarchical data is reorganized through an association container.
3. The hierarchical data management method of claim 2, wherein the second hierarchical data is organized by a mapping table and a multiple mapping table, wherein the mapping table and the multiple mapping table bottom layer store data by means of a balanced tree, the mapping table stores a mapping relationship from a child node to a parent node, and the multiple mapping table stores multiple mapping relationships from a parent node to a child node.
4. The hierarchical data management method of claim 2, wherein the second hierarchical data is organized by a chaotic map and a chaotic multiple map; the unordered mapping table and the unordered multiple mapping table bottom layer rely on the hash table to store data, the mapping relation from a child node to a parent node is stored in the unordered mapping table, and the multiple mapping relation from the parent node to the child node is stored in the unordered multiple mapping table.
5. The hierarchical data management method of claim 1, wherein the second hierarchical data in the cache is managed by a least recently used algorithm.
6. The hierarchical data management method according to any one of claims 1 to 5, wherein the query method for the hierarchical data comprises:
and querying the second-level data: if the query result is obtained, outputting the query result; if not, then,
querying the first-level data: if the query result is obtained, outputting the query result, constructing second level data corresponding to the query result, and updating the second level data in the cache according to the constructed result; otherwise, the query fails.
7. The hierarchical data management method according to any one of claims 1 to 5, wherein the method of changing the hierarchical data includes:
acquiring first-level data to be changed;
changing the first-level data, and writing the changed first-level data into the non-relational database;
reorganizing the changed first-level data, constructing second-level data corresponding to the changed first-level data, and updating the second-level data in the cache according to the constructed result.
8. A hierarchical data management system, comprising:
the first-level data management module is used for storing the first-level data in a non-relational database in a key value pair mode; in the non-relational database, a key value pair represents a tree, the root node of the tree is used as a key, and all nodes of the tree and the hierarchical relationship among the nodes are stored in values corresponding to the key in an adjacent table mode;
the second level data construction module is used for reading the first level data into the memory and reorganizing the first level data in the memory to generate second level data;
and the cache management module is used for creating a cache in an internal memory, storing the second-level data in the cache, and managing the second-level data by using a cache elimination algorithm.
9. The hierarchical data management system of claim 8, further comprising an interface module comprising an associated interface for hierarchical data management.
10. The hierarchical data management system of claim 9, wherein the interface module comprises a query interface, a delete interface, and/or an insert interface.
11. The hierarchical data management system according to claim 9 or 10, wherein the interface module includes an internal migration interface and an external migration interface,
the internal migration interface is used for moving one or more nodes to another position in the hierarchical data with the same root node, if the moved nodes are multiple, the hierarchical relationship between the moved nodes before and after the movement is kept unchanged,
the external migration interface to move one or more nodes in a first set of hierarchical data into a second set of hierarchical data, the first set of hierarchical data having a different root node than the second set of hierarchical data; if there are a plurality of moved nodes, the hierarchical relationship between the moved nodes before and after the movement remains unchanged.
12. An instant messaging system comprising the hierarchical data management system of claim 8, wherein:
the hierarchical data management system comprises an interface module, wherein the interface module comprises a relevant interface for hierarchical data management;
further comprising:
the system comprises an instant messaging server and at least one instant messaging client;
the instant communication client is used for sending out an operation instruction;
and the instant communication server is used for receiving the operation instruction and interacting with the hierarchical data management system by calling a relevant interface of the interface module.
13. A storage medium storing a computer software product comprising instructions to cause a computer device to perform a method of hierarchical data management according to any of claims 1-7.
14. A computer terminal, characterized in that it comprises a processor and a memory, said processor performing a method of hierarchical data management according to any of claims 1-7 by executing program instructions/modules stored in said memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610680284.2A CN107766355B (en) | 2016-08-17 | 2016-08-17 | Hierarchical data management method, hierarchical data management system and instant messaging system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610680284.2A CN107766355B (en) | 2016-08-17 | 2016-08-17 | Hierarchical data management method, hierarchical data management system and instant messaging system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107766355A CN107766355A (en) | 2018-03-06 |
| CN107766355B true CN107766355B (en) | 2021-07-16 |
Family
ID=61259792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610680284.2A Active CN107766355B (en) | 2016-08-17 | 2016-08-17 | Hierarchical data management method, hierarchical data management system and instant messaging system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107766355B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177206B (en) * | 2018-11-09 | 2023-07-11 | 盒马(中国)有限公司 | Method, device and system for processing pivot table |
| CN109656950B (en) * | 2018-12-12 | 2020-08-07 | 上海达梦数据库有限公司 | Recursive query method, device, server and storage medium |
| CN111352915A (en) * | 2018-12-20 | 2020-06-30 | 北京奇虎科技有限公司 | Machine learning system, machine learning parameter server and implementation method |
| CN114077619A (en) * | 2020-08-20 | 2022-02-22 | 北京字节跳动网络技术有限公司 | Data query method and device, electronic equipment and storage medium |
| CN117828127A (en) * | 2023-10-27 | 2024-04-05 | 北京大学 | Tree-level cluster user management method based on semi-structured storage |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102243660A (en) * | 2011-07-18 | 2011-11-16 | 中兴通讯股份有限公司 | Data access method and device |
| CN103336849A (en) * | 2013-07-24 | 2013-10-02 | 昆明理工大学 | Method and device for increasing retrieval speed in database retrieval system |
| CN104239511A (en) * | 2014-09-15 | 2014-12-24 | 西安交通大学 | File system in user space implementation method oriented to MongoDB |
| CN105095237A (en) * | 2014-04-30 | 2015-11-25 | 国际商业机器公司 | Method and device used for generating schema of not-only-structured-query-language database |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120117067A1 (en) * | 2010-10-29 | 2012-05-10 | Navteq North America, Llc | Method and apparatus for providing a range ordered tree structure |
-
2016
- 2016-08-17 CN CN201610680284.2A patent/CN107766355B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102243660A (en) * | 2011-07-18 | 2011-11-16 | 中兴通讯股份有限公司 | Data access method and device |
| CN103336849A (en) * | 2013-07-24 | 2013-10-02 | 昆明理工大学 | Method and device for increasing retrieval speed in database retrieval system |
| CN105095237A (en) * | 2014-04-30 | 2015-11-25 | 国际商业机器公司 | Method and device used for generating schema of not-only-structured-query-language database |
| CN104239511A (en) * | 2014-09-15 | 2014-12-24 | 西安交通大学 | File system in user space implementation method oriented to MongoDB |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107766355A (en) | 2018-03-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109254733B (en) | Method, device and system for storing data | |
| CN107766355B (en) | Hierarchical data management method, hierarchical data management system and instant messaging system | |
| KR102177190B1 (en) | Managing data with flexible schema | |
| US10331641B2 (en) | Hash database configuration method and apparatus | |
| US10242050B2 (en) | Database caching in a database system | |
| US10275489B1 (en) | Binary encoding-based optimizations at datastore accelerators | |
| CN102663117B (en) | OLAP (On Line Analytical Processing) inquiry processing method facing database and Hadoop mixing platform | |
| CN103106286B (en) | Method and device for managing metadata | |
| US11561930B2 (en) | Independent evictions from datastore accelerator fleet nodes | |
| CN107807932B (en) | Hierarchical data management method and system based on path enumeration | |
| US20090012932A1 (en) | Method and System For Data Storage And Management | |
| CN104239511B (en) | A kind of user's space file system implementation method towards MongoDB | |
| CN102915278A (en) | Data deduplication method | |
| CN105701219B (en) | A kind of implementation method of distributed caching | |
| CN103473239A (en) | Method and device for updating data of non relational database | |
| CN104572920A (en) | Data arrangement method and data arrangement device | |
| CN104054071A (en) | Method for accessing storage device and storage device | |
| CN106682139A (en) | Method and system for achieving HBase multi-condition query based on Solr | |
| CN103365987B (en) | Clustered database system and data processing method based on shared-disk framework | |
| CN110399096A (en) | Metadata of distributed type file system caches the method, apparatus and equipment deleted again | |
| CN111324305A (en) | Data writing/reading method in distributed storage system | |
| CN103942301A (en) | Distributed file system oriented to access and application of multiple data types | |
| CN109086462A (en) | The management method of metadata in a kind of distributed file system | |
| US10146833B1 (en) | Write-back techniques at datastore accelerators | |
| CN103530067B (en) | A kind of method and apparatus of data manipulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230922 Address after: 518057 Tencent Building, No. 1 High-tech Zone, Nanshan District, Shenzhen City, Guangdong Province, 35 floors Patentee after: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. Patentee after: TENCENT CLOUD COMPUTING (BEIJING) Co.,Ltd. Address before: 2, 518000, East 403 room, SEG science and Technology Park, Zhenxing Road, Shenzhen, Guangdong, Futian District Patentee before: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. |