CN114185678A - Data storage method, device, equipment and storage medium - Google Patents
Data storage method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114185678A CN114185678A CN202111528534.8A CN202111528534A CN114185678A CN 114185678 A CN114185678 A CN 114185678A CN 202111528534 A CN202111528534 A CN 202111528534A CN 114185678 A CN114185678 A CN 114185678A
- Authority
- CN
- China
- Prior art keywords
- cluster
- node
- hash
- hash ring
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data storage method, a device, equipment and a storage medium, and data to be stored are obtained; calculating a hash value corresponding to the data to be stored by adopting a preset hash function; determining a target node of the hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in a preset hash ring; different hash ring nodes are mapped with different fragmentation cluster examples; respectively storing data to be stored into a fragmentation cluster example mapped by a target node and a fragmentation cluster example mapped by a hash ring node adjacent to the target node; the method and the device for storing the data in the distributed database cluster can solve the problem that the existing distributed database cluster is poor in reliability of data storage.
Description
Technical Field
The present application belongs to the technical field of data storage, and in particular, to a data storage method, apparatus, device, and storage medium.
Background
With the increasing of the total data production amount of various industries in recent years, the single-node fragmentation cluster example has difficulty in meeting the requirements of mass data storage and high-concurrency data access quick response at the same time. Distributed fragmented cluster examples come from the beginning, which effectively alleviate the problem of mass data storage by segmenting data and data tables through multiple nodes, and the realization of high-availability and low-delay data access through load balancing, a dual-computer cluster system (HA) and other modes HAs become the mainstream choice of the big data industry. However, as the total amount of data storage is continuously increased, the distributed database cluster inevitably needs to be expanded by adding new nodes to meet the service requirement of larger data volume.
The existing distributed database cluster adopts a fragment cluster to store data, and when a certain fragment cluster is unavailable, the storage mode can cause the loss of corresponding stored data, destroy the integrity of the stored data and have poor reliability of the stored data.
Disclosure of Invention
The embodiment of the application provides a data storage method, a data storage device, data storage equipment and a data storage medium, and can solve the problem that the existing distributed database cluster is poor in reliability of data storage.
In a first aspect, an embodiment of the present application provides a data storage method, including:
acquiring data to be stored;
calculating a hash value corresponding to the data to be stored by adopting a preset hash function;
determining a target node of the hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in a preset hash ring; different hash ring nodes are mapped with different fragmentation cluster examples;
and respectively storing the data to be stored into the fragment cluster example mapped by the target node and the fragment cluster example mapped by the hash ring node adjacent to the target node.
In one embodiment, before determining a target node of a hash ring corresponding to a hash value by comparing the hash value with a predetermined label hash value interval of each node in the hash ring, the method further includes:
dividing each fragment cluster instance in the database cluster into a preset number of virtual nodes respectively;
and respectively establishing a mapping relation between the virtual node and each node in a preset hash ring.
In one embodiment, the method further comprises:
under the condition that a database cluster capacity expansion instruction is received, a fragmentation cluster example corresponding to the database cluster capacity expansion instruction is newly added in the database cluster, and a hash ring node is newly added in a preset hash ring;
creating a mapping relation between the newly added Hash link point and each virtual node in the newly added fragment cluster example;
and copying the data stored in the newly added fragment cluster example to the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node, and copying the data stored in the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node to the newly added fragment cluster example.
In one embodiment, the method further comprises:
and under the condition of receiving the database cluster capacity reduction instruction, deleting the fragment cluster instance corresponding to the database cluster capacity reduction instruction, and deleting the hash ring node mapped by the deleted fragment cluster instance.
In a second aspect, an embodiment of the present application provides a data storage device, including:
the acquisition module is used for acquiring data to be stored;
the calculation module is used for calculating a hash value corresponding to the data to be stored by adopting a preset hash function;
the determining module is used for determining a target node of the hash ring corresponding to the hash value by comparing the hash value with the label hash value interval of each node in the preset hash ring; different hash ring nodes are mapped with different fragmentation cluster examples;
and the storage module is used for respectively storing the data to be stored into the fragment cluster example mapped by the target node and the fragment cluster example mapped by the hash ring node adjacent to the target node.
In one embodiment, the apparatus further comprises a partitioning module and a creating module;
the dividing module is used for dividing each fragment cluster instance in the database cluster into a preset number of virtual nodes respectively before determining a target node of a hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in the preset hash ring;
and the creating module is used for creating a mapping relation between the virtual node and each node in a preset hash ring.
In one embodiment, the apparatus further comprises an adding module and a copying module;
the system comprises a newly-added module, a hash ring node and a database cluster expansion module, wherein the newly-added module is used for adding a fragment cluster instance corresponding to a database cluster expansion instruction in a database cluster and adding a hash ring node in a preset hash ring under the condition of receiving the database cluster expansion instruction;
the creating module is further used for creating a mapping relation between the newly added hash link point and each virtual node in the newly added fragment cluster example;
and the copying module is used for copying the data stored in the newly added fragment cluster example to the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node, and copying the data stored in the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node to the newly added fragment cluster example.
In one embodiment, the apparatus further comprises a deletion module;
and the deleting module is used for deleting the fragmentation cluster instance corresponding to the database cluster capacity reduction instruction and deleting the hash ring node mapped by the deleted fragmentation cluster instance under the condition of receiving the database cluster capacity reduction instruction.
In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the data storage method of the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which an implementation program for information transfer is stored, where the program, when executed by a processor, implements the data storage method of the first aspect.
In a fifth aspect, the present application provides a computer program product, and when executed by a processor of an electronic device, the instructions in the computer program product cause the electronic device to execute the data storage method of the first aspect.
According to the data storage method, the data storage device, the data storage equipment and the data storage medium, the data to be stored are stored in a mode of mapping data by using nodes of a hash ring, the hash value corresponding to the data to be stored is calculated by adopting a preset hash function, then the target node of the hash ring corresponding to the hash value is determined by comparing the hash value with the label hash value interval of each node in the preset hash ring, and further the fragmentation cluster example for storing the data to be stored is determined, so that the load balance of the fragmentation cluster example is ensured; and the data to be stored are respectively stored in the fragmented cluster instance mapped by the target node and the fragmented cluster instance mapped by the hash ring node adjacent to the target node, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragmented cluster instance, the integrity of the stored data is kept, and the high reliability of the database cluster is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data storage method provided in an embodiment of the present application;
fig. 2 is a schematic diagram of a hash ring according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a data storage device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application will be described in detail below, and in order to make objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by illustrating examples thereof.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The existing distributed database cluster adopts a fragment cluster to store data, and when a certain fragment cluster is unavailable, the storage mode can cause the loss of corresponding stored data, destroy the integrity of the stored data and have poor reliability of the stored data. For example, the distributed sharded cluster instance MongoDB is a powerful, flexible and easily extensible universal sharded cluster instance, and is widely applied in the current market. The existing expansion mode of the MongoDB shard cluster capacity is to create a plurality of MongoDB shard cluster instances and then distribute requests to different MongoDB shard clusters through a reverse proxy server (HAproxy), thereby realizing load balancing and achieving the purpose of capacity expansion. The problem that the high availability of data cannot be guaranteed by a mode of realizing MongoDB fragmentation cluster expansion by introducing a reverse proxy server. When a certain MongoDB fragment cluster is unavailable, the accessed data is lost and integrity is affected, and the processing flow of the service is affected.
In order to solve the problems in the prior art, embodiments of the present application provide a data storage method, apparatus, device, and storage medium. According to the embodiment of the application, the data to be stored is stored in a mode of mapping data by using nodes of the hash ring, the hash value corresponding to the data to be stored is calculated by adopting a preset hash function, then the target node of the hash ring corresponding to the hash value is determined by comparing the hash value with the label hash value interval of each node in the preset hash ring, and then the fragmentation cluster example for storing the data to be stored is determined, so that the load balance of the fragmentation cluster example storage is ensured; and the data to be stored are respectively stored in the fragmented cluster instance mapped by the target node and the fragmented cluster instance mapped by the hash ring node adjacent to the target node, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragmented cluster instance, the integrity of the stored data is kept, and the high reliability of the database cluster is ensured. First, a data storage method provided in an embodiment of the present application is described below.
Fig. 1 shows a schematic flow chart of a data storage method according to an embodiment of the present application. As shown in fig. 1, the method may include the steps of:
and S110, acquiring data to be stored.
When the data to be stored is obtained, the data to be stored can be obtained through communication connection with the equipment for storing the data to be stored.
And S120, calculating a hash value corresponding to the data to be stored by adopting a preset hash function.
The preset hash function can calculate a corresponding hash value based on the tape storage data, and the preset hash function can be a Murmur3 hash algorithm.
And S130, determining a target node of the hash ring corresponding to the hash value by comparing the hash value with the label hash value interval of each node in the preset hash ring.
Wherein, different hash ring nodes are mapped with different fragmentation cluster instances, and a hash ring may include four nodes T1, T2, T3, and T4 as shown in fig. 2. Each node in the preset hash ring is pre-labeled with a labeled hash value interval according to a preset rule, for example, the preset rule may be: the hash ring gradually increases the label hash value interval marked by each node clockwise.
And S140, respectively storing the data to be stored into the fragment cluster example mapped by the target node and the fragment cluster example mapped by the hash ring node adjacent to the target node.
As shown in fig. 2, for example, the data to be stored is M, the target node is T1, and when storing, M may be stored in each of the following combinations: the sharded cluster instances corresponding to T1 and T2 respectively; the sharded cluster instances corresponding to T1 and T4 respectively; t1, T2, and T4 correspond to shard cluster instances, respectively. In the storage mode, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash link point adjacent to the hash ring node mapped by the fragmented cluster instance, so that the integrity of the stored data is maintained, and the high reliability of the database cluster is ensured.
According to the embodiment of the application, the data to be stored is stored in a mode of mapping data by using nodes of the hash ring, the hash value corresponding to the data to be stored is calculated by adopting a preset hash function, then the target node of the hash ring corresponding to the hash value is determined by comparing the hash value with the label hash value interval of each node in the preset hash ring, and then the fragmentation cluster example for storing the data to be stored is determined, so that the load balance of the fragmentation cluster example storage is ensured; and the data to be stored are respectively stored in the fragmented cluster instance mapped by the target node and the fragmented cluster instance mapped by the hash ring node adjacent to the target node, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragmented cluster instance, the integrity of the stored data is kept, and the high reliability of the database cluster is ensured.
In one embodiment, at S130: before determining the target node of the hash ring corresponding to the hash value by comparing the hash value with the label hash value interval of each node in the preset hash ring, the method may further include:
and dividing each fragment cluster instance in the database cluster into a preset number of virtual nodes.
And respectively establishing a mapping relation between the virtual node and each node in a preset hash ring.
In consideration of the diversity of hash values corresponding to data to be stored, which are calculated by using a preset hash function, the efficiency of determining that the sharded cluster instances to be stored are to be stored by comparing a single sharded cluster instance with massive hash values is low, firstly, each sharded cluster instance in a database cluster is divided into a preset number of virtual nodes, for example, into 128 virtual nodes, then the virtual nodes are respectively mapped with each node in a preset hash ring, when comparing, the virtual nodes corresponding to the hash ring can be determined firstly, then the virtual nodes are used for determining the sharded cluster instances to be responded, and the efficiency of determining that the sharded cluster instances to be stored are to be stored by the data is improved.
In one embodiment, the method may further comprise:
under the condition that a database cluster capacity expansion instruction is received, a fragmentation cluster example corresponding to the database cluster capacity expansion instruction is newly added in the database cluster, and a hash ring node is newly added in a preset hash ring; creating a mapping relation between the newly added Hash link point and each virtual node in the newly added fragment cluster example; and copying the data stored in the newly added fragment cluster example to the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node, and copying the data stored in the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node to the newly added fragment cluster example.
In the embodiment of the application, under the condition of database cluster expansion, data stored in a newly-added fragment cluster instance is copied to a fragment cluster instance mapped by a hash ring node adjacent to the newly-added hash ring node, and data stored in a fragment cluster instance mapped by a hash ring node adjacent to the newly-added hash ring node is copied to the newly-added fragment cluster instance, so that even if a certain fragment cluster instance is unavailable, the data stored in the unavailable fragment cluster instance can still be accessed in the fragment cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragment cluster instance.
In one embodiment, the method further comprises:
and under the condition of receiving the database cluster capacity reduction instruction, deleting the fragment cluster instance corresponding to the database cluster capacity reduction instruction, and deleting the hash ring node mapped by the deleted fragment cluster instance.
In the embodiment of the application, under the condition of database cluster capacity reduction, although the fragmentation cluster instance corresponding to the database cluster capacity reduction instruction and the hash ring node mapped by the deleted fragmentation cluster instance are deleted, the data stored in the deleted fragmentation cluster instance can still be accessed in the fragmentation cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the deleted fragmentation cluster instance, so that the high reliability of the database cluster is ensured.
According to the embodiment of the application, the data to be stored is stored in a mode of mapping data by using nodes of the hash ring, the hash value corresponding to the data to be stored is calculated by adopting a preset hash function, then the target node of the hash ring corresponding to the hash value is determined by comparing the hash value with the label hash value interval of each node in the preset hash ring, and then the fragmentation cluster example for storing the data to be stored is determined, so that the load balance of the fragmentation cluster example storage is ensured; and the data to be stored are respectively stored in the fragmented cluster instance mapped by the target node and the fragmented cluster instance mapped by the hash ring node adjacent to the target node, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragmented cluster instance, the integrity of the stored data is kept, and the high reliability of the database cluster is ensured.
Fig. 1-2 illustrate a data storage method, and the following describes a device provided by an embodiment of the present application with reference to fig. 3 and 4.
Fig. 3 is a schematic structural diagram of a data storage device according to an embodiment of the present application, where each module in the device shown in fig. 3 has a function of implementing each step in fig. 1, and can achieve its corresponding technical effect. As shown in fig. 3, the apparatus may include:
the obtaining module 310 is configured to obtain data to be stored.
The calculating module 320 is configured to calculate a hash value corresponding to the data to be stored by using a preset hash function.
The determining module 330 is configured to determine a target node of the hash ring corresponding to the hash value by comparing the hash value with a preset label hash value interval of each node in the hash ring. Different hash ring nodes are mapped with different fragmentation cluster instances.
The storage module 340 is configured to store data to be stored in the shard cluster instance mapped by the target node and the shard cluster instance mapped by the hash ring node adjacent to the target node.
According to the embodiment of the application, the data to be stored is stored through the mode of the node mapping data of the hash ring, the hash value corresponding to the data to be stored is calculated through the preset hash function, then the target node of the hash ring corresponding to the hash value is determined by comparing the hash value with the label hash value interval of each node in the preset hash ring, and then the fragment cluster example for storing the data to be stored is determined, so that the load balance of the fragment cluster example storage is ensured. And the data to be stored are respectively stored in the fragmented cluster instance mapped by the target node and the fragmented cluster instance mapped by the hash ring node adjacent to the target node, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragmented cluster instance, the integrity of the stored data is kept, and the high reliability of the database cluster is ensured.
In one embodiment, the apparatus further comprises a partitioning module 350 and a creating module 360.
The dividing module 350 is configured to divide each shard cluster instance in the database cluster into a preset number of virtual nodes respectively before determining a target node of the hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in a preset hash ring.
A creating module 360, configured to create a mapping relationship between the virtual node and each node in the preset hash ring.
In one embodiment, the apparatus further includes an add module 370 and a copy module 380.
And an adding module 370, configured to, in a case that the database cluster capacity expansion instruction is received, add a new hash ring node in the preset hash ring and a fragmented cluster instance corresponding to the database cluster capacity expansion instruction in the database cluster.
The creating module 360 is further configured to create a mapping relationship between the newly added hash link point and each virtual node in the newly added segment cluster instance.
The copying module 380 is configured to copy data stored in the newly added segment cluster instance to the segment cluster instance mapped by the hash ring node adjacent to the newly added hash ring node, and copy data stored in the segment cluster instance mapped by the hash ring node adjacent to the newly added hash ring node to the newly added segment cluster instance.
In one embodiment, the apparatus further comprises a deletion module 390.
The deleting module 390 is configured to, in a case that the database cluster capacity reduction instruction is received, delete the shard cluster instance corresponding to the database cluster capacity reduction instruction, and delete the hash ring node mapped by the deleted shard cluster instance.
According to the embodiment of the application, the data to be stored is stored in a mode of mapping data by using nodes of the hash ring, the hash value corresponding to the data to be stored is calculated by adopting a preset hash function, then the target node of the hash ring corresponding to the hash value is determined by comparing the hash value with the label hash value interval of each node in the preset hash ring, and then the fragmentation cluster example for storing the data to be stored is determined, so that the load balance of the fragmentation cluster example storage is ensured; and the data to be stored are respectively stored in the fragmented cluster instance mapped by the target node and the fragmented cluster instance mapped by the hash ring node adjacent to the target node, even if a certain fragmented cluster instance is unavailable, the data stored in the unavailable fragmented cluster instance can be accessed in the fragmented cluster instance corresponding to the hash ring node adjacent to the hash ring node mapped by the fragmented cluster instance, the integrity of the stored data is kept, and the high reliability of the database cluster is ensured.
Fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 4, the apparatus may include a processor 401 and a memory 402 storing computer program instructions.
Specifically, the processor 401 may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement the embodiments of the present Application.
In one example, the Memory 402 may be a Read Only Memory (ROM). In one example, the ROM may be mask programmed ROM, programmable ROM (prom), erasable prom (eprom), electrically erasable prom (eeprom), electrically rewritable ROM (earom), or flash memory, or a combination of two or more of these.
The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement the method in the embodiment shown in fig. 1, and achieve the corresponding technical effect achieved by the embodiment shown in fig. 1 executing the method, which is not described herein again for brevity.
In one example, the electronic device can also include a communication interface 403 and a bus 410. As shown in fig. 4, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 to complete communication therebetween.
The communication interface 403 is mainly used for implementing communication between modules, apparatuses, units and/or devices in the embodiments of the present application.
The electronic device can execute the data storage method in the embodiment of the application, so that the corresponding technical effects of the data storage method described in fig. 1 are achieved.
In addition, in combination with the data storage method in the foregoing embodiments, the embodiments of the present application may provide a computer storage medium to implement. The computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement any of the data storage methods of the above embodiments.
It is to be understood that the present application is not limited to the particular arrangements and instrumentality described above and shown in the attached drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions or change the order between the steps after comprehending the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic Circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware for performing the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As described above, only the specific embodiments of the present application are provided, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the module and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, and these modifications or substitutions should be covered within the scope of the present application.
Claims (11)
1. A method of storing data, comprising:
acquiring data to be stored;
calculating a hash value corresponding to the data to be stored by adopting a preset hash function;
determining a target node of the hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in a preset hash ring; different hash ring nodes are mapped with different fragmentation cluster examples;
and respectively storing the data to be stored into the fragment cluster example mapped by the target node and the fragment cluster example mapped by the hash ring node adjacent to the target node.
2. The data storage method according to claim 1, wherein before determining the target node of the hash ring corresponding to the hash value by comparing the hash value with the label hash value interval of each node in the preset hash ring, the method further comprises:
dividing each fragment cluster instance in the database cluster into a preset number of virtual nodes respectively;
and respectively establishing a mapping relation between the virtual node and each node in the preset hash ring.
3. The data storage method of claim 2, wherein the method further comprises:
under the condition that a database cluster capacity expansion instruction is received, adding a fragmentation cluster example corresponding to the database cluster capacity expansion instruction in the database cluster and adding a hash ring node in the preset hash ring;
creating a mapping relation between the newly added Hash link point and each virtual node in the newly added fragment cluster example;
and copying the data stored in the newly added fragment cluster example to the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node, and copying the data stored in the fragment cluster example mapped by the hash ring node adjacent to the newly added hash ring node to the newly added fragment cluster example.
4. A data storage method according to any one of claims 1-3, wherein the method further comprises:
and under the condition of receiving a database cluster capacity reduction instruction, deleting a fragmentation cluster instance corresponding to the database cluster capacity reduction instruction, and deleting a hash ring node mapped by the deleted fragmentation cluster instance.
5. A data storage device, comprising:
the acquisition module is used for acquiring data to be stored;
the calculation module is used for calculating a hash value corresponding to the data to be stored by adopting a preset hash function;
the determining module is used for determining a target node of the hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in a preset hash ring; different hash ring nodes are mapped with different fragmentation cluster examples;
and the storage module is used for respectively storing the data to be stored into the fragment cluster example mapped by the target node and the fragment cluster example mapped by the hash ring node adjacent to the target node.
6. The data storage device of claim 5, wherein the device further comprises a partitioning module and a creating module;
the dividing module is used for dividing each fragment cluster instance in the database cluster into a preset number of virtual nodes respectively before determining a target node of a hash ring corresponding to the hash value by comparing the hash value with a label hash value interval of each node in a preset hash ring;
and the creating module is used for creating a mapping relationship between the virtual node and each node in the preset hash ring.
7. The data storage device of claim 6, wherein the device further comprises an add-on module and a copy module;
the newly-added module is used for newly adding a fragment cluster instance corresponding to the database cluster capacity expansion instruction in the database cluster and newly adding a hash ring node in the preset hash ring under the condition that the database cluster capacity expansion instruction is received;
the creating module is further configured to create a mapping relationship between the newly added hash link point and each virtual node in the newly added fragmentation cluster instance;
the replication module is configured to replicate data stored in the newly added segment cluster instance to a segment cluster instance mapped by a hash ring node adjacent to the newly added hash ring node, and replicate data stored in the segment cluster instance mapped by the hash ring node adjacent to the newly added hash ring node to the newly added segment cluster instance.
8. The data storage device of any of claims 5-7, wherein the device further comprises a deletion module;
and the deleting module is used for deleting the fragmentation cluster instance corresponding to the database cluster capacity reduction instruction and deleting the hash ring node mapped by the deleted fragmentation cluster instance under the condition of receiving the database cluster capacity reduction instruction.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, implements a data storage method as claimed in any one of claims 1 to 4.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon an implementation program of information transfer, which when executed by a processor implements the data storage method of any one of claims 1 to 4.
11. A computer program product, wherein instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the data storage method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111528534.8A CN114185678A (en) | 2021-12-14 | 2021-12-14 | Data storage method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111528534.8A CN114185678A (en) | 2021-12-14 | 2021-12-14 | Data storage method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114185678A true CN114185678A (en) | 2022-03-15 |
Family
ID=80543786
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111528534.8A Pending CN114185678A (en) | 2021-12-14 | 2021-12-14 | Data storage method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114185678A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114666338A (en) * | 2022-05-19 | 2022-06-24 | 杭州指令集智能科技有限公司 | Message-based multi-instance load balancing method and system |
| CN115001969A (en) * | 2022-05-24 | 2022-09-02 | 中欣链证数字科技有限公司 | Data storage node deployment method, data storage method, device and equipment |
| CN115442439A (en) * | 2022-08-31 | 2022-12-06 | 云知声智能科技股份有限公司 | Distributed cache cluster management method, system, terminal and storage medium |
| CN118409717A (en) * | 2024-07-03 | 2024-07-30 | 济南浪潮数据技术有限公司 | Data distribution method, system, computer program product, equipment and medium |
-
2021
- 2021-12-14 CN CN202111528534.8A patent/CN114185678A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114666338A (en) * | 2022-05-19 | 2022-06-24 | 杭州指令集智能科技有限公司 | Message-based multi-instance load balancing method and system |
| CN114666338B (en) * | 2022-05-19 | 2022-08-26 | 杭州指令集智能科技有限公司 | Message-based multi-instance load balancing method and system |
| CN115001969A (en) * | 2022-05-24 | 2022-09-02 | 中欣链证数字科技有限公司 | Data storage node deployment method, data storage method, device and equipment |
| CN115442439A (en) * | 2022-08-31 | 2022-12-06 | 云知声智能科技股份有限公司 | Distributed cache cluster management method, system, terminal and storage medium |
| CN118409717A (en) * | 2024-07-03 | 2024-07-30 | 济南浪潮数据技术有限公司 | Data distribution method, system, computer program product, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114185678A (en) | Data storage method, device, equipment and storage medium | |
| CN110574020B (en) | Method, system and computer readable storage medium for snapshot management in blockchain | |
| CN109492013B (en) | Data processing method, device and system applied to database cluster | |
| CN108769264B (en) | Block chain domain division method | |
| CN109951508B (en) | Zookeeper-based request coordination method and device, computer equipment and storage medium | |
| CN111858146B (en) | Method, apparatus and computer program product for recovering data | |
| KR20150111952A (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN111464353A (en) | Block link point management method, device, computer and readable storage medium | |
| CN112671908B (en) | Network management method and device, electronic equipment and readable storage medium | |
| CN112671881B (en) | Node organization management method and device, electronic equipment and readable storage medium | |
| CN111444043A (en) | Data backup method, device, node, system and medium | |
| CN112436962B (en) | Block chain consensus network dynamic expansion method, electronic device, system and medium | |
| CN117014318A (en) | Method, device, equipment and medium for adding links between multi-scale network nodes | |
| CN115361295B (en) | TOPSIS-based resource backup method, device, equipment and medium | |
| CN113448770A (en) | Method, electronic device and computer program product for recovering data | |
| CN111198782A (en) | Data redistribution method, device, equipment and storage medium | |
| CN111833608B (en) | Road condition information processing method and device | |
| CN111125163A (en) | Method and apparatus for processing data | |
| CN114791912A (en) | Data processing method, system, electronic equipment and storage medium | |
| CN107203559B (en) | Method and device for dividing data strips | |
| CN114911515A (en) | Configuration management method, device, equipment and medium | |
| CN110166473B (en) | Network data transmission detection method, device, equipment and medium | |
| CN114579662A (en) | Memory database synchronization method, system, equipment and storage medium | |
| CN112910936A (en) | Data processing method, device and system, electronic equipment and readable storage medium | |
| CN111831211A (en) | Data transmission method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |