CN106970960A - A kind of data placement method of distributed memory system - Google Patents
A kind of data placement method of distributed memory system Download PDFInfo
- Publication number
- CN106970960A CN106970960A CN201710165599.8A CN201710165599A CN106970960A CN 106970960 A CN106970960 A CN 106970960A CN 201710165599 A CN201710165599 A CN 201710165599A CN 106970960 A CN106970960 A CN 106970960A
- Authority
- CN
- China
- Prior art keywords
- node
- copy
- intermediate node
- count value
- memory system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
Abstract
The invention discloses a kind of data placement method of distributed memory system, including：First node sends the copy of data block and copy count value any one intermediate node into node listing when receiving the operation for writing data block；Intermediate node is after copy is received, stored copies, and judges whether current copy count value is 0, if being not 0, current copy count value is subtracted 1, copy and current copy count value are sent into present node list any one remaining intermediate node, if 0, then terminate.Due to this method and the expense of ghost need not be concentrated on one node, therefore the performance of system will not be reduced with the increase of redundancy, and the reliability of system is ensure that to a certain extent.Further, since being the node of any selection, therefore transmission rule need not be pre-set, the Placement Problems of the copy under dynamic network environment can be efficiently solved.
Description
Technical field
The present invention relates to technical field of distributed memory, the data placement side of more particularly to a kind of distributed memory system
Method.
Background technology
In distributed memory system, include multiple nodes, node centered on one of those, multiple nodes cooperate with work
Make to complete various tasks.In specific implementation, in order to improve the reliability of system, it usually needs improve the redundancy of data.When
When writing the operation of data block, not only first node will store the data block, and need other nodes to store the data block
Copy.Here first node refers to initiate the node where the client of write request, that is, is written directly the node of data block.
In the prior art, the prevention method for the copy of same data block mainly includes two kinds.One kind is that order is put
Strategy is put, one kind is to randomly place strategy.Order Placement Strategy：Each node is typically regarded as logic orderly, right
First all copies of same data block are numbered when copy is allocated, it is then using fixed mapping mode that each is secondary
Originally on the node for being placed into correspondence sequence number.It has the disadvantage that (i.e. node increase or deletion, such as failure, expansion subtract when there is node turnover
Hold) when, the harmony of copy it is difficult to ensure that, and rearranging node sequence and recovering original data distribution can bring huge
Network transmission expense.Randomly place strategy：The placement catalogue that some hash function carrys out determination data is normally based on, thus it is more
When be properly termed as pseudo-random placement strategy.Which is to be responsible for determining the placement location of all copies by Centroid,
The generation of multiple copies is completed by first node, the placement location then obtained according to Centroid is placed.Its shortcoming is put with order
Put tactful identical, the performance that node dynamically passes in and out (namely node increase or deletion, such as failure, expansion volume reduction) central node turns into
Bottleneck, the performance to system produces considerable influence, while the central node failure, can also influence the reliability of whole system.
As can be seen here, how to overcome because node is dynamically passed in and out and causes copy can not be deposited according to original plan
Storage, so that the shortcoming for causing system reliability reduction is those skilled in the art's urgent problem to be solved.
The content of the invention
It is an object of the invention to provide a kind of data placement method of distributed memory system, for overcoming because node is dynamic
State passes in and out and causes copy can not be stored according to original plan, so as to cause the problem of system reliability is reduced.
In order to solve the above technical problems, the present invention provides a kind of data placement method of distributed memory system, including：
First node sends the copy of the data block and copy count value to section when receiving the operation for writing data block
Any one intermediate node in point list；
Intermediate node stores the copy after the copy is received, and judges whether current copy count value is 0,
If being not 0, current copy count value is subtracted 1, the copy and current copy count value are sent to present node list
In any one remaining intermediate node, if 0, then terminate.
Preferably, also include in the case where current copy count value is not 0：
Intermediate node judges whether also there is the intermediate node for not receiving the copy in present node list；
If it is present subtract 1 by current copy count value, the copy and current copy count value are sent to working as prosthomere
Any one remaining intermediate node in point list；
If it does not exist, then judging whether present node list has new intermediate node to add；
If new intermediate node is added, then current copy count value is subtracted 1, the copy and current copy are counted
Value is sent to new intermediate node；
If added without new intermediate node, return and described judge whether present node list has new intermediate node
The step of addition.
Preferably, IP address of the node listing comprising the first node and each intermediate node.
Preferably, the copy count value is less than the total quantity of the node listing interior joint.
Preferably, in addition to：The first node record is received in the copy and the copy count value that it sends
Intermediate node.
Preferably, in addition to：Each intermediate node record is received in the copy and current copy count value that it sends
Intermediate node.
Preferably, in addition to：Centroid records the data block and the copy in the first node and each middle node
The transmission locus of point.
Preferably, in addition to：The quantity of copy described in the current distributed memory system of Centroid real-time update.
Preferably also include：The Centroid is pointed out in the quantity of the copy less than threshold value alarm.
The data placement method of distributed memory system provided by the present invention, copy is realized by the forwarding between node
Storage, and then realize the asynchronous write operation in network, due to this method and the expense of ghost need not be concentrated on one
On individual node, but share on each node, therefore the performance of system will not be reduced with the increase of redundancy, to a certain degree
On ensure that the reliability of system.Further, since being the node of any selection, therefore transmission rule, energy need not be pre-set
Enough efficiently solve the Placement Problems of the copy under dynamic network environment.
Brief description of the drawings
In order to illustrate the embodiments of the present invention more clearly, the required accompanying drawing used in embodiment will be done simply below
Introduce, it should be apparent that, drawings in the following description are only some embodiments of the present invention, for ordinary skill people
For member, on the premise of not paying creative work, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is a kind of flow chart of the data placement method of distributed memory system provided in an embodiment of the present invention；
Fig. 2 is a kind of flow chart of the data placement method of intermediate node provided in an embodiment of the present invention；
Fig. 3 is the flow chart of the data placement method of another distributed memory system provided in an embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.Based on this
Embodiment in invention, those of ordinary skill in the art are not under the premise of creative work is made, and what is obtained is every other
Embodiment, belongs to the scope of the present invention.
The core of the present invention is to provide a kind of data placement method of distributed memory system.
In order that those skilled in the art more fully understand the present invention program, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.
Fig. 1 is a kind of flow chart of the data placement method of distributed memory system provided in an embodiment of the present invention.Such as Fig. 1
Shown, the flow chart of the data placement method of distributed memory system includes：
S10：First node sends the copy of data block and copy count value to section when receiving the operation for writing data block
Any one intermediate node in point list.
S11：Intermediate node is after copy is received, stored copies, and judges whether current copy count value is 0, if
It is not 0, then current copy count value is subtracted 1, copy and current copy count value are sent remaining into present node list
Any one intermediate node, if 0, then terminates.
It should be noted that the first node in the present embodiment refers to initiate the node where the client of write request, i.e., it is straight
Receive the node of data block.In distributed memory system, include multiple nodes, except remaining node of first node is referred to as
Intermediate node.It is understood that due to including Centroid in distributed memory system, therefore, Centroid may be
First node may also be intermediate node.First node is different from the method for work of intermediate node, and the working method of each intermediate node
All same.It is used as preferred, total quantity of the copy count value less than node listing interior joint.Here the sum in node listing
Amount refers to initial value.
In Fig. 1, although only show two steps, but not representing each step can only perform once, for example, for same
For one data block, step S10 only needs to perform once, and step S11 then needs to perform repeatedly, and step S11 walks for repetition
Suddenly.
Due to being dynamic change in distributed memory system, therefore, node listing is also real-time change, in first node
When receiving data block, node listing may have 10 nodes, and in the transmitting procedure of copy, possible node listing only has 5
Individual node, even if or have 10 nodes, but node therein with 10 initial node differences.In specific implementation,
The information of other nodes in current distributed memory system is recorded on each node, i.e., each node has node row
Table, and the node of turnover system is added, deleted in real time by periodic refreshing.
In step s 11, if an intermediate node is judged to need to send copy to next intermediate node, need
A remaining intermediate node is selected by present node list, remaining intermediate node here refers to not receive copy
Node.If for example, there is ten nodes in a node listing, first node is first node, and first node is by copy
Sent with copy count value to second node, when second node retransmits copy and copy count value, first node
It is not just remaining node with second node, ten nodes of the 3rd node-the are only the node of remaining node, i.e., second
Copy and copy count value can only be sent to any one node in ten nodes of the 3rd node-the.The mesh so done
Be able to avoid copy repeat be sent to a node.
In specific implementation, as preferred embodiment, node listing is with including the IP of first node and each intermediate node
Location.First node or intermediate node is so allowd directly to pass through when sending copy and copy count value to other intermediate nodes
IP address on node listing is sent, without inquiring about IP address again.
Fig. 2 is provided in order to more understand in the workflow for the intermediate node mentioned in the embodiment of the present invention, the present embodiment.
Fig. 2 is a kind of flow chart of the data placement method of intermediate node provided in an embodiment of the present invention.
The data placement method for the distributed memory system that the present embodiment is provided, copy is realized by the forwarding between node
Storage, and then realize the asynchronous write operation in network, due to this method and the expense of ghost need not be concentrated on one
On individual node, but share on each node, therefore the performance of system will not be reduced with the increase of redundancy, to a certain degree
On ensure that the reliability of system.Further, since being the node of any selection, therefore transmission rule, energy need not be pre-set
Enough efficiently solve the Placement Problems of the copy under dynamic network environment.
On the basis of Fig. 2, also include in the case where current copy count value is not 0：
S20：Intermediate node judges whether also there is the intermediate node for not receiving copy in present node list；If deposited
Into step S112, if it does not, into step S21.
S21：Judge whether present node list has new intermediate node to add；If new intermediate node is added, then
Into step S22, otherwise, return to step S21.
S22：Current copy count value is subtracted 1, copy and current copy count value are sent to new intermediate node；
S20：If added without new intermediate node, return judges whether present node list has new intermediate node
The step of addition.
By increasing step S20, current copy count value can be avoided to be more than the quantity of remaining intermediate node and caused point
In cloth storage system, the actual quantity of copy is not reaching to copy count value.If for example, copy count value is 10, one
Individual intermediate node is stored after copy, and current copy count value is 3, but the intermediate node in now present node list is
Saved copy, it is now also poor 3 with copy count value 10, illustrate that the redundancy of copy is also not reaching to desired value, system
Reliability it is high not enough.With the operation of system, when there is new intermediate node to add, then the work of intermediate node is continued executing with
Make flow, this place is repeated no more.It is understood that an intermediate node stores copy, then after copy is received
When new copy is sent to next intermediate node again, if the middle node for not receiving copy is not present in present node list
Point, then this intermediate node will preserve this copy, wait new intermediate node to add.Therefore, in the present embodiment, one is played
The effect of individual caching, it is to avoid the loss of copy, and cause the redundancy of system to be not reaching to predetermined requirement.
As preferred embodiment, in addition to：First node record is received in the copy and copy count value that it sends
Intermediate node.
As preferred embodiment, in addition to：Each intermediate node record receives its copy and current copy meter for sending
The intermediate node of numerical value.
As preferred embodiment, in addition to：Centroid record data block and copy are in first node and each middle node
The transmission locus of point.
Above-mentioned three kinds preferred embodiment, can provide a user copy in placement process when user inquires about
In transmission locus.Because Centroid is the important node of distributed memory system, its reliability can than other nodes
It is higher by property, therefore, the security of transmission locus ensure that by the record of Centroid.
As preferred embodiment, in addition to：Copy in the current distributed memory system of Centroid real-time update
Quantity.
Due to distributed memory system in the process of running, the turnover of node is unfixed, with the turnover of node, all
It is possible to influence the quantity of the copy of a data block, for example, a node stores a copy, if the event of this node
Barrier, after logging off, then the quantity of this copy in current system has just lacked one, it is clear that reduce the redundancy of copy
Degree, also reduces the reliability of system.The running situation of system can be understood at any time by the quantity of more latest copy, or
Follow-up remedial measure provides foundation.
As preferred embodiment, in addition to：Centroid is pointed out in the quantity of copy less than threshold value alarm.
It is very important information for some copies, once the consequence that loss is caused is very serious, therefore, is
Avoid in a large amount of loss of copy, the present embodiment, pointed out in copy amount less than threshold value alarm.It is understood that threshold
Depending on the determination of value is needed according to actual conditions, the present embodiment is repeated no more.
The data placement method to distributed memory system provided by the present invention is described in detail above.Specification
In each embodiment described by the way of progressive, what each embodiment was stressed be it is different from other embodiment it
Place, between each embodiment identical similar portion mutually referring to.For device disclosed in embodiment, due to itself and reality
Apply that method disclosed in example is corresponding, so description is fairly simple, related part is referring to method part illustration.It should refer to
Go out, for those skilled in the art, under the premise without departing from the principles of the invention, can also be to the present invention
Some improvement and modification are carried out, these are improved and modification is also fallen into the protection domain of the claims in the present invention.
It should also be noted that, in this manual, such as first and second or the like relational terms be used merely to by
One entity or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or operation
Between there is any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant meaning
Covering including for nonexcludability, so that process, method, article or equipment including a series of key elements not only include that
A little key elements, but also other key elements including being not expressly set out, or also include be this process, method, article or
The intrinsic key element of equipment.In the absence of more restrictions, the key element limited by sentence "including a ...", is not arranged
Except also there is other identical element in the process including the key element, method, article or equipment.
Claims (9)
1. a kind of data placement method of distributed memory system, it is characterised in that including：
The copy of the data block and copy count value are sent to node and arranged when receiving the operation for writing data block by first node
Any one intermediate node in table；
Intermediate node stores the copy after the copy is received, and judges whether current copy count value is 0, if
It is not 0, then current copy count value is subtracted 1, the copy and current copy count value is sent into present node list surplus
Any one remaining intermediate node, if 0, then terminates.
2. the data placement method of distributed memory system according to claim 1, it is characterised in that in current copy meter
Numerical value be 0 in the case of also include：
Intermediate node judges whether also there is the intermediate node for not receiving the copy in present node list；
If it is present subtracting 1 by current copy count value, the copy and current copy count value are sent to present node and arranged
Any one remaining intermediate node in table；
If it does not exist, then judging whether present node list has new intermediate node to add；
If new intermediate node is added, then current copy count value is subtracted 1, the copy and current copy count value are sent out
Deliver to new intermediate node；
If added without new intermediate node, return and described judge whether present node list has new intermediate node addition
The step of.
3. the data placement method of distributed memory system according to claim 1, it is characterised in that the node listing
IP address comprising the first node and each intermediate node.
4. the data placement method of distributed memory system according to claim 1, it is characterised in that the copy is counted
Total quantity of the value less than the node listing interior joint.
5. the data placement method of distributed memory system according to claim 1, it is characterised in that also include：It is described
First node record receives the copy of its transmission and the intermediate node of the copy count value.
6. the data placement method of distributed memory system according to claim 1, it is characterised in that also include：In each
Intermediate node record receives the copy of its transmission and the intermediate node of current copy count value.
7. the data placement method of the distributed memory system according to claim 1 to 6 any one, it is characterised in that
Also include：Centroid records the data block and the copy in the first node and the transmission locus of each intermediate node.
8. the data placement method of distributed memory system according to claim 7, it is characterised in that also include：It is described
The quantity of copy described in the current distributed memory system of Centroid real-time update.
9. the data placement method of distributed memory system according to claim 8, it is characterised in that also include：It is described
Centroid is pointed out in the quantity of the copy less than threshold value alarm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710165599.8A CN106970960A (en) | 2017-03-20 | 2017-03-20 | A kind of data placement method of distributed memory system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710165599.8A CN106970960A (en) | 2017-03-20 | 2017-03-20 | A kind of data placement method of distributed memory system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106970960A true CN106970960A (en) | 2017-07-21 |
Family
ID=59329470
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710165599.8A Pending CN106970960A (en) | 2017-03-20 | 2017-03-20 | A kind of data placement method of distributed memory system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106970960A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101741911A (en) * | 2009-12-18 | 2010-06-16 | 中兴通讯股份有限公司 | Multi-copy collaboration-based write operation method, system and node |
| CN104050270A (en) * | 2014-06-23 | 2014-09-17 | 成都康赛信息技术有限公司 | Distributed storage method based on consistent Hash algorithm |
-
2017
- 2017-03-20 CN CN201710165599.8A patent/CN106970960A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101741911A (en) * | 2009-12-18 | 2010-06-16 | 中兴通讯股份有限公司 | Multi-copy collaboration-based write operation method, system and node |
| CN104050270A (en) * | 2014-06-23 | 2014-09-17 | 成都康赛信息技术有限公司 | Distributed storage method based on consistent Hash algorithm |
Non-Patent Citations (3)
| Title |
|---|
| LIN, MENG-JANG, AND KEITH MARZULLO: "Directional gossip: Gossip in a wide area network", 《DEPENDABLE COMPUTING -EDCC3》 * |
| 刘翔；汪海玲: "分布式存储中的一种数据放置策略", 《计算机与数字工程》 * |
| 陈惟康；杜松: "分布式存储中数据放置策略的研究", 《计算机应用与软件》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106375404B (en) | Data storage control method, date storage method, data capture method and device | |
| CN105224255B (en) | A kind of storage file management method and device | |
| CN103593147B (en) | A kind of method and device of digital independent | |
| CN103064960B (en) | Data base query method and equipment | |
| CN109471744A (en) | The more subchain system architectures of main chain adduction row based on block chain | |
| CN106850710A (en) | A kind of safe data cloud storage system, client terminal, storage server and application process | |
| CN109684086A (en) | A kind of distributed caching automatic loading method and device based on AOP | |
| CN109684273A (en) | A kind of snapshot management method, apparatus, equipment and readable storage medium storing program for executing | |
| CN108011846A (en) | The method and device of management business in network function virtualization architecture | |
| CN107608632A (en) | A kind of communication means, the apparatus and system of distributed storage cluster | |
| CN109918261A (en) | Failure monitoring method, device, equipment and computer readable storage medium | |
| CN108111325A (en) | A kind of resource allocation methods and device | |
| CN109756566A (en) | Date storage method, relevant device and storage medium based on block chain | |
| CN105391601B (en) | A kind of performance test methods and system of Network Management Equipment | |
| CN107967117A (en) | A kind of data storage, reading, method for cleaning, device and cloud storage system | |
| CN104965793B (en) | A kind of cloud storage data node device | |
| CN108063832A (en) | A kind of cloud storage system and its storage method | |
| CN104407816A (en) | Method and system for evaluating data reliability of distribution type storage system | |
| CN104956346B (en) | Control error propagation caused by the failure in the calculate node of distributed computing system | |
| CN103714059B (en) | A kind of method and device of more new data | |
| CN105045542B (en) | A kind of method and device for realizing snapshot management | |
| US20210382872A1 (en) | Blockchain-based Data Storage Method, Related Device and Storage Medium | |
| CN106970960A (en) | A kind of data placement method of distributed memory system | |
| EP1929706A1 (en) | Device management system using log management object and method for generating and controlling logging data therein | |
| JPWO2014199568A1 (en) | Method for controlling data writing to persistent storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170721 |
|
| RJ01 | Rejection of invention patent application after publication |