CN106951445A - A kind of distributed file system and its memory node loading method - Google Patents
A kind of distributed file system and its memory node loading method Download PDFInfo
- Publication number
- CN106951445A CN106951445A CN201710087210.2A CN201710087210A CN106951445A CN 106951445 A CN106951445 A CN 106951445A CN 201710087210 A CN201710087210 A CN 201710087210A CN 106951445 A CN106951445 A CN 106951445A
- Authority
- CN
- China
- Prior art keywords
- osd
- file system
- group
- order
- distributed file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of distributed file system, including, memory cell cluster：Including multiple OSD, the OSD carries at least one and puts in order group；Monitors M ON：The multiple OSD is detected and safeguarded；Member's notification module：When there is new OSD to add, given notice to monitors M ON.The invention also discloses a kind of distributed file system memory node loading method, when new OSD adds distributed file system, member's notification module is given notice to monitors M ON, the availability and efficiency of raising system, the OSD newly added can calculate oneself carrying and all put in order group, and associate with other OSD simultaneously and group carry out a corresponding operating to putting in order, after the completion of monitors M ON changes OSD state and file system service chart, deployment program is more simplified.
Description
Technical field
The present invention relates to field of distributed storage, more particularly to a kind of distributed file system and its memory node side of reaching the standard grade
Method.
Background technology
OSD：Object Storage Device, object storage device.
MON：Monitor, monitor.
With the development of information technology, global metadata is in explosive growth, and centralised storage server can not meet system
Reliability, scalability etc. require, it is easy to as performance bottleneck, and distributed system solves these problems well.
Distribution refers to that the important control information of file system is distributed across on multiple computer nodes, so as to improve file system
Overall reliability, when wherein any one or more nodes break down, file system still can be with normal work.
Distributed file system is typically made up of two parts, a kind of to be responsible for completing data storage and maintenance function
OSD, it is another, it is the MON that several are responsible for completing system mode detection and safeguarding（Monitor）.Between OSD and MON mutually
Transmission node status information, draws the overall operational status of distributed file system, and form a global system state jointly
Record data structure.
The description of OSD states is divided into two dimensions：Up or down（Show OSD whether normal work）, in or out
（Show OSD whether at least one puts in order group）, therefore, for any one OSD, have four kinds of possible states：
（1）Up and in：Illustrate that the OSD is normally run, but do not carry it is any put in order group, wherein also without data；
（2）Up and out：Illustrate that the OSD is normal, but do not carry it is any put in order group, wherein also without data；
（3）Down and in：Illustrate that the OSD occurs abnormal, but still carry at least one to put in order group, wherein still store number
According to.
（4）Down and out：Illustrate that the OSD thoroughly breaks down, and no longer carried and any put in order group.
Distributed file system program in deployment is relatively complicated, as one during distributed file system use urgently
To be solved the problem of.
The content of the invention
In order to solve the above-mentioned technical problem, it is an object of the invention to provide a kind of relatively simple distribution text of deployment program
Part system memory node loading method.
In order to solve the above-mentioned technical problem, it is an object of the invention to provide a kind of distribution for making deployment program relatively simple
File system.
The technical solution adopted in the present invention is：A kind of distributed file system memory node loading method, including step
S1, when new OSD adds distributed file system, member's notification module is given notice to monitors M ON, when monitors M ON is received
To after notice, new OSD is added to memory cell cluster；
The OSD states newly added are set to up and out by step S2, monitor, and monitor more new file system service chart will more
File system service chart after new issues the OSD newly added；
Step S3, the OSD newly added calculate itself all of carrying and put in order group, it is described put in order group and include main group and the copy put in order return
Put group；
Step S4, the OSD contacts newly added and same other OSD for putting in order group of oneself carrying, carry out data duplication and/or failure
Repair；
The OSD newly added status indication is up and in by step S5, monitor, and the content of file system service chart is corresponding more
Newly.
As the further improvement of such scheme, the step S4 includes sub-step：
Step S41, the OSD contacts newly added and same other OSD for putting in order group of oneself carrying, judging carrying, certain puts in order group A's
Whether OSD forms are to put in order whether group degrades less than normal value, if it is not, the OSD then newly added will replace an existing bag
Include this and put in order group A OSD, and undertake the data for the OSD being replaced, the OSD being replaced exit this put in order group A but state still
It is so up and in, puts in order and select main OSD in group A again, if so, then enters step S42；
Step S42, is less than normal value, other OSD return this when carrying the OSD numbers for putting in order group A in distributed file system
All objects and metadata replication in group A are put to the OSD newly added.
A kind of distributed file system, for implementing a kind of above-mentioned distributed file system memory node loading method,
System includes：Memory cell cluster：Including multiple OSD, the OSD carries at least one and puts in order group；Monitors M ON：To described
Multiple OSD are detected and safeguarded；Member's notification module：When there is new OSD to add, given notice to monitors M ON.
As the further improvement of such scheme, group is put in order described in the OSD carryings, the OSD, which has to calculate, oneself to be held
Load puts in order group and the function of being interacted with other OSD.
As the further improvement of such scheme, the OSD also has the function of rebalancing with fault restoration.
Root as such scheme further improvement, it is described put in order group include object and metadata.
The beneficial effects of the invention are as follows：
The present invention discloses a kind of distributed file system memory node loading method, when new OSD adds distributed file system
When, member's notification module is given notice to monitors M ON, improves the availability and efficiency of system, and the OSD newly added can be calculated certainly
Carrying is all puts in order group for oneself, and associate with other OSD simultaneously and group carry out a corresponding operating to putting in order, after the completion of monitors M ON changes
OSD state and file system service chart, deployment program is more simplified.
The present invention discloses a kind of distributed file system, and when new OSD adds distributed file system, member notifies mould
Block is given notice to monitors M ON, improves the availability and efficiency of system, and the OSD newly added can calculate oneself and carry all return
Put group, and associate with other OSD simultaneously and group carry out a corresponding operating to putting in order, after the completion of monitors M ON changes OSD state and text
Part system operation figure, deployment program is more simplified.
Brief description of the drawings
The embodiment to the present invention is described further below in conjunction with the accompanying drawings：
Fig. 1 is a kind of distributed file system memory node loading method flow chart of the invention.
Embodiment
It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the application can phase
Mutually combination.
A kind of distributed file system, including：Memory cell cluster：Including multiple OSD, the OSD carries at least one
Put in order group；Monitors M ON：The multiple OSD is detected and safeguarded；Member's notification module：When there is new OSD to add,
Given notice to monitors M ON.There is OSD calculating oneself carrying to put in order group and the function of being interacted with other OSD；OSD also has
There is the function of rebalancing with fault restoration.Putting in order group includes object and metadata.
A kind of distributed file system, also including file system service chart, the file system service chart includes：
Monitor figure：Name, address and port comprising distributed file system, also including current time knot, monitor figure wound
Build time, nearest modification time；
OSD schemes：Including OSD figures creation time, nearest modification time, pool list is stored, copy amount puts in order a group quantity, OSD row
Table and its state；
Put in order a group figure：Including putting in order a group version, its timestamp, the newest OSD figure times tie, take rate, respectively put in order a group details bag
Include a group ID, the state etc. of putting in order, the data service condition statistics of each storage pool.
The process of distributed storage data is：The data for needing to store are divided into data block, then map the data into each
Put in order in group, then be mapped to group is put in order in the physical memory location OSD of data.
The purposes for putting in order group is that the storage to the data block in distributed file system carries out tissue and position mapping, one
Responsible several data blocks of tissue of group are put in order, but a data block can only be mapped to one and put in order in group, that is, put in order group and number
It is " one-to-many " mapping relations according to block, meanwhile, put in order group meeting for one and be mapped on multiple OSD, and can be carried on each OSD
Substantial amounts of to put in order group, that is, it is " multi-to-multi " mapping relations to put in order between group and OSD.
The present invention discloses a kind of distributed file system, and when new OSD adds distributed file system, member notifies mould
Block is given notice to monitors M ON, improves the availability and efficiency of system, and the OSD newly added can calculate oneself and carry all return
Put group, and associate with other OSD simultaneously and group carry out a corresponding operating to putting in order, after the completion of monitors M ON changes OSD state and text
Part system operation figure, deployment program is more simplified.
Fig. 1 is a kind of distributed file system memory node loading method flow chart of the invention, a kind of distributed field system
Unite memory node loading method, including step S1, when new OSD adds distributed file system, and member's notification module is to monitoring
Device MON is given notice, and after monitors M ON is notified, new OSD is added into memory cell cluster OSDMAP；Step S2,
The OSD states newly added are set to up and out, monitor more new file system service chart, by the file after renewal by monitor
System operation figure issues the OSD newly added；Step S3, the OSD newly added calculate itself all of carrying and put in order group, described to put in order
Group includes main group and the copy put in order and puts in order group；Step S4, newly add OSD contact with oneself carrying it is same put in order group other
OSD, carries out data duplication and/or fault restoration, and step S4 includes sub-step：Step S41, the OSD contacts newly added are held with oneself
Same other OSD for putting in order group are carried, judge to carry whether certain OSD form for putting in order group A puts in order whether group drops less than normal value
Level, includes this and puts in order group A OSD, and undertake what is be replaced if it is not, the OSD then newly added will replace existing one
OSD data, the OSD being replaced exits that this puts in order group A but state remains as up and in, puts in order and selects main OSD in group A again,
If so, then entering step S42；Step S42, is less than normally when carrying the OSD numbers for putting in order group A in distributed file system
This is put in order all objects and metadata replication in group A to the OSD newly added by value, other OSD；Step S5, monitor will be new
The OSD of addition status indication is up and in, and the content of file system service chart accordingly updates.
The present invention discloses a kind of distributed file system memory node loading method, when new OSD adds distributed document
During system, member's notification module is given notice to monitors M ON, improves the availability and efficiency of system, the OSD accounting newly added
Calculate oneself carrying be all and puts in order group, and associate OSD to putting in order a group progress corresponding operating with other simultaneously, after the completion of monitors M ON
OSD state and file system service chart is changed, deployment program is more simplified.
Above is the preferable implementation to the present invention is illustrated, but the invention is not limited to the implementation
Example, those of ordinary skill in the art can also make a variety of equivalent variations or replace on the premise of without prejudice to spirit of the invention
Change, these equivalent deformations or replacement are all contained in the application claim limited range.
Claims (6)
1. a kind of distributed file system memory node loading method, it is characterised in that it includes step S1, new OSD additions
During distributed file system, member's notification module is given notice to monitors M ON, will be new after monitors M ON is notified
OSD is added to memory cell cluster；
The OSD states newly added are set to up and out by step S2, monitor, and monitor more new file system service chart will more
File system service chart after new issues the OSD newly added；
Step S3, the OSD newly added calculate itself all of carrying and put in order group, it is described put in order group and include main group and the copy put in order return
Put group；
Step S4, the OSD contacts newly added and same other OSD for putting in order group of oneself carrying, carry out data duplication and/or failure
Repair；
The OSD newly added status indication is up and in by step S5, monitor, and the content of file system service chart is corresponding more
Newly.
2. a kind of distributed file system memory node loading method according to claim 1, it is characterised in that the step
Rapid S4 includes sub-step：
Step S41, the OSD contacts newly added and same other OSD for putting in order group of oneself carrying, judging carrying, certain puts in order group A's
Whether OSD forms are to put in order whether group degrades less than normal value, if it is not, the OSD then newly added will replace an existing bag
Include this and put in order group A OSD, and undertake the data for the OSD being replaced, the OSD being replaced exit this put in order group A but state still
It is so up and in, puts in order and select main OSD in group A again, if so, then enters step S42；
Step S42, is less than normal value, other OSD return this when carrying the OSD numbers for putting in order group A in distributed file system
All objects and metadata replication in group A are put to the OSD newly added.
3. a kind of distributed file system, it is characterised in that it is used to implement one kind as described in any one of claim 1 to 2
Distributed file system memory node loading method, system includes：
File system service chart：Including monitor figure, OSD figures, put in order a group figure；
Memory cell cluster：Including multiple OSD, the OSD carries at least one and puts in order group；
Monitors M ON：The multiple OSD is detected and safeguarded；
Member's notification module：When there is new OSD to add, given notice to monitors M ON.
4. a kind of distributed file system according to claim 1, it is characterised in that the OSD, which has to calculate, oneself to be held
Load puts in order group and the function of being interacted with other OSD.
5. a kind of distributed file system according to claim 1, it is characterised in that the OSD, which also has, to be rebalanced
With the function of fault restoration.
6. a kind of distributed file system according to claim 1, it is characterised in that the group of putting in order includes object and member
Data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710087210.2A CN106951445A (en) | 2017-02-17 | 2017-02-17 | A kind of distributed file system and its memory node loading method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710087210.2A CN106951445A (en) | 2017-02-17 | 2017-02-17 | A kind of distributed file system and its memory node loading method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106951445A true CN106951445A (en) | 2017-07-14 |
Family
ID=59467031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710087210.2A Pending CN106951445A (en) | 2017-02-17 | 2017-02-17 | A kind of distributed file system and its memory node loading method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106951445A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107817951A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of method and device for realizing the fusion of Ceph clusters |
| CN109597689A (en) * | 2018-12-10 | 2019-04-09 | 浪潮(北京)电子信息产业有限公司 | A kind of distributed file system Memory Optimize Method, device, equipment and medium |
| CN114237520A (en) * | 2022-02-28 | 2022-03-25 | 广东睿江云计算股份有限公司 | Ceph cluster data balancing method and system |
| CN117519992A (en) * | 2024-01-04 | 2024-02-06 | 中电云计算技术有限公司 | Lookup table type load balancing method and device based on CRUSH algorithm |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090019443A1 (en) * | 2007-07-12 | 2009-01-15 | Jakob Holger | Method and system for function-specific time-configurable replication of data manipulating functions |
| CN104994168A (en) * | 2015-07-14 | 2015-10-21 | 苏州科达科技股份有限公司 | distributed storage method and distributed storage system |
-
2017
- 2017-02-17 CN CN201710087210.2A patent/CN106951445A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090019443A1 (en) * | 2007-07-12 | 2009-01-15 | Jakob Holger | Method and system for function-specific time-configurable replication of data manipulating functions |
| CN104994168A (en) * | 2015-07-14 | 2015-10-21 | 苏州科达科技股份有限公司 | distributed storage method and distributed storage system |
Non-Patent Citations (1)
| Title |
|---|
| 瞧见风: ""Ceph浅析"系列之四——Ceph的工作原理及流程", 《HTTPS://BLOG.CSDN.NET/SCALEQIAO/ARTICLE/DETAILS/46415217》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107817951A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of method and device for realizing the fusion of Ceph clusters |
| CN109597689A (en) * | 2018-12-10 | 2019-04-09 | 浪潮(北京)电子信息产业有限公司 | A kind of distributed file system Memory Optimize Method, device, equipment and medium |
| CN109597689B (en) * | 2018-12-10 | 2022-06-10 | 浪潮(北京)电子信息产业有限公司 | Distributed file system memory optimization method, device, equipment and medium |
| CN114237520A (en) * | 2022-02-28 | 2022-03-25 | 广东睿江云计算股份有限公司 | Ceph cluster data balancing method and system |
| CN114237520B (en) * | 2022-02-28 | 2022-07-08 | 广东睿江云计算股份有限公司 | Ceph cluster data balancing method and system |
| CN117519992A (en) * | 2024-01-04 | 2024-02-06 | 中电云计算技术有限公司 | Lookup table type load balancing method and device based on CRUSH algorithm |
| CN117519992B (en) * | 2024-01-04 | 2024-04-16 | 中电云计算技术有限公司 | Lookup table type load balancing method and device based on CRUSH algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106951445A (en) | A kind of distributed file system and its memory node loading method | |
| CN105187249A (en) | Fault recovery method and device | |
| CN111104283B (en) | Fault detection method, device, equipment and medium of distributed storage system | |
| CN105243004A (en) | Failure resource detection method and apparatus | |
| CN106802854A (en) | A kind of failure monitoring system of multi controller systems | |
| CN106294073A (en) | Service calling method and device | |
| CN106339297B (en) | Method and system for real-time alarming of storage system fault | |
| CN104462612A (en) | Method and device for monitoring database information | |
| CN103535014A (en) | A network storage system, data processing method and the client | |
| CN111049670A (en) | Method and device for fusing isolation of micro-service | |
| CN106911519A (en) | A kind of data acquisition monitoring method and device | |
| CN114154035A (en) | Data processing system for dynamic loop monitoring | |
| JP6002856B2 (en) | Monitoring system and monitoring method | |
| CN104679623A (en) | Server hard disk maintaining method, system and server monitoring equipment | |
| CN105183627A (en) | Server performance prediction method and system | |
| CN108241565A (en) | A kind of system and method for being used to implement application system automation O＆M | |
| US7620855B2 (en) | Self-defining counters | |
| CN107861829A (en) | A kind of method, system, device and the storage medium of disk failure detection | |
| CN109510730A (en) | Distributed system and its monitoring method, device, electronic equipment and storage medium | |
| CN107122254A (en) | A kind of computer repairs control method and system, restorative procedure and system | |
| CN112256202B (en) | Distributed storage system and method for deleting volumes in distributed storage system | |
| CN107886992A (en) | A kind of RAID method for detecting health status, system and relevant apparatus | |
| CN106292911A (en) | A kind of fusion architecture server | |
| CN112306815B (en) | Method, device, equipment and medium for monitoring IO information between OSD side and master slave in Ceph | |
| CN115150253B (en) | Fault root cause determining method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190902 Address after: 100089 Floor 1-4, No. 2 Building, No. 9 Courtyard, Dijin Road, Haidian District, Beijing Applicant after: Beijing Toyou Feiji Electronics Co., Ltd. Address before: 518000 Room 1402, Feiyada Science and Technology Building, Nanshan District, Shenzhen City, Guangdong Province Applicant before: Shenzhen City Rui Bo deposit information technology Co. Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170714 |
|
| RJ01 | Rejection of invention patent application after publication |