CN103973497A - Method and device for realizing multi-path concurrent storage based on high-density micro server - Google Patents
Method and device for realizing multi-path concurrent storage based on high-density micro server Download PDFInfo
- Publication number
- CN103973497A CN103973497A CN201410220203.1A CN201410220203A CN103973497A CN 103973497 A CN103973497 A CN 103973497A CN 201410220203 A CN201410220203 A CN 201410220203A CN 103973497 A CN103973497 A CN 103973497A
- Authority
- CN
- China
- Prior art keywords
- data
- microserver
- storage
- management server
- polymerization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 17
- 230000005540 biological transmission Effects 0.000 claims abstract description 15
- 238000012795 verification Methods 0.000 claims abstract description 6
- 238000007726 management method Methods 0.000 claims description 37
- 238000006116 polymerization reaction Methods 0.000 claims description 32
- 238000012546 transfer Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 4
- 230000005465 channeling Effects 0.000 claims description 3
- 230000004931 aggregating effect Effects 0.000 abstract description 2
- 238000004364 calculation method Methods 0.000 abstract description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention relates to the technical field of computer storage, in particular to a method and a device for realizing multi-path concurrent storage based on a high-density micro server. The technical scheme of the invention realizes the great improvement of the storage performance on the micro server with lower performance, and achieves the purpose of doubling the data throughput by aggregating the Ethernet between the micro server and the switch into a parallel transmission bus with high bandwidth, taking all the micro servers as a storage medium and then carrying out data forwarding and verification calculation by the management server.
Description
Technical field
The present invention relates to computer memory technical field, particularly a kind of method and apparatus of realizing multipath concurrence storage based on high density microserver.
Background technology
Along with the development of the information technology such as cloud computing, the Internet, the form of data storage is proposed to new requirement, in the face of different application scene needs dissimilar computer system.For example mass storage system (MSS) be may just need towards high-performance calculation, high bandwidth, high power capacity and high fault-tolerant etc. are characterized in requiring; The microserver device of Internet also arises at the historic moment simultaneously, microserver generally adopts ATOM, ARM or other low sides CPU as processing unit, performance is high but have the feature such as high density, low cost, is more and more subject to extensive use in the cold field of storage that performance is not required.How by cost and performance all lower microserver be applied to other more areas, it is large technological difficulties that real performance significantly promotes.
Summary of the invention
In order to solve the problem of prior art, the invention provides a kind of method and apparatus of realizing multipath concurrence storage based on high density microserver, it is by aggregating into the Ethernet between microserver and switch the parallel transmission bus of a high bandwidth, it is a storage medium that all microservers are used as, then carry out data retransmission by management server and verification is calculated, reach the object that data throughout is doubled.
The technical solution adopted in the present invention is as follows:
A method that realizes multipath concurrence storage based on high density microserver, comprises the following steps:
A, first the network polymerization of many microservers is become to one group of high speed parallel bus, utilize polymerization bus to transmit data simultaneously;
B, management server calculate data and checking treatment, according to the destination address of each network, data are stored in respectively on different servers.
In steps A, the each network channel in polymerization bus has a destination address, and described destination address is the address during for data storage.
In step B, management server provides a kind of storage checking algorithm, and each data that are transmitted all calculates check information position by distance algorithm, and data and check information bit strip shape distributed store are under different passages.
In step B, also comprise a file directory system in management server, file directory system is for catalogue, structure and the storage information of store data.
Method also comprises step C: in the time that certain network channel of polymerization bus the inside breaks down, management server eliminates faulty channel polymerization bus at once, and recalculate generation new polymerization bus, destination address etc., turn by other transmission path and retransmit by other normal data.
Method also comprises step D: inside polymerization bus, have again new passage to add fashionable, management server will be attempted the passage initialization newly adding, then regenerate new polymerization bus, then recalculate transmission and the verification rule of data, finally continue transmission by new passage.
Realize a device for multipath concurrence storage based on high density microserver, formed 1 by following part), some microservers; 2) a, interconnect switch; 3, a management server; 4, a cabinet topcross is changed planes, wherein:
Described microserver is the main body of the concurrent storage system of channeling, and every microserver has CPU, internal memory, hard disk and the network interface of oneself, by the ethernet network of all microservers being aggregated into one group of parallel bus, carries out transfer of data simultaneously;
The switch of described interconnection, for the interconnection of microserver, forms polymerization ethernet network;
Described management server is responsible for configuration and the data retransmission of memory channel and is realized, and is also responsible for foundation and the storage checking algorithm of data directory file;
Described cabinet topcross is changed planes and is responsible for communicating by letter with outside operation system, and management server is connected to cabinet topcross by the interface of 10Gb or higher rate and changes planes.
The descending of interconnect switch interconnects by 1Gb bandwidth port and microserver, and the up interface by 10Gb or higher rate is connected with management server.
Management server is the quantity of capable of dynamic configuration network polymerization microserver and the scale of multipath concurrence storage system first; Secondly by the descending 1Gb memory channel of interconnect switch, by data block concurrent storing on different microservers simultaneously.
The beneficial effect that technical scheme provided by the invention is brought is:
1, utilize the feature that microserver density is high, first the network polymerization of many microservers is become to one group of high speed parallel bus, utilize polymerization bus to transmit data simultaneously.In the time carrying out read or write, these servers are operated simultaneously, thereby can reach the data object that multipath concurrence transmits simultaneously, improve memory property;
2, some the main bodys that microserver is the concurrent storage system of channeling, every microserver has CPU, internal memory, hard disk and the network interface of oneself.In the present invention, by the ethernet network of all microservers being aggregated into one group of parallel bus, carry out transfer of data simultaneously;
3, interconnect switch, for the interconnection of microserver, forms polymerization ethernet network.In order to ensure the non-blocking of data bandwidth, the descending of interconnect switch interconnects by 1Gb bandwidth port and microserver, and the up interface by 10Gb or higher rate is connected with management server;
4, the first quantity of capable of dynamic configuration network polymerization microserver and the scale of multipath concurrence storage system of management server; Secondly by the descending 1Gb memory channel of interconnect switch, by data block concurrent storing on different microservers simultaneously, thereby realize superior performance of all microservers compositions, storage server group with low cost;
Management server is also responsible for foundation and the storage checking algorithm of data directory file.Catalogue file is for information such as the size of recording data files, quantity, types, so that can be to being passed the fast finding of data;
5, storage checking algorithm, for improving the fault-tolerance of system file transmission, is calculated and is produced check information by XOR, and check information is stored with data simultaneously, the integrality of (comprising microserver fault, hard disk failure etc.) data while ensureing link failure.
Technical scheme of the present invention has realized the significantly lifting that realizes memory property on the lower microserver of performance, by the Ethernet between microserver and switch being aggregated into the parallel transmission bus of a high bandwidth, it is a storage medium that all microservers are used as, then carry out data retransmission by management server and verification is calculated, finally reached the object that data throughout is doubled.
Brief description of the drawings
Fig. 1 is system configuration topological diagram of the present invention;
Fig. 2 is data file multipath concurrence storage schematic diagram of the present invention.
Embodiment
For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.
Embodiment mono-
A kind of method and apparatus of realizing the concurrent storage in data multiplex footpath based on high density microserver, specific implementation is the bus that the Ethernet of many microservers is aggregated into a high-speed parallel, management server is according to the destination address of each microserver, data are transmitted by polymerization bus simultaneously, be stored in respectively on different microservers, thereby improve memory bandwidth.
Its system forms as shown in Figure 1, comprising: 1, some microservers; 2, an interconnect switch; 3, a management server; 4, a cabinet topcross is changed planes.
Because microserver is to be used as storage server to use in the present invention, so be referred to as Storage Server(storage server), each Storage Server is an independently hardware system, comprises CPU, internal memory, hard disk (HDD or SSD), network etc.
The 1Gb Ethernet of Storage Server 1 to Storage Server N is all connected to switch, and N depends on the quantity of Storage Server.In order to ensure the coupling of bandwidth, switch is connected to " management server " by 10Gb or 40Gb network more at a high speed.
The Ethernet of Storage Server 1 to N has been aggregated into a high speed parallel bus by management server, and the destination address of distributing each Ethernet interface, address when this destination address is used for data storage.
Management server calculates data to be transferred, generate a lot of data blocks, each data block has its corresponding target address information, and in the time carrying out transfer of data, these data blocks will be stored N Storage Server the inside according to address information.
In order to ensure the reliability of data when the parallel transmission, management server provides a kind of storage checking algorithm, be that each data that are transmitted all calculates check information position by distance algorithm, data and check information bit strip shape distributed store are under different passages.Adopt the benefit of check information storage to be that check information exists respectively in the hard disk under different passages, can not go wrong and occur that data are imperfect because of a certain passage.
In management server, also comprise a file directory system, file directory system is for catalogue, structure and the storage information of store data.For instance, in the time that system is carried out a transfer of data, these data can be deposited into different storage server the insides.For the ease of these data are managed, initial data and distributed data block message are all stored in " data directory system ".
Cabinet topcross is changed planes, and responsible " multipath concurrence storage device " communicates by letter with outside operation system, and management server is connected to cabinet topcross by the interface of 10Gb or higher rate and changes planes.In the time having external business to come in, planned as a whole to distribute by management server, then stored into above different Storage Server by polymerization bus.
Multipath concurrence storage means and device based on high density microserver have load balancing transfer function, and the ethernet channel of the N in polymerization bus transmits simultaneously, and data uniform distribution on each passage, and the data volume that ensures each passage is consistent.
In the time that certain network channel of polymerization bus the inside breaks down, management server eliminates faulty channel polymerization bus at once, and recalculate generation new polymerization bus, destination address etc., turned by other normal data and retransmitted by other transmission path, soon the transmission of quick-recovery data.
Inside polymerization bus, have again new passage to add fashionable, management server, by attempting the passage initialization newly adding, then regenerates new polymerization bus, then recalculates transmission and the verification rule of data, finally continues transmission by new passage.
Arrange by management server, can neatly many Storage Server be aggregated into one group of parallel bus or several groups of parallel buss, thereby improve the flexibility of system.
The foregoing is only preferred embodiment of the present invention, in order to limit the present invention, within the spirit and principles in the present invention not all, any amendment of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.
Claims (9)
1. a method that realizes multipath concurrence storage based on high density microserver, comprises the following steps:
A, first the network polymerization of many microservers is become to one group of high speed parallel bus, utilize polymerization bus to transmit data simultaneously;
B, management server calculate data and checking treatment, according to the destination address of each network, data are stored in respectively on different servers.
2. a kind of method that realizes multipath concurrence storage based on high density microserver according to claim 1, it is characterized in that, in described steps A, the each network channel in polymerization bus has a destination address, and described destination address is the address during for data storage.
3. a kind of method that realizes multipath concurrence storage based on high density microserver according to claim 1, it is characterized in that, in described step B, management server provides a kind of storage checking algorithm, be that each data that are transmitted all calculates check information position by distance algorithm, data and check information bit strip shape distributed store are under different passages.
4. a kind of method that realizes multipath concurrence storage based on high density microserver according to claim 3, it is characterized in that, in described step B, also comprise a file directory system in management server, file directory system is for catalogue, structure and the storage information of store data.
5. a kind of method that realizes multipath concurrence storage based on high density microserver according to claim 1, it is characterized in that, described method also comprises step C: in the time that certain network channel of polymerization bus the inside breaks down, management server eliminates faulty channel polymerization bus at once, and recalculate generation new polymerization bus, destination address etc., turn by other transmission path and retransmit by other normal data.
6. a kind of method that realizes multipath concurrence storage based on high density microserver according to claim 1, it is characterized in that, described method also comprises step D: inside polymerization bus, have again new passage to add fashionable, management server will be attempted the passage initialization newly adding, then regenerate new polymerization bus, then recalculate transmission and the verification rule of data, finally continue transmission by new passage.
7. a device of realizing multipath concurrence storage based on high density microserver, is made up of 1 following part), some microservers; 2) a, interconnect switch; 3, a management server; 4, a cabinet topcross is changed planes, wherein:
Described microserver is the main body of the concurrent storage system of channeling, and every microserver has CPU, internal memory, hard disk and the network interface of oneself, by the ethernet network of all microservers being aggregated into one group of parallel bus, carries out transfer of data simultaneously;
The switch of described interconnection, for the interconnection of microserver, forms polymerization ethernet network;
Described management server is responsible for configuration and the data retransmission of memory channel and is realized, and is also responsible for foundation and the storage checking algorithm of data directory file;
Described cabinet topcross is changed planes and is responsible for communicating by letter with outside operation system, and management server is connected to cabinet topcross by the interface of 10Gb or higher rate and changes planes.
8. a kind of device of realizing multipath concurrence storage based on high density microserver according to claim 7, it is characterized in that, the descending of interconnect switch interconnects by 1Gb bandwidth port and microserver, and the up interface by 10Gb or higher rate is connected with management server.
9. a kind of device of realizing multipath concurrence storage based on high density microserver according to claim 7, is characterized in that, described management server is the quantity of capable of dynamic configuration network polymerization microserver and the scale of multipath concurrence storage system first; Secondly by the descending 1Gb memory channel of interconnect switch, by data block concurrent storing on different microservers simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410220203.1A CN103973497A (en) | 2014-05-23 | 2014-05-23 | Method and device for realizing multi-path concurrent storage based on high-density micro server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410220203.1A CN103973497A (en) | 2014-05-23 | 2014-05-23 | Method and device for realizing multi-path concurrent storage based on high-density micro server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103973497A true CN103973497A (en) | 2014-08-06 |
Family
ID=51242564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410220203.1A Pending CN103973497A (en) | 2014-05-23 | 2014-05-23 | Method and device for realizing multi-path concurrent storage based on high-density micro server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103973497A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106533804A (en) * | 2016-12-22 | 2017-03-22 | 成都西加云杉科技有限公司 | Network operation support system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1567237A (en) * | 2003-06-09 | 2005-01-19 | 浪潮电子信息产业股份有限公司 | Method for constructing high-available distributed memory system |
| CN101471948A (en) * | 2007-12-29 | 2009-07-01 | 英业达股份有限公司 | Storage LAN server capable of in-parallel processing and rapidly fetching, and access method thereof |
| CN102968279A (en) * | 2012-11-13 | 2013-03-13 | 浪潮电子信息产业股份有限公司 | Thin provisioning method for storage system |
| CN103034295A (en) * | 2012-12-26 | 2013-04-10 | 无锡江南计算技术研究所 | Input/output capability-enhanced reconfigurable micro-server |
| CN103076849A (en) * | 2012-12-26 | 2013-05-01 | 无锡江南计算技术研究所 | Reconfigurable micro server system |
-
2014
- 2014-05-23 CN CN201410220203.1A patent/CN103973497A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1567237A (en) * | 2003-06-09 | 2005-01-19 | 浪潮电子信息产业股份有限公司 | Method for constructing high-available distributed memory system |
| CN101471948A (en) * | 2007-12-29 | 2009-07-01 | 英业达股份有限公司 | Storage LAN server capable of in-parallel processing and rapidly fetching, and access method thereof |
| CN102968279A (en) * | 2012-11-13 | 2013-03-13 | 浪潮电子信息产业股份有限公司 | Thin provisioning method for storage system |
| CN103034295A (en) * | 2012-12-26 | 2013-04-10 | 无锡江南计算技术研究所 | Input/output capability-enhanced reconfigurable micro-server |
| CN103076849A (en) * | 2012-12-26 | 2013-05-01 | 无锡江南计算技术研究所 | Reconfigurable micro server system |
Non-Patent Citations (1)
| Title |
|---|
| 张晓波: "基于高性能集群计算的并行文件系统关键技术研究", 《CNKI优秀硕士论文全文数据库》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106533804A (en) * | 2016-12-22 | 2017-03-22 | 成都西加云杉科技有限公司 | Network operation support system |
| US10944628B2 (en) | 2016-12-22 | 2021-03-09 | Chengdu SK Spruce Technologies Co., Ltd. | Network operation support system and network device management method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108228082B (en) | Storage system and method for storage control | |
| CN103763383B (en) | Integrated cloud storage system and its storage method | |
| CN102523234B (en) | A kind of application server cluster implementation method and system | |
| US20150149819A1 (en) | Parity chunk operating method and data server apparatus for supporting the same in distributed raid system | |
| WO2014138483A1 (en) | Application delivery controller and global server load balancer | |
| CN102833237B (en) | InfiniBand protocol conversion method and system based on bridging | |
| US11394777B2 (en) | Reliable data storage for decentralized computer systems | |
| US11005663B2 (en) | Secure audit scheme in a distributed data storage system | |
| CN101540685B (en) | PCIe shared storage blade for blade server | |
| WO2016107512A1 (en) | Method and apparatus for establishing multicast group in fat-tree network, and fat-tree network | |
| US20110258290A1 (en) | Bandwidth-Proportioned Datacenters | |
| CN102868727B (en) | Method for realizing high availability of logical volume | |
| CN108064374A (en) | A kind of data access method, device and system | |
| CN105867843A (en) | Data transmission method and device | |
| CN104639469A (en) | Computing and storing cluster system based on PCIE (Peripheral Component Interconnect Express) interconnection | |
| CN105187502A (en) | Method and system based on distributed elastic block storage | |
| CN108206839A (en) | One kind is based on majority's date storage method, apparatus and system | |
| US20210288830A1 (en) | Early acknowledgment for write operations | |
| US20050188123A1 (en) | System and method for insertion of markers into a data stream | |
| CN103209219A (en) | Distributed cluster file system | |
| US11275698B2 (en) | Termination of non-volatile memory networking messages at the drive level | |
| US11720413B2 (en) | Systems and methods for virtualizing fabric-attached storage devices | |
| CN103973497A (en) | Method and device for realizing multi-path concurrent storage based on high-density micro server | |
| CN112104729A (en) | Storage system and caching method thereof | |
| US20230044165A1 (en) | Systems, methods, and apparatus for remote data transfers to memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140806 |