CN104102742A - High-performance mass storage system and high-performance mass storage method - Google Patents
High-performance mass storage system and high-performance mass storage method Download PDFInfo
- Publication number
- CN104102742A CN104102742A CN201410371871.4A CN201410371871A CN104102742A CN 104102742 A CN104102742 A CN 104102742A CN 201410371871 A CN201410371871 A CN 201410371871A CN 104102742 A CN104102742 A CN 104102742A
- Authority
- CN
- China
- Prior art keywords
- network
- data
- net
- file server
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a high-performance mass storage system and a high-performance mass storage method, and belongs to the technical field of storage. The mass storage system comprises a file server and a network disk array group, wherein the network disk array group comprises a plurality of network disk arrays, all network disk arrays are mounted onto the file server through a peripheral channel in a DAS mode, and then, all network disk arrays and the file server are connected onto a switching network through network channels; a network user is connected onto the switching network, and realizes interaction with the file server and the network disk arrays through the switching network; the file server is in charge of monitoring all operations in the mass storage system and provides data description for the network user; the network disk arrays are in charge of the data access of the network user; and the peripheral channel is an SCSI bus or an FC bus or a network passage. The high-performance mass storage system and the high-performance mass storage method have the beneficial effects that the advantages of a centralized storage system and a distributed storage system are simultaneously achieved, and the file centralized management and the data distributed storage are realized.
Description
Technical field
The present invention relates to a kind of technical field of memory, specifically a kind of high Performance Mass Storage System and storage means.
Background technology
The develop rapidly of modern information technologies, has caused the explosive increase of user to storage products and stores service demand.The performance of computer memory system can not meet the demand of many practical applications far away at present, thereby how to set up the focus that high performance storage system becomes people's concern.
And the network request of traditional mass storage system (MSS) adopts the roundabout transmission mode centered by server, be that all-network request is by server unified management, causing data stream to come and go between server and memory device transmits and forwards, in server there is potential bottleneck in CPU, internal memory and I/O bus, affected greatly the average response speed of system bandwidth and server.And, cannot realize and share storage, the extensibility of system is also very restricted.
The storage system that employing has novel I/O channel system structure can greatly improve performance, but relies on simply the hardware performance of expanding system eventually to have the upper limit.Therefore,, though heritage storage system architecture can provide large memory capacity, be not suitable for constructing mass storage system (MSS).Constructing a high performance mass storage system (MSS) not only needs great memory capacity and high I/O bandwidth, also must have enhanced scalability, high availability, manageability and low price.Thereby employing distributed memory system structure is to realize the selection of magnanimity information storage.Data Segmentation (such as piece) is left in respectively on different memory devices, utilize the performance that the parallel work-flow of different storage device is improved to system.But the storage administration of distributed system exists very large difficulty, statistics shows, under distributional environment, storage administration approximately will consume 55% of carrying cost total value.Centralized storage can reduce administration overhead and improve management quality.Visible, if adopt centralized storage administration, particularly for the management of storage resources a large amount of in mass storage system (MSS), can greatly reduce the expense of storage administration.
Therefore how to construct a kind of high performance mass storage system (MSS), the architecture of its centralized file management and distributed data storage is combined, can speeding up data transmission speed, reduce storage administration expense, realize separating data flow from command flow, dilatation simultaneously and synchronize the performance that has greatly improved whole storage system with speedup and become an important and urgent problem.
Summary of the invention
Technical assignment of the present invention is to provide a kind of advantage that simultaneously possesses centralised storage system and distributed memory system, realizes a kind of high Performance Mass Storage System and the storage means of centralized file management, distributed data storage.
Technical assignment of the present invention is realized in the following manner,
A kind of high Performance Mass Storage System, mass storage system (MSS) comprises file server and net-raid group, net-raid group comprises many net-raids, all net-raids are articulated on file server by peripheral channel by DAS mode, and then all net-raids and file server are connected on exchange network by network channel again; The network user is connected on exchange network, and the network user is mutual by exchange network and file server and net-raid; All operations in this mass storage system (MSS) of file server charge of overseeing, and provide data description for the network user; Net-raid is responsible for the access of the network user's data; Peripheral channel is SCSI bus or FC bus or network channel.
Software configuration in mass storage system (MSS) is divided into three levels: passage key-course, order key-course and network communication layer;
(1), passage key-course: the order between responsible file server and net-raid group and the transmission of control information; The channel controller and the channel controller that resides in net-raid group that are divided into presence document server; Channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in net-raid group, is responsible for receiving order and the message from file server;
(2), order key-course: for resolving the network user's I/O request, obtain the network address of data place net-raid, realize request service; Be divided into command analysis device in file server and the command executer in net-raid group; The command analysis device that resides in file server authenticates and processes for network user's request; Reside in command executer in net-raid group for network user's request is decomposed, merged or dispatch, carries out;
(3), network communication layer: for the treatment of the order of memory node and the Internet Transmission of data of a plurality of network storage arrays; Be divided into the network communication layer of file server, the network user's network communication layer, net-raid group's network communication layer.
Passage key-course: the channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in file server is only carried out the I/O task that wherein relates to control documents system architecture and file attribute; The channel controller of file server directly utilizes the existing SCSI channels drive of file server program; Channel controller in net-raid group, be responsible for to receive all I/O orders and message from file server principal and subordinate peripheral channel, by execution result loopback to file server.
Order key-course: the command analysis device that resides in file server comprises that user asks to process, address resolution, access control module, it is according to network user's information of recording, network user's request is authenticated, judge its legitimacy and request permissions, type according to request is processed respectively, file read-write request for the network user, after address resolution, generate the network address information of corresponding net-raid, after address resolution completes, subscriber authorisation is passed through to passage key-course informing network disk array group, by authorizing key, the network address information of net-raid is by network communication layer informing network user, the command executer residing in net-raid group comprises that I/O scheduling, order are decomposed/merged, Buffer administration module, dynamically the I/O order from peripheral channel and network channel is decomposed, merged and dispatches, carry out actual magnetic disc i/o operation, after having operated, via network communication layer and network user's swap file data, or via passage key-course to file server loopback execution result.
Network communication layer: the network communication layer of file server is accepted the network user's services request, sends order key-course to and the analysis result of order key-course is fed back to the network user; The network user's network communication layer is responsible for submitting to and asking and receive the result that file server returns to file server, and the network user's network communication layer is also responsible for setting up being connected with net-raid group's network; Net-raid group's network communication layer, together with the network user's network communication layer, completes the task of directly transmitting data between net-raid group and the network user.
A high-performance mass memory method, is used mass storage system (MSS) to carry out the storage to data, and file organization mode is divided into two kinds:
(1), file organization mode independently: net-raid is as storage unit independently, every net-raid has independent directory node on file server, in each directory node, can set up different file system, store dissimilar file (as audiovisual, text etc.), and process separately access and the Internet Transmission of network user's data; When the network user sends when request to different directory node, a plurality of requests can be on a plurality of net-raids Parallel Service;
(2), distributed file organization mode: the storage space on the memory node of a plurality of net-raids is invented the unity logic view of a storage space, and deblocking leaves on a plurality of memory nodes; During in the face of the request of a plurality of small data quantities of the network user, can be by a plurality of memory nodes complete independently concurrently; While facing the request of a big data quantity of the network user, data are distributed on a plurality of memory nodes, a plurality of memory node parallel work-flows.
A high-performance mass memory method, file organization mode is file organization mode independently, the step of data storage is:
(1), when a network user attempts access data, it sends request by data representation to file server, the access control mechanism that file server starts it judges whether this network user has the authority of these data of storage, and then checks whether there are these data of other Subscriber Lockeds; If the network user can not access data, file server will send refusal or error message to the network user;
(2), if the network user has access right and is not refused by file locking, file server just judges the network address of data by data structure, the network address of data is that the memory node of the data place net-raid position in mass storage system (MSS) is (because Data Position is not only LBA (Logical Block Addressing), also comprise the additional network address or the title of memory node) and memory location, then send it to the network user, the network user reorganizes data structure after receiving the network address of data, then pass through the mutual also access data of network channel net-raid direct and this data place.
Now both can adopt the file I/O based on NFS or CIFS to carry out alternately, and also can adopt the block I/O based on SCSI agreement or FC agreement to carry out alternately.What independently file organization manager embodied is exactly a kind of thought of centralized management, so that reduce administration overhead, improves management quality.
A kind of high-performance mass memory method, file organization mode is distributed file organization mode, distributed data storage in mass storage system (MSS) is on the memory node of a plurality of net-raids, and the network user of all these mass storage system (MSS)s of use uses the network address of same file organization system Image Data; This document organization system realizes this reflection by usage data structure, the part of its data structure using the network address of all memory nodes as addressing data algorithm; Use RAID algorithm in storage system, to process file organization and data redundancy; Adopt partition that catalogue data, file data and data structure are crossed over and left on the memory node of a plurality of net-raids.
A high-performance mass memory method, the network user's I/O request adopts block I/O; Data flow from the network user's block I/O on mass storage system (MSS): the block I/O request from the network user itself has comprised the data structure of request msg on objective network disk array, the network address that comprises data in data structure, when file server receives after the network user's block I/O request by ICP/IP protocol stack, this I/O request is handed to volume manager, then by the device driver of net-raid, access corresponding data.
A kind of high Performance Mass Storage System of the present invention and storage means have the following advantages:
1, by setting up contacting directly between net-raid and the network user, make the separating data flow from command flow of net-raid, exempt the storage forwarding of data stream between file server and net-raid, overcome the file server bottleneck of original system;
If 2 access many net-raids on file server, just a network channel has been expanded in one of every access, a plurality of network channels can parallel transmission to improve the I/O bandwidth of system, realized capacity and speed is synchronously expanded.Fully demonstrated the principle of hyperchannel, parallel processing, it has not only removed the constraint of I/O operation to file server performance to a great extent, also greatly improved the overall performance of storage system, meanwhile, its intensive data way to manage can be saved a large amount of managerial cost;
3, along with the increase of memory node, not only the memory capacity of system has obtained expansion, and contributes to improve the performance of system; Specifically, the request of a plurality of small data quantities, can be by a plurality of memory nodes complete independently concurrently, thereby improves the handling capacity of system; The request of a big data quantity, data are distributed on a plurality of memory nodes, and a plurality of memory node parallel work-flows can improve several biography rates of single request; Use-pattern can meet different environments for use flexibly, both can meet the burst request of a plurality of small data quantities such as issued transaction, also can meet the continuous request of big data quantity;
4, file organization mode is distributed file organization mode, because file data block redundancy leaves on a plurality of memory nodes, still can continue operation when some memory nodes in storage system are made a mistake; Meanwhile, by increasing the quantity of memory node, the degree of parallelism that can increase physical data channel in system architecture improves the performance of system;
5, block I/O has been avoided the switching of user's space and system space with respect to file I/O, avoided the expense of all I/O request and data passing through network file system, Virtual File System and local disk file system, the expense of search index node while simultaneously also having avoided disk file system to convert file I/O to block I/O; Therefore, mass storage system (MSS) adopts piece level I/O, and directly the physical address information by data visits net-raid, the speed while making the speed of data retrieval and access be better than adopting file I/O;
6, the mass storage system (MSS) of realizing possesses the advantage of centralised storage system and distributed memory system simultaneously, not only accelerate data rate, reduced storage administration expense, also realized the target that separating data flow from command flow, dilatation are synchronizeed with speedup simultaneously, thus the performance of the whole storage system greatly improving.
Accompanying drawing explanation
Below in conjunction with accompanying drawing, the present invention is further described.
The structured flowchart that accompanying drawing 1 is a kind of high Performance Mass Storage System;
Accompanying drawing 2 is a kind of file organization structured flowchart of high-performance mass memory method;
The software configuration architecture block diagram that accompanying drawing 3 is a kind of high Performance Mass Storage System;
Accompanying drawing 4 is file I/O and the data stream block diagram of block I/O on objective network disk array.
Embodiment
With reference to Figure of description and specific embodiment, a kind of high Performance Mass Storage System of the present invention and storage means are described in detail below.
Embodiment 1:
As shown in Figure 1, Figure 3, a kind of high Performance Mass Storage System of the present invention, mass storage system (MSS) comprises file server and net-raid group, net-raid group comprises many net-raids, all net-raids are articulated on file server by peripheral channel by DAS mode, and then all net-raids and file server are connected on exchange network by network channel again; The network user is connected on exchange network, and the network user is mutual by exchange network and file server and net-raid; All operations in this mass storage system (MSS) of file server charge of overseeing, and provide data description for the network user; Net-raid is responsible for the access of the network user's data; Peripheral channel is SCSI bus or FC bus or network channel.
Software configuration in mass storage system (MSS) is divided into three levels: passage key-course, order key-course and network communication layer;
(1), passage key-course: the order between responsible file server and net-raid group and the transmission of control information; The channel controller and the channel controller that resides in net-raid group that are divided into presence document server; Channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in net-raid group, is responsible for receiving order and the message from file server;
(2), order key-course: for resolving the network user's I/O request, obtain the network address of data place net-raid, realize request service; Be divided into command analysis device in file server and the command executer in net-raid group; The command analysis device that resides in file server authenticates and processes for network user's request; Reside in command executer in net-raid group for network user's request is decomposed, merged or dispatch, carries out;
(3), network communication layer: for the treatment of the order of memory node and the Internet Transmission of data of a plurality of network storage arrays; Be divided into the network communication layer of file server, the network user's network communication layer, net-raid group's network communication layer.
Embodiment 2:
A kind of high Performance Mass Storage System of the present invention, mass storage system (MSS) comprises file server and net-raid group, net-raid group comprises many net-raids, all net-raids are articulated on file server by peripheral channel by DAS mode, and then all net-raids and file server are connected on exchange network by network channel again; The network user is connected on exchange network, and the network user is mutual by exchange network and file server and net-raid; All operations in this mass storage system (MSS) of file server charge of overseeing, and provide data description for the network user; Net-raid is responsible for the access of the network user's data; Peripheral channel is SCSI bus or FC bus or network channel.
Software configuration in mass storage system (MSS) is divided into three levels: passage key-course, order key-course and network communication layer;
(1), passage key-course: the order between responsible file server and net-raid group and the transmission of control information; The channel controller and the channel controller that resides in net-raid group that are divided into presence document server; Channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in net-raid group, is responsible for receiving order and the message from file server;
(2), order key-course: for resolving the network user's I/O request, obtain the network address of data place net-raid, realize request service; Be divided into command analysis device in file server and the command executer in net-raid group; The command analysis device that resides in file server authenticates and processes for network user's request; Reside in command executer in net-raid group for network user's request is decomposed, merged or dispatch, carries out;
(3), network communication layer: for the treatment of the order of memory node and the Internet Transmission of data of a plurality of network storage arrays; Be divided into the network communication layer of file server, the network user's network communication layer, net-raid group's network communication layer.
Passage key-course: the channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in file server is only carried out the I/O task that wherein relates to control documents system architecture and file attribute; The channel controller of file server directly utilizes the existing SCSI channels drive of file server program; Channel controller in net-raid group, be responsible for to receive all I/O orders and message from file server principal and subordinate peripheral channel, by execution result loopback to file server.
Order key-course: the command analysis device that resides in file server comprises that user asks to process, address resolution, access control module, it is according to network user's information of recording, network user's request is authenticated, judge its legitimacy and request permissions, type according to request is processed respectively, file read-write request for the network user, after address resolution, generate the network address information of corresponding net-raid, after address resolution completes, subscriber authorisation is passed through to passage key-course informing network disk array group, by authorizing key, the network address information of net-raid is by network communication layer informing network user, the command executer residing in net-raid group comprises that I/O scheduling, order are decomposed/merged, Buffer administration module, dynamically the I/O order from peripheral channel and network channel is decomposed, merged and dispatches, carry out actual magnetic disc i/o operation, after having operated, via network communication layer and network user's swap file data, or via passage key-course to file server loopback execution result.
Network communication layer: the network communication layer of file server is accepted the network user's services request, sends order key-course to and the analysis result of order key-course is fed back to the network user; The network user's network communication layer is responsible for submitting to and asking and receive the result that file server returns to file server, and the network user's network communication layer is also responsible for setting up being connected with net-raid group's network; Net-raid group's network communication layer, together with the network user's network communication layer, completes the task of directly transmitting data between net-raid group and the network user.
Embodiment 3:
As shown in Figure 2, a kind of high-performance mass memory method of the present invention, is used mass storage system (MSS) to carry out the storage to data, and file organization mode is divided into two kinds:
(1), file organization mode independently: net-raid is as storage unit independently, every net-raid has independent directory node on file server, in each directory node, can set up different file system, store dissimilar file (as audiovisual, text etc.), and process separately access and the Internet Transmission of network user's data; When the network user sends when request to different directory node, a plurality of requests can be on a plurality of net-raids Parallel Service;
(2), distributed file organization mode: the storage space on the memory node of a plurality of net-raids is invented the unity logic view of a storage space, and deblocking leaves on a plurality of memory nodes; During in the face of the request of a plurality of small data quantities of the network user, can be by a plurality of memory nodes complete independently concurrently; While facing the request of a big data quantity of the network user, data are distributed on a plurality of memory nodes, a plurality of memory node parallel work-flows.
Along with the increase of memory node, not only the memory capacity of system has obtained expansion, and contributes to improve the performance of system; Specifically, the request of a plurality of small data quantities, can be by a plurality of memory nodes complete independently concurrently, thereby improves the handling capacity of system.The request of a big data quantity, data are distributed on a plurality of memory nodes, and a plurality of memory node parallel work-flows can improve several biography rates of single request.Use-pattern can meet different environments for use flexibly, both can meet the burst request of a plurality of small data quantities such as issued transaction, also can meet the continuous request of big data quantity.
Embodiment 4:
A kind of high-performance mass memory method of the present invention, is used mass storage system (MSS) to carry out the storage to data, and file organization mode is divided into two kinds:
(1), file organization mode independently: net-raid is as storage unit independently, every net-raid has independent directory node on file server, in each directory node, can set up different file system, store dissimilar file, and process separately access and the Internet Transmission of network user's data; When the network user sends when request to different directory node, a plurality of requests can be on a plurality of net-raids Parallel Service;
(2), distributed file organization mode: the storage space on the memory node of a plurality of net-raids is invented the unity logic view of a storage space, and deblocking leaves on a plurality of memory nodes; During in the face of the request of a plurality of small data quantities of the network user, can be by a plurality of memory nodes complete independently concurrently; While facing the request of a big data quantity of the network user, data are distributed on a plurality of memory nodes, a plurality of memory node parallel work-flows.
File organization mode is file organization mode independently, and the step of data storage is:
(1), when a network user attempts access data, it sends request by data representation to file server, the access control mechanism that file server starts it judges whether this network user has the authority of these data of storage, and then checks whether there are these data of other Subscriber Lockeds; If the network user can not access data, file server will send refusal or error message to the network user;
(2), if the network user has access right and is not refused by file locking, file server just judges the network address of data by data structure, the network address of data is that the memory node of the data place net-raid position in mass storage system (MSS) is (because Data Position is not only LBA (Logical Block Addressing), also comprise the additional network address or the title of memory node) and memory location, then send it to the network user, the network user reorganizes data structure after receiving the network address of data, then pass through the mutual also access data of network channel net-raid direct and this data place.
Now both can adopt the file I/O based on NFS or CIFS to carry out alternately, and also can adopt the block I/O based on SCSI agreement or FC agreement to carry out alternately.What independently file organization manager embodied is exactly a kind of thought of centralized management, so that reduce administration overhead, improves management quality.
File organization mode is distributed file organization mode, distributed data storage in mass storage system (MSS) is on the memory node of a plurality of net-raids, and the network user of all these mass storage system (MSS)s of use uses the network address of same file organization system Image Data; This document organization system realizes this reflection by usage data structure, the part of its data structure using the network address of all memory nodes as addressing data algorithm; Use RAID algorithm in storage system, to process file organization and data redundancy; Adopt partition that catalogue data, file data and data structure are crossed over and left on the memory node of a plurality of net-raids.
Because file data block redundancy leaves on a plurality of memory nodes, when being made a mistake, still can continue some memory nodes in storage system operation.Meanwhile, by increasing the quantity of memory node, the degree of parallelism that can increase physical data channel in system architecture improves the performance of system.
Embodiment 5:
A kind of high-performance mass memory method of the present invention, is used mass storage system (MSS) to carry out the storage to data, and file organization mode is divided into two kinds:
(1), file organization mode independently: net-raid is as storage unit independently, every net-raid has independent directory node on file server, in each directory node, can set up different file system, store dissimilar file, and process separately access and the Internet Transmission of network user's data; When the network user sends when request to different directory node, a plurality of requests can be on a plurality of net-raids Parallel Service;
(2), distributed file organization mode: the storage space on the memory node of a plurality of net-raids is invented the unity logic view of a storage space, and deblocking leaves on a plurality of memory nodes; During in the face of the request of a plurality of small data quantities of the network user, can be by a plurality of memory nodes complete independently concurrently; While facing the request of a big data quantity of the network user, data are distributed on a plurality of memory nodes, a plurality of memory node parallel work-flows.
File organization mode is file organization mode independently, and the step of data storage is:
(1), when a network user attempts access data, it sends request by data representation to file server, the access control mechanism that file server starts it judges whether this network user has the authority of these data of storage, and then checks whether there are these data of other Subscriber Lockeds; If the network user can not access data, file server will send refusal or error message to the network user;
(2) if the network user has access right and is not refused by file locking, file server just judges the network address of data by data structure, the network address of data is the memory node of data place net-raid position and the memory location in mass storage system (MSS), then send it to the network user, the network user reorganizes data structure after receiving the network address of data, then by network channel directly with the net-raid at this data place access data also alternately.
File organization mode is distributed file organization mode, distributed data storage in mass storage system (MSS) is on the memory node of a plurality of net-raids, and the network user of all these mass storage system (MSS)s of use uses the network address of same file organization system Image Data; This document organization system realizes this reflection by usage data structure, the part of its data structure using the network address of all memory nodes as addressing data algorithm; Use RAID algorithm in storage system, to process file organization and data redundancy; Adopt partition that catalogue data, file data and data structure are crossed over and left on the memory node of a plurality of net-raids.
The network user's I/O request adopts block I/O; Data flow from the network user's block I/O on mass storage system (MSS): the block I/O request from the network user itself has comprised the data structure of request msg on objective network disk array, the network address that comprises data in data structure, when file server receives after the network user's block I/O request by ICP/IP protocol stack, this I/O request is handed to volume manager, then by the device driver of net-raid, access corresponding data.
As shown in Figure 4, if the network user's I/O request adopts file I/O: the Virtual File System in the (SuSE) Linux OS of file server (VFS) is managed respectively the local disk file system that the network file system(NFS) of file-sharing and the data structure of control local network disk array are provided to the network user; When receive from the network user file I/O request time, file server is given Virtual File System, and find corresponding local disk file system by Virtual File System, then by local disk file system, by searching index node information, convert file I/O request to block I/O, finally by volume manager and block device driver, to objective network disk array, visit corresponding data.Block I/O has been avoided the switching of user's space and system space with respect to file I/O, avoided the expense of all I/O request and data passing through network file system, Virtual File System and local disk file system, the expense of search index node while simultaneously also having avoided disk file system to convert file I/O to block I/O.Therefore, mass storage system (MSS) adopts piece level I/O, and directly the physical address information by data visits net-raid, the speed while making the speed of data retrieval and access be better than adopting file I/O.
By embodiment above, described those skilled in the art can be easy to realize the present invention.But should be appreciated that the present invention is not limited to 5 kinds of above-mentioned embodiments.On the basis of disclosed embodiment, described those skilled in the art can the different technical characterictic of combination in any, thereby realizes different technical schemes.
Claims (9)
1. a high Performance Mass Storage System, it is characterized in that mass storage system (MSS) comprises file server and net-raid group, net-raid group comprises many net-raids, all net-raids are articulated on file server by peripheral channel by DAS mode, and then all net-raids and file server are connected on exchange network by network channel again; The network user is connected on exchange network, and the network user is mutual by exchange network and file server and net-raid; All operations in this mass storage system (MSS) of file server charge of overseeing, and provide data description for the network user; Net-raid is responsible for the access of the network user's data; Peripheral channel is SCSI bus or FC bus or network channel.
2. a kind of high Performance Mass Storage System according to claim 1, is characterized in that the software configuration in mass storage system (MSS) is divided into three levels: passage key-course, order key-course and network communication layer;
(1), passage key-course: the order between responsible file server and net-raid group and the transmission of control information; The channel controller and the channel controller that resides in net-raid group that are divided into presence document server; Channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in net-raid group, is responsible for receiving order and the message from file server;
(2), order key-course: for resolving the network user's I/O request, obtain the network address of data place net-raid, realize request service; Be divided into command analysis device in file server and the command executer in net-raid group; The command analysis device that resides in file server authenticates and processes for network user's request; Reside in command executer in net-raid group for network user's request is decomposed, merged or dispatch, carries out;
(3), network communication layer: for the treatment of the order of memory node and the Internet Transmission of data of a plurality of network storage arrays; Be divided into the network communication layer of file server, the network user's network communication layer, net-raid group's network communication layer.
3. a kind of high Performance Mass Storage System according to claim 2, it is characterized in that passage key-course: the channel controller in file server is for setting up file server and net-raid group's command channel, and the channel controller in file server is only carried out the I/O task that wherein relates to control documents system architecture and file attribute; The channel controller of file server directly utilizes the existing SCSI channels drive of file server program; Channel controller in net-raid group, be responsible for to receive all I/O orders and message from file server principal and subordinate peripheral channel, by execution result loopback to file server.
4. a kind of high Performance Mass Storage System according to claim 2, it is characterized in that order key-course: the command analysis device that resides in file server comprises that user asks to process, address resolution, access control module, it is according to network user's information of recording, network user's request is authenticated, judge its legitimacy and request permissions, type according to request is processed respectively, file read-write request for the network user, after address resolution, generate the network address information of corresponding net-raid, after address resolution completes, subscriber authorisation is passed through to passage key-course informing network disk array group, by authorizing key, the network address information of net-raid is by network communication layer informing network user, the command executer residing in net-raid group comprises that I/O scheduling, order are decomposed/merged, Buffer administration module, dynamically the I/O order from peripheral channel and network channel is decomposed, merged and dispatches, carry out actual magnetic disc i/o operation, after having operated, via network communication layer and network user's swap file data, or via passage key-course to file server loopback execution result.
5. a kind of high Performance Mass Storage System according to claim 2, it is characterized in that network communication layer: the network communication layer of file server is accepted the network user's services request, send order key-course to and the analysis result of order key-course is fed back to the network user; The network user's network communication layer is responsible for submitting to and asking and receive the result that file server returns to file server, and the network user's network communication layer is also responsible for setting up being connected with net-raid group's network; Net-raid group's network communication layer, together with the network user's network communication layer, completes the task of directly transmitting data between net-raid group and the network user.
6. a high-performance mass memory method, is characterized in that using mass storage system (MSS) to carry out the storage to data, and file organization mode is divided into two kinds:
(1), file organization mode independently: net-raid is as storage unit independently, every net-raid has independent directory node on file server, in each directory node, can set up different file system, store dissimilar file, and process separately access and the Internet Transmission of network user's data; When the network user sends when request to different directory node, a plurality of requests can be on a plurality of net-raids Parallel Service;
(2), distributed file organization mode: the storage space on the memory node of a plurality of net-raids is invented the unity logic view of a storage space, and deblocking leaves on a plurality of memory nodes; During in the face of the request of a plurality of small data quantities of the network user, can be by a plurality of memory nodes complete independently concurrently; While facing the request of a big data quantity of the network user, data are distributed on a plurality of memory nodes, a plurality of memory node parallel work-flows.
7. a kind of high-performance mass memory method according to claim 6, is characterized in that file organization mode is for file organization mode independently, and the step of data storage is:
(1), when a network user attempts access data, it sends request by data representation to file server, the access control mechanism that file server starts it judges whether this network user has the authority of these data of storage, and then checks whether there are these data of other Subscriber Lockeds; If the network user can not access data, file server will send refusal or error message to the network user;
(2) if the network user has access right and is not refused by file locking, file server just judges the network address of data by data structure, the network address of data is the memory node of data place net-raid position and the memory location in mass storage system (MSS), then send it to the network user, the network user reorganizes data structure after receiving the network address of data, then by network channel directly with the net-raid at this data place access data also alternately.
8. a kind of high-performance mass memory method according to claim 6, it is characterized in that file organization mode is distributed file organization mode, distributed data storage in mass storage system (MSS) is on the memory node of a plurality of net-raids, and the network user of all these mass storage system (MSS)s of use uses the network address of same file organization system Image Data; This document organization system realizes this reflection by usage data structure, the part of its data structure using the network address of all memory nodes as addressing data algorithm; Use RAID algorithm in storage system, to process file organization and data redundancy; Adopt partition that catalogue data, file data and data structure are crossed over and left on the memory node of a plurality of net-raids.
9. a kind of high-performance mass memory method according to claim 6, is characterized in that the network user's I/O request adopts block I/O; Data flow from the network user's block I/O on mass storage system (MSS): the block I/O request from the network user itself has comprised the data structure of request msg on objective network disk array, the network address that comprises data in data structure, when file server receives after the network user's block I/O request by ICP/IP protocol stack, this I/O request is handed to volume manager, then by the device driver of net-raid, access corresponding data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410371871.4A CN104102742A (en) | 2014-07-31 | 2014-07-31 | High-performance mass storage system and high-performance mass storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410371871.4A CN104102742A (en) | 2014-07-31 | 2014-07-31 | High-performance mass storage system and high-performance mass storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104102742A true CN104102742A (en) | 2014-10-15 |
Family
ID=51670895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410371871.4A Pending CN104102742A (en) | 2014-07-31 | 2014-07-31 | High-performance mass storage system and high-performance mass storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104102742A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN105892943A (en) * | 2016-03-30 | 2016-08-24 | 上海爱数信息技术股份有限公司 | Access method and system for block storage data in distributed storage system |
| CN107277106A (en) * | 2017-05-11 | 2017-10-20 | 芜湖威灵数码科技有限公司 | A kind of line holographic projections educational interactive storage system |
| CN108052286A (en) * | 2017-12-12 | 2018-05-18 | 郑州云海信息技术有限公司 | A kind of method, apparatus and equipment for managing heterogeneous storage devices |
| CN108369575A (en) * | 2015-10-05 | 2018-08-03 | 维卡艾欧有限公司 | Electronic storage system |
| CN108519863A (en) * | 2018-04-12 | 2018-09-11 | 郑州云海信息技术有限公司 | A kind of the IO management methods and device of storage system |
| CN110324429A (en) * | 2019-07-10 | 2019-10-11 | 中国工商银行股份有限公司 | Backup method and back-up device based on Distributed Storage |
| WO2020000275A1 (en) * | 2018-06-27 | 2020-01-02 | 华为技术有限公司 | Storage system, and method for switching operating mode of storage system |
| CN112965790A (en) * | 2021-03-29 | 2021-06-15 | 华云数据控股集团有限公司 | PXE protocol-based virtual machine starting method and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1655129A (en) * | 2005-02-25 | 2005-08-17 | 清华大学 | Universal method for dynamical management of storage resource under Windows platform |
| CN1829230A (en) * | 2006-03-21 | 2006-09-06 | 华南理工大学 | Intelligent network disc storage system and its realizing method |
| US20140025909A1 (en) * | 2012-07-10 | 2014-01-23 | Storone Ltd. | Large scale storage system |
-
2014
- 2014-07-31 CN CN201410371871.4A patent/CN104102742A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1655129A (en) * | 2005-02-25 | 2005-08-17 | 清华大学 | Universal method for dynamical management of storage resource under Windows platform |
| CN1829230A (en) * | 2006-03-21 | 2006-09-06 | 华南理工大学 | Intelligent network disc storage system and its realizing method |
| US20140025909A1 (en) * | 2012-07-10 | 2014-01-23 | Storone Ltd. | Large scale storage system |
Non-Patent Citations (1)
| Title |
|---|
| 李洁琼 等: "一种基于网络磁盘阵列的高性能海量存储系统", 《小型微型计算机系统》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN105282244B (en) * | 2015-09-29 | 2018-10-02 | 华为技术有限公司 | a kind of data processing method, device, server and controller |
| US11102322B2 (en) | 2015-09-29 | 2021-08-24 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| US10708378B2 (en) | 2015-09-29 | 2020-07-07 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| CN108369575A (en) * | 2015-10-05 | 2018-08-03 | 维卡艾欧有限公司 | Electronic storage system |
| CN108369575B (en) * | 2015-10-05 | 2021-08-27 | 维卡艾欧有限公司 | Memory system, operating method thereof and machine readable memory |
| CN105892943A (en) * | 2016-03-30 | 2016-08-24 | 上海爱数信息技术股份有限公司 | Access method and system for block storage data in distributed storage system |
| CN105892943B (en) * | 2016-03-30 | 2019-03-01 | 上海爱数信息技术股份有限公司 | The access method and system of block storing data in a kind of distributed memory system |
| CN107277106A (en) * | 2017-05-11 | 2017-10-20 | 芜湖威灵数码科技有限公司 | A kind of line holographic projections educational interactive storage system |
| CN108052286A (en) * | 2017-12-12 | 2018-05-18 | 郑州云海信息技术有限公司 | A kind of method, apparatus and equipment for managing heterogeneous storage devices |
| CN108519863A (en) * | 2018-04-12 | 2018-09-11 | 郑州云海信息技术有限公司 | A kind of the IO management methods and device of storage system |
| CN108519863B (en) * | 2018-04-12 | 2021-06-11 | 郑州云海信息技术有限公司 | IO management method and device of storage system |
| CN110998510B (en) * | 2018-06-27 | 2021-02-23 | 华为技术有限公司 | Storage system and method for switching working modes of storage system |
| CN110998510A (en) * | 2018-06-27 | 2020-04-10 | 华为技术有限公司 | Storage system and method for switching working modes of storage system |
| WO2020000275A1 (en) * | 2018-06-27 | 2020-01-02 | 华为技术有限公司 | Storage system, and method for switching operating mode of storage system |
| US11275699B2 (en) | 2018-06-27 | 2022-03-15 | Huawei Technologies Co., Ltd. | Storage system and method for switching working mode of storage system |
| US11550739B2 (en) | 2018-06-27 | 2023-01-10 | Huawei Technologies Co., Ltd. | Storage system and method for switching working mode of storage system |
| CN110324429A (en) * | 2019-07-10 | 2019-10-11 | 中国工商银行股份有限公司 | Backup method and back-up device based on Distributed Storage |
| CN112965790A (en) * | 2021-03-29 | 2021-06-15 | 华云数据控股集团有限公司 | PXE protocol-based virtual machine starting method and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104102742A (en) | High-performance mass storage system and high-performance mass storage method | |
| US11444641B2 (en) | Data storage system with enforced fencing | |
| US11042311B2 (en) | Cluster system with calculation and storage converged | |
| US11438411B2 (en) | Data storage system with redundant internal networks | |
| US11237772B2 (en) | Data storage system with multi-tier control plane | |
| US11467732B2 (en) | Data storage system with multiple durability levels | |
| JP5026283B2 (en) | Collaborative shared storage architecture | |
| US7389393B1 (en) | System and method for write forwarding in a storage environment employing distributed virtualization | |
| US6976060B2 (en) | Symmetric shared file storage system | |
| US7877545B2 (en) | Online restriping technique for distributed network based virtualization | |
| US9146780B1 (en) | System and method for preventing resource over-commitment due to remote management in a clustered network storage system | |
| WO2014183708A1 (en) | Method and system for realizing block storage of distributed file system | |
| WO2011120791A1 (en) | Transmission of map-reduce data based on a storage network or a storage network file system | |
| KR20080096547A (en) | Virtual network storage system, network storage device and virtual method | |
| US8924656B1 (en) | Storage environment with symmetric frontend and asymmetric backend | |
| US11461156B2 (en) | Block-storage service supporting multi-attach and health check failover mechanism | |
| KR102376152B1 (en) | Apparatus and method for providing storage for providing cloud services | |
| CN104994135B (en) | The method and device of SAN and NAS storage architectures is merged in storage system | |
| JP4208506B2 (en) | High-performance storage device access environment | |
| US10929041B1 (en) | Block-storage service supporting multi-attach | |
| US20130166670A1 (en) | Networked storage system and method including private data network | |
| KR101470857B1 (en) | Network distributed file system and method using iSCSI storage system | |
| WO2014180395A1 (en) | Mass data fusion storage method and system | |
| US20160357780A1 (en) | Hierarchical file block variant tracker apparatus coupled to a Librarian and to a remote file service | |
| US11625180B1 (en) | Fault tolerant reservation state distribution for multi-partition logical volumes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141015 |