CN111078154A - Method and system for quickly loading data - Google Patents
Method and system for quickly loading data Download PDFInfo
- Publication number
- CN111078154A CN111078154A CN201911328147.2A CN201911328147A CN111078154A CN 111078154 A CN111078154 A CN 111078154A CN 201911328147 A CN201911328147 A CN 201911328147A CN 111078154 A CN111078154 A CN 111078154A
- Authority
- CN
- China
- Prior art keywords
- data
- protocol
- client
- san storage
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0661—Format or protocol conversion arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention discloses a method and a system for quickly loading data, wherein the method comprises the following steps: the client sends SCSI command, and after being encapsulated into IP data packet by ISCSI layer and TCP/IP protocol stack, the SCSI command is transmitted to the application server through IP network; wherein, the SCSI command is a block I/O command; the application server de-encapsulates the IP data packet, then restores the IP data packet into a SCSI command, and sends a block I/O read-write request to SAN storage equipment through the SCSI command; the SAN storage equipment acquires corresponding request data from cached data based on the block I/O read-write request, encapsulates the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmits the PDU data packet back to the client through an IP network; the data cached by the SAN storage equipment is acquired from the disk array by the NAS equipment; and the client receives the PDU data packet, then decapsulates the PDU data packet and combines the PDU data packet into a file. The invention also discloses a system for rapidly loading data, which realizes rapid loading of data.
Description
Technical Field
The invention relates to the field of computer communication, in particular to a method and a system for quickly loading data.
Background
The network storage systems in the mainstream at present mainly include a Network Attached Storage (NAS) and a Storage Area Network (SAN). As defined by the storage network industry association, NAS is a storage device that can be directly connected to a network to provide file-level services to users, and SAN is a network that can directly transfer data between servers and storage systems connected using an interconnection protocol such as Fibre Channel. The NAS is a storage device which is provided with a simplified real-time operating system, effectively integrates hardware and software together to provide file services, and has good sharing, openness and expandability. The storage devices of SAN technology are connected by a private network, which is a fibre channel protocol based network. Performance is high due to the separation of the fibre channel storage network and the LAN. In SAN, capacity expansion, data migration, local data backup and remote disaster recovery data backup are all convenient, and the whole SAN becomes a storage pool with unified management.
However, in practical applications, NAS and SAN have many defects, and are increasingly unable to meet the rapid development of IT technology and the explosive growth of digital information.
For example, the NAS device has the following defects:
(1) the data transfer speed is slow because NAS can only provide file level and not block level data transfer;
(2) the performance is low when the data is backed up, the NAS occupies most of the network bandwidth when the data is backed up, and other I/O performance is influenced;
(3) only a single NAS can be managed, and it is difficult to centrally manage a plurality of NAS located in the same local area network.
SAN also suffers from the following drawbacks:
(1) the interoperability of the equipment is poor, and the equipment of different manufacturers is difficult to interoperate;
(2) the cost for constructing SAN is high, and only enterprises with high strength construct the SAN;
(3) the management and maintenance cost is high, and enterprises need to spend money to train special management and maintenance personnel;
(4) a SAN can only provide storage space sharing and cannot provide file sharing in heterogeneous environments.
Disclosure of Invention
Aiming at the problems, the invention aims to provide a method and a system for rapidly loading data, which realize a unified storage network system fusing NAS and SAN under an IP protocol, can simultaneously support a file protocol and a block protocol through a global multi-protocol file system, realize the seamless fusion of NAS equipment and SAN equipment on the IP, meet the requirements of application openness, high expansion and mass storage, realize the unified storage network through iSCSI and simultaneously provide file I/O and block I/O services for clients, and have the advantages of both NAS and SAN.
The invention provides a method for quickly loading data, which comprises the following steps:
the client sends SCSI command, and after being encapsulated into IP data packet by ISCSI layer and TCP/IP protocol stack, the SCSI command is transmitted to the application server through IP network; wherein, the SCSI command is a block I/O command;
the application server de-encapsulates the IP data packet, then restores the IP data packet into a SCSI command, and sends a block I/O read-write request to SAN storage equipment through the SCSI command;
the SAN storage equipment acquires corresponding request data from cached data based on the block I/O read-write request, encapsulates the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmits the PDU data packet back to the client through an IP network; the data cached by the SAN storage equipment is acquired from the disk array by the NAS equipment;
and the client receives the PDU data packet, then decapsulates the PDU data packet and combines the PDU data packet into a file.
Preferably, the method further comprises the following steps:
the NAS device obtains data from the disk array through the first data switch; the data in the disk array are provided with data labels;
the NAS device encapsulates the data through a TCP/IP protocol, and sends the encapsulated data to the SAN storage device based on a preset protocol mapping relation; the protocol mapping relation is a mapping relation between an iSCSI protocol and a TCP/IP protocol;
and the SAN storage equipment caches the received data sent by the NAS equipment.
Preferably, the first and second electrodes are formed of a metal,
the application server and the SAN storage device realize communication through a second data interaction machine, and the first data switch and the second data switch are both optical fiber switches.
Preferably, the method further comprises the following steps:
binding the IP of the first data switch and the IP of the second data switch through a gateway; the IP of the second data switch is an external network IP, and the IP of the first data switch is an internal network IP correspondingly.
Preferably, the first and second electrodes are formed of a metal,
and the PDU data packet is forwarded to the client through the application server or is directly transmitted to the client through a high-speed network attachment channel.
Preferably, the first and second electrodes are formed of a metal,
when the data volume of the requested data is larger than a preset threshold value, sending the data cached by the SAN storage equipment to a client through a network-attached high-speed channel;
and when the data volume of the requested data is not greater than the preset threshold value, sending the data cached by the SAN storage device to an application server through a server channel, and forwarding the data to the client by the application server.
The invention also provides a data rapid loading system, which comprises:
the system comprises a client, an application server, SAN storage equipment, NAS equipment and a disk array; wherein the content of the first and second substances,
the client is used for sending SCSI commands, is packaged into IP data packets by an ISCSI layer and a TCP/IP protocol stack, and then is transmitted to the application server through an IP network; wherein, the SCSI command is a block I/O command;
the application server is used for recovering the IP data packet into a SCSI command after de-encapsulation, and sending a block I/O read-write request to the SAN storage equipment through the SCSI command;
the SAN storage device is used for acquiring corresponding request data from cached data based on the block I/O read-write request, packaging the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmitting the PDU data packet back to the client through an IP network; the data cached by the SAN storage equipment is acquired from the disk array by the NAS equipment;
and the client is used for de-encapsulating and combining the PDU data packets into a file after receiving the PDU data packets.
Preferably, the method further comprises the following steps:
a first data switch; wherein the content of the first and second substances,
the NAS device is configured to acquire data from the disk array through the first data switch; the data in the disk array are provided with data labels;
the NAS device is further configured to encapsulate the data through a TCP/IP protocol, and send the encapsulated data to the SAN storage device based on a preset protocol mapping relationship; the protocol mapping relation is a mapping relation between an iSCSI protocol and a TCP/IP protocol;
the SAN storage device is used for caching the received data sent by the NAS device.
Preferably, the method further comprises the following steps:
a second data switch; wherein the content of the first and second substances,
the application server and the SAN storage device realize communication through the second data interaction machine, and the first data switch and the second data switch are both optical fiber switches.
Preferably, the method further comprises the following steps:
binding the IP of the first data switch and the IP of the second data switch through a gateway; the IP of the second data switch is an external network IP, and the IP of the first data switch is an internal network IP correspondingly.
In the technical scheme, the NAS device obtains data of a corresponding data label from the disk array through the first data switch, the NAS device encapsulates the data through a TCP/IP protocol, and sends the encapsulated data to the SAN storage device based on a preset protocol mapping relation (a mapping relation between an iSCSI protocol and the TCP/IP protocol), and the SAN storage device caches the received data sent by the NAS device. The method realizes the seamless integration of the NAS device and the SAN storage device on the IP, meets the requirements of application openness, high expansion and mass storage, simultaneously provides file I/O and block I/O services for the client through iSCSI, integrates the advantages of the NAS device and the SAN storage device, selects a server channel according to the attribute of data related to the user I/O request, autonomously adjusts according to the I/O information quantity of the whole system, improves the I/O performance service of the user, and reduces the bottleneck of the server.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a data fast loading method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a data fast loading system according to an embodiment of the present invention.
A description of the reference numerals;
the system comprises a data fast loading system 100, a client 101, an application server 102, a SAN storage device 103, a NAS device 104, a disk array 105, a first switch 106 and a second switch 107.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a data fast loading method according to an embodiment of the present invention.
The invention provides a data rapid loading method, which at least comprises the following steps:
s1, the client 101 sends SCSI order, which is packaged into IP data packet by ISCSI layer and TCP/IP protocol stack, and then transmitted to the application server 102 through IP network; wherein the SCSI command is a block I/O command.
In this embodiment, the SCSI Command is defined in a Command Descriptor Block (CDB), which contains the operation code defining the particular operation to be performed, as well as a number of operation-specific parameters.
SCSI commands support read-write data (four variables each) as well as many non-data commands such as test-unit-ready (whether the device is ready), inquiry (retrieving basic information about the target device), read-capacity (retrieving the storage capacity of the target device), and so on. What kind of commands the target device supports depends on the type of device, and the initiator identifies the device type by inquiry command.
In this embodiment, Transmission Control Protocol is abbreviated as TCP, which is translated into a Transmission Control Protocol, and is a connection-oriented, reliable, byte stream-based transport layer communication Protocol.
Internet Protocol is abbreviated as IP, also translated into Internet Protocol or Internet Protocol, and is a network layer Protocol used in a TCP/IP Protocol cluster. The IP protocol is located at the network layer of the TCP/IP protocol, and the protocol located at the same layer also comprises the following ARP and RARP, and the upper ICMP (Internet control message protocol) and the upper IGMP (Internet group management protocol). Almost all data, except ARP and RARP messages, is transmitted over the IP protocol. The ARP and RARP messages are not encapsulated in IP datagrams, while ICMP and IGMP data are encapsulated in IP datagrams for transmission. The network layer of the TCP/IP protocol is also referred to as the IP layer, since the IP protocol has an important position in the network layer.
In this embodiment, the iSCSI protocol defines the mapping from SCSI to TCP/IP, that is, the SCSI command of the client 101 is encapsulated into an IP data packet, transmitted over the IP network, and restored to the SCSI command before encapsulation after reaching the destination node (application server 102), thereby implementing direct and transparent transmission of the SCSI command over the IP network.
The iSCSI protocol integrates two main protocols of the existing storage protocol SCSI and the main network protocol TCP/IP, and realizes seamless integration of storage and network. From the application point of view, on one hand, the iSCSI realizes the command level interaction with remote storage equipment through the remote transmission of SCSI commands, so that a user can access the remote SCSI equipment as conveniently as local SCSI equipment and has high speed; on the other hand, the method can also be used for modifying the traditional NAS and SAN technologies to realize the fusion of the NAS and the SAN. The design of iSCSI implements an IP-based data block access mechanism.
The current iSCSI implementation may use the following three approaches: pure software mode, intelligent iSCSI network card implementation mode and iSCSI HBA card implementation mode.
S2, the application server 102 decapsulates the IP data packet, restores the IP data packet to a SCSI command, and sends a block I/O read/write request to the SAN storage device 103 through the SCSI command.
In the embodiment, the encapsulated IP data packet is transmitted to the application server 102 through an IP network, many protocols of an application layer are based on a client/server (C/S) mode, the client and the server refer to two application processes involved in communication, the C/S mode describes a service and a served relationship between the processes, the client 101 is a service requester, and the application server 102 is a service provider.
Particularly, in this embodiment, the application server 102 is installed with a GMPFS system, the GMPFS may reside on multiple operating system platforms (UNIX, Windows, Linux), support access of users with various protocols (NFS, CIFS, iSCSI), provide data access service to a network storage system for users or application programs, and the GMPFS collects user application information by expanding metadata used by the current storage system and using a heuristic method to provide a uniform, convenient, and fast storage access interface and a reasonable data storage scheme for users.
S3, the SAN storage device 103 obtains corresponding request data from the cached data based on the block I/O read-write request, encapsulates the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmits the PDU data packet back to the client 101 through an IP network; the data cached by the SAN storage device 103 is acquired from the disk array 105 by the NAS device 104.
In this embodiment, the disk array 105 is a disk assembly with a large capacity composed of many independent disks, the performance of the entire disk system is improved by the additive effect of providing data by individual disks, the data is divided into many segments and stored in each hard disk, the data can be read when any hard disk in the array fails by the concept of parity check, and the data is calculated and then re-placed into a new hard disk when the data is reconstructed.
In this embodiment, the PDU, abbreviated as PDU, refers to a protocol data unit that is to be established in each layer of a transmission system in a layered network structure, for example, in an Open System Interconnection (OSI) model.
Specifically, the NAS device 104 obtains data of a corresponding data tag from the disk array 105 through the first data switch, the NAS device 104 encapsulates the data through a TCP/IP protocol, and sends the encapsulated data to the SAN storage device 103 based on a preset protocol mapping relationship (a mapping relationship between an iSCSI protocol and a TCP/IP protocol), and the SAN storage device 103 caches the received data sent from the NAS device 104, so that when the SAN storage device 103 receives a data request, the cached data can be quickly returned to the application server 102 or the client 101.
S4, the client 101 receives the PDU packets, decapsulates them and assembles them into a file.
In this embodiment, the PDU packet is forwarded to the client 101 through the application server 102 or directly transmitted to the client 101 through a high speed network channel.
Specifically, when the data volume of the requested data is greater than a preset threshold, the data cached by the SAN storage device 103 is sent to the client 101 through the network-attached high-speed channel;
when the data volume of the requested data is not greater than the preset threshold, the data cached by the SAN storage device 103 is sent to the application server 102 through the server channel, and then forwarded to the client 101 by the application server 102.
In this embodiment, a proper manner is selected to send the cache data to the client 101 according to the request of the data size, so as to improve the I/O performance service of the user, reduce the server bottleneck, and construct based on the IP technology, so that the present storage system can be compatible, it is very convenient to add and delete the storage device, the performance and the expansibility of the whole system are very good, and the unification of the NAS and the SAN is really realized, that is, both the NAS device and the SAN structure exist in the same storage network.
In this embodiment, the application server 102 and the SAN storage device 103 implement communication through a second data exchange, and the first data exchange and the second data exchange are both optical fiber exchanges.
In this embodiment, the optical fiber switch is a high-speed network transmission relay device, which is called a fibre channel switch and a SAN switch, and it adopts an optical fiber cable as a transmission medium compared with a general switch. The optical fiber transmission has the advantages of high speed and strong anti-interference capability. There are two main types of fabric switches, one is used to connect storage FC switches. The other is an ethernet switch, and the ports are fiber optic interfaces that look the same as ordinary electrical interfaces, but are of a different type.
It should be noted that the IPs of the first data switch and the second data switch are bound through a gateway; the IP of the second data switch is an external network IP, and the IP of the first data switch is an internal network IP correspondingly.
In summary, the embodiment implements seamless integration of the NAS device and the SAN storage device on the IP, meets the requirements of application openness, high expansion, and mass storage, provides file I/O and block I/O services to the client through iSCSI, integrates the advantages of the NAS device and the SAN storage device, selects a server channel according to the attribute of data related to the user I/O request, autonomously adjusts according to the I/O information amount of the entire system, improves the I/O performance service of the user, and reduces the server bottleneck.
Referring to fig. 2, fig. 2 is a block diagram illustrating a data fast loading system 100 according to an embodiment of the present invention.
The method comprises the following steps: a client 101, an application server 102, a SAN storage device 103, a NAS device 104, and a disk array 105; wherein the content of the first and second substances,
the client 101 is configured to send an SCSI command, and after being encapsulated into an IP data packet by an ISCSI layer and a TCP/IP protocol stack, the IP data packet is transmitted to the application server 102 through an IP network; wherein, the SCSI command is a block I/O command;
the application server 102 is configured to recover to an SCSI command after decapsulating the IP data packet, and send a block I/O read-write request to the SAN storage device through the SCSI command;
the SAN storage device 103 is configured to obtain corresponding request data from cached data based on the block I/O read-write request, encapsulate the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmit the PDU data packet back to the client 101 through an IP network; the data cached by the SAN storage device 103 is acquired from the disk array 105 by the NAS device 104;
and the client 101 is configured to decapsulate and combine the PDU packets into a file after receiving the PDU packets.
In a preferred embodiment of the present invention provided,
further comprising: a first data switch 106; wherein the content of the first and second substances,
the NAS device 104 is configured to obtain data from the disk array 105 through the first data switch 106; wherein, the data in the disk array 105 has a data tag;
the NAS device 104 is further configured to encapsulate the data through a TCP/IP protocol, and send the encapsulated data to the SAN storage device 103 based on a preset protocol mapping relationship; the protocol mapping relation is a mapping relation between an iSCSI protocol and a TCP/IP protocol;
the SAN storage device 103 is configured to cache the received data sent from the NAS device 104.
The present invention also provides a preferred embodiment of the method,
further comprising:
a second data switch 107; wherein the content of the first and second substances,
the application server 102 and the SAN storage device 103 realize communication through the second data exchange 107, and the first data exchange 106 and the second data exchange 107 are both fabric switches.
The present invention also provides a preferred embodiment of the method,
further comprising:
binding the IP of the first data switch 106 and the second data switch 107 through a gateway; the IP of the second data switch 107 is an external network IP, and the IP of the first data switch 106 is an internal network IP.
To sum up, the NAS device 104 first obtains data of a corresponding data tag from the disk array 105 through the first data switch, the NAS device 104 encapsulates the data through a TCP/IP protocol, and sends the encapsulated data to the SAN storage device 103 based on a preset protocol mapping relationship (a mapping relationship between an iSCSI protocol and a TCP/IP protocol), and the SAN storage device 103 caches the received data sent from the NAS device 104, so that when the SAN storage device 103 receives a data request, the cached data can be quickly returned to the application server 102 or the client 101.
The whole SAN storage device 103 is realized to be a storage pool which is managed uniformly, thereby being convenient for the data request of the client 101, increasing the data extraction speed, meanwhile, the storage array is adopted to effectively carry out multi-data backup, prevent data loss caused by hardware failure, support file protocol and block protocol, realize seamless integration of the NAS device 104 and the SAN storage device 103 on IP, meet the requirements of application openness, high expansion and mass storage, providing both file I/O and block I/O services to the client 101 over iSCSI, merging the advantages of both the NAS device 104 and the SAN storage device 103, the ASA selects a server channel according to the attribute of data related to the user I/O request, autonomously adjusts according to the I/O information quantity of the whole system, improves the I/O performance service of the user and reduces the server bottleneck.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Claims (10)
1. A data fast loading method is characterized by comprising the following steps:
the client sends SCSI command, and after being encapsulated into IP data packet by ISCSI layer and TCP/IP protocol stack, the SCSI command is transmitted to the application server through IP network; wherein, the SCSI command is a block I/O command;
the application server de-encapsulates the IP data packet, then restores the IP data packet into a SCSI command, and sends a block I/O read-write request to SAN storage equipment through the SCSI command;
the SAN storage equipment acquires corresponding request data from cached data based on the block I/O read-write request, encapsulates the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmits the PDU data packet back to the client through an IP network; the data cached by the SAN storage equipment is acquired from the disk array by the NAS equipment;
and the client receives the PDU data packet, then decapsulates the PDU data packet and combines the PDU data packet into a file.
2. The method for fast loading data according to claim 1, further comprising:
the NAS device obtains data from the disk array through the first data switch; the data in the disk array are provided with data labels;
the NAS device encapsulates the data through a TCP/IP protocol, and sends the encapsulated data to the SAN storage device based on a preset protocol mapping relation; the protocol mapping relation is a mapping relation between an iSCSI protocol and a TCP/IP protocol;
and the SAN storage equipment caches the received data sent by the NAS equipment.
3. The method for fast loading data according to claim 2, wherein the application server and the SAN storage device communicate with each other through a second data exchange, and the first data exchange and the second data exchange are both fabric switches.
4. The data fast loading method according to claim 3, further comprising:
binding the IP of the first data switch and the IP of the second data switch through a gateway; the IP of the second data switch is an external network IP, and the IP of the first data switch is an internal network IP correspondingly.
5. The method according to claim 1, wherein the PDU packet is forwarded to the client through the application server or directly transmitted to the client through a high speed network attachment channel.
6. The method of claim 5,
when the data volume of the requested data is larger than a preset threshold value, sending the data cached by the SAN storage equipment to a client through a network-attached high-speed channel;
and when the data volume of the requested data is not greater than the preset threshold value, sending the data cached by the SAN storage device to an application server through a server channel, and forwarding the data to the client by the application server.
7. A system for fast loading data, comprising:
the system comprises a client, an application server, SAN storage equipment, NAS equipment and a disk array; wherein the content of the first and second substances,
the client is used for sending SCSI commands, is packaged into IP data packets by an ISCSI layer and a TCP/IP protocol stack, and then is transmitted to the application server through an IP network; wherein, the SCSI command is a block I/O command;
the application server is used for recovering the IP data packet into a SCSI command after de-encapsulation, and sending a block I/O read-write request to the SAN storage equipment through the SCSI command;
the SAN storage device is used for acquiring corresponding request data from cached data based on the block I/O read-write request, packaging the request data into a PDU data packet through an ISCSI layer and a TCP/IP protocol stack, and transmitting the PDU data packet back to the client through an IP network; the data cached by the SAN storage equipment is acquired from the disk array by the NAS equipment;
and the client is used for de-encapsulating and combining the PDU data packets into a file after receiving the PDU data packets.
8. The data fastload system of claim 7, further comprising:
a first data switch; wherein the content of the first and second substances,
the NAS device is configured to acquire data from the disk array through the first data switch; the data in the disk array are provided with data labels;
the NAS device is further configured to encapsulate the data through a TCP/IP protocol, and send the encapsulated data to the SAN storage device based on a preset protocol mapping relationship; the protocol mapping relation is a mapping relation between an iSCSI protocol and a TCP/IP protocol;
the SAN storage device is used for caching the received data sent by the NAS device.
9. The data fastload system of claim 8, further comprising:
a second data switch; wherein the content of the first and second substances,
the application server and the SAN storage device realize communication through the second data interaction machine, and the first data switch and the second data switch are both optical fiber switches.
10. The data fastload system of claim 9, further comprising:
binding the IP of the first data switch and the IP of the second data switch through a gateway; the IP of the second data switch is an external network IP, and the IP of the first data switch is an internal network IP correspondingly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911328147.2A CN111078154A (en) | 2019-12-20 | 2019-12-20 | Method and system for quickly loading data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911328147.2A CN111078154A (en) | 2019-12-20 | 2019-12-20 | Method and system for quickly loading data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111078154A true CN111078154A (en) | 2020-04-28 |
Family
ID=70316389
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911328147.2A Pending CN111078154A (en) | 2019-12-20 | 2019-12-20 | Method and system for quickly loading data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078154A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897492A (en) * | 2020-07-15 | 2020-11-06 | 杭州海康威视系统技术有限公司 | Data processing method and device based on block device driver and electronic device |
| CN113542412A (en) * | 2021-07-16 | 2021-10-22 | 中国电信股份有限公司 | Data transmission method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102223409A (en) * | 2011-06-13 | 2011-10-19 | 浪潮(北京)电子信息产业有限公司 | Network storage resource application system and method |
| CN202856789U (en) * | 2012-08-17 | 2013-04-03 | 广州中长康达信息技术有限公司 | Data resource storage system integrating NAS and SAN |
| US20180278686A1 (en) * | 2017-03-23 | 2018-09-27 | Netapp, Inc. | Methods and systems for managing networked storage system resources |
-
2019
- 2019-12-20 CN CN201911328147.2A patent/CN111078154A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102223409A (en) * | 2011-06-13 | 2011-10-19 | 浪潮(北京)电子信息产业有限公司 | Network storage resource application system and method |
| CN202856789U (en) * | 2012-08-17 | 2013-04-03 | 广州中长康达信息技术有限公司 | Data resource storage system integrating NAS and SAN |
| US20180278686A1 (en) * | 2017-03-23 | 2018-09-27 | Netapp, Inc. | Methods and systems for managing networked storage system resources |
Non-Patent Citations (1)
| Title |
|---|
| 韩德志，余顺争，谢长生: "融合NAS和SAN的存储网络设计与实现", 《电子学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897492A (en) * | 2020-07-15 | 2020-11-06 | 杭州海康威视系统技术有限公司 | Data processing method and device based on block device driver and electronic device |
| CN111897492B (en) * | 2020-07-15 | 2023-02-28 | 杭州海康威视系统技术有限公司 | Data processing method and device based on block device driver and electronic device |
| CN113542412A (en) * | 2021-07-16 | 2021-10-22 | 中国电信股份有限公司 | Data transmission method and device, electronic equipment and storage medium |
| CN113542412B (en) * | 2021-07-16 | 2024-01-05 | 天翼云科技有限公司 | Data transmission method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5026283B2 (en) | Collaborative shared storage architecture | |
| US6389432B1 (en) | Intelligent virtual volume access | |
| US8682968B2 (en) | Method, system, and node for node interconnection on content delivery network | |
| US7404000B2 (en) | Protocol translation in a storage system | |
| US7089293B2 (en) | Switching system method for discovering and accessing SCSI devices in response to query | |
| US6985956B2 (en) | Switching system | |
| US8060695B1 (en) | System and method for proxying data access commands in a clustered storage system | |
| US7865596B2 (en) | Switching system for managing storage in digital networks | |
| US20170237668A1 (en) | Tcp/udp acceleration | |
| US7185062B2 (en) | Switch-based storage services | |
| US20050138184A1 (en) | Efficient method for sharing data between independent clusters of virtualization switches | |
| US20040117438A1 (en) | Switching system | |
| US11921597B2 (en) | Cross-platform replication | |
| CN1985492B (en) | Method and system for supporting iSCSI read operations and iSCSI chimney | |
| US7707304B1 (en) | Storage switch for storage area network | |
| US7864758B1 (en) | Virtualization in a storage system | |
| CN111078154A (en) | Method and system for quickly loading data | |
| WO2002037300A1 (en) | Switching system | |
| JP2003517765A (en) | Encapsulation protocol that links storage area networks via packet-based networks | |
| Long | Storage Networking Protocol Fundamentals | |
| US10798159B2 (en) | Methods for managing workload throughput in a storage system and devices thereof | |
| US7539711B1 (en) | Streaming video data with fast-forward and no-fast-forward portions | |
| JP2005505819A (en) | Packet classification in storage systems | |
| Ortenberg et al. | Overview of Storage Networking and Storage Networks | |
| Godbole | Performance Evaluation of iSCSI Protocol for Mirroring Application. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200428 |
|
| RJ01 | Rejection of invention patent application after publication |