WO2018045860A1 - File system mounting method, apparatus and equipment - Google Patents

File system mounting method, apparatus and equipment Download PDF

Info

Publication number
WO2018045860A1
WO2018045860A1 PCT/CN2017/097681 CN2017097681W WO2018045860A1 WO 2018045860 A1 WO2018045860 A1 WO 2018045860A1 CN 2017097681 W CN2017097681 W CN 2017097681W WO 2018045860 A1 WO2018045860 A1 WO 2018045860A1
Authority
WO
WIPO (PCT)
Prior art keywords
communication
storage nodes
client
target
target storage
Prior art date
Application number
PCT/CN2017/097681
Other languages
French (fr)
Chinese (zh)
Inventor
刘渊
张大力
黎德雄
文海
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018045860A1 publication Critical patent/WO2018045860A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present invention relates to the field of communications, and in particular, to a method, device, and device for mounting a file system.
  • Network storage implements data transmission based on standard network protocols, and provides file sharing and data backup for clients of various operating systems such as Windows/Linux/Mac OS in the network. Multiple clients connect to the same storage system (ie, network storage) to achieve different business needs of different clients. For example, basic reading of files in the storage system, or writing files to the storage system. Since the storage system has a large storage capacity requirement, the storage system generally consists of multiple storage nodes.
  • a single client When a client accesses multiple files in the storage system, a single client requires a very high stable bandwidth (usually >2GB/s). However, as people's needs are higher and higher, more and more businesses need higher and more stable bandwidth, such as video production, high-performance computer group (English name: High Performance Computing, English abbreviation: HPC) Waiting for business.
  • HPC High Performance Computing
  • a single client can only establish a connection with a single storage node of the storage system. When the stability of a single storage node in a storage system cannot meet the requirements of a single workstation, it will affect the normal operation of the service, such as video services, and may be stuck.
  • the present application provides a method, device, and device for mounting a file system, which are used to improve the transmission rate between a client and a storage system, and improve the operational stability of a service requiring high bandwidth.
  • the first aspect of the present application provides a method for mounting a file system, where the file system is used to manage files stored in a storage system, and the method includes:
  • the storage system includes a plurality of storage nodes, and the client selects a plurality of storage nodes from the storage system as the target storage node, where the target storage node is a storage node that the client needs to establish a communication connection, and the client obtains the identifiers of the plurality of target storage nodes.
  • the identifier may be a unique identifier of the storage node, such as a network interconnection protocol (English full name: Internet Protocol, abbreviation: IP) address, or a serial number of a storage node, or a node name of a storage node, etc., according to the client.
  • IP Internet Protocol
  • the identifier of the target storage node establishes a plurality of communication connections with the plurality of target storage nodes, and the client uses the plurality of communication connections to mount the file system to create a mount point.
  • the client obtains the identifiers of the plurality of target storage nodes, and the client establishes multiple communication connections with the plurality of target storage nodes according to the identifiers of the plurality of target storage nodes, and the client uses multiple communication connections to mount the file system.
  • multiple communication connections established with multiple storage nodes are used, so that the client can access multiple storage nodes in parallel, thereby improving the transmission between the client and the storage system.
  • the rate also increases the operational stability of the high bandwidth requirements of the service.
  • the method may further include:
  • the client determines, from the storage nodes included in the storage system, a storage node storing any of the data blocks in the target file as the target storage node.
  • a single storage node In a storage system, a single storage node generally stores only data blocks of a file.
  • the client preferentially selects a storage node that establishes a connection and stores a data block of the target file that the client needs to obtain, for example, the storage node includes The first node, the second node, and the third node, the object file is divided into two partial data blocks and stored in the first node and the second node, respectively.
  • the first node and the second node storing the target file data block are preferentially selected, so that the client can directly obtain the target file from the first node and the second node.
  • the target file reduces the time for the storage node connected to the client to obtain the target file data block from other storage nodes, thereby improving the data transmission efficiency between the client and the storage system.
  • the client establishes multiple communication connections with multiple target storage nodes according to the identifiers of the multiple target storage nodes, which may be:
  • the client determines a connection index of the plurality of target storage nodes from the connection index library according to the identifier of the target storage node, and the client establishes multiple communication connections with the plurality of target storage nodes according to the connection index of the plurality of target storage nodes.
  • the client saves a connection index library, which stores a connection index used by each storage node to establish a communication connection with the client for the first time. So that the next time the client needs to establish a communication connection with the storage node, it is only necessary to match the identifier of the target storage node acquired by the client with the identifier of the storage node of the connection index library, and the corresponding connection index can be found.
  • the client does not need to send a remote call request and return an entry handle to the storage node to complete a preparation operation for establishing a communication connection, thereby improving the efficiency of establishing a communication connection between the client and the storage node.
  • the method may further include:
  • the client After the client mounts the file system, the client determines at least one communication connection from the plurality of communication connections, and the client accesses the corresponding target storage node based on the at least one communication connection to acquire the at least one target file.
  • the client and the storage system transmit at a file-level granularity, when the client acquires multiple target files at the same time, the client can access multiple target storage nodes in parallel through multiple communication connections to obtain multiple target files. For example, the client can access three storage nodes in parallel to obtain three target files, thus improving the service running speed of the client.
  • the second aspect of the present application provides an apparatus for mounting a file system, where the file system is used to manage files stored in a storage system, and the device may be implemented by software or by a combination of software and hardware, in an implementation manner.
  • the device includes:
  • the obtaining unit is configured to obtain identifiers of the plurality of target storage nodes, where the storage system includes multiple storage nodes, where the identifier may be a unique identifier of the storage node, such as an IP address, or a serial number of the storage node, or a node name of the storage node. ;
  • a connecting unit configured to establish multiple communication connections with multiple target storage nodes according to identifiers of multiple target storage nodes
  • Mount unit for mounting a file system using multiple communication connections.
  • the apparatus includes: a transceiver, a memory, and a processor, the transceiver, the memory, and the processor are connected by a bus, the memory stores the computer instructions, and the processor implements the above by executing computer instructions in the memory.
  • the method of mounting a file system provided by the first aspect.
  • a third aspect provides an apparatus comprising at least one processor, a memory, and a communication interface.
  • the at least one processor, the memory, and the communication interface are each connected by a bus; the memory storage computer executes instructions; the at least one processor executes computer stored instructions of the memory storage such that the device passes
  • the communication interface performs data interaction with the storage system to perform the method of mounting the file system provided by the foregoing first aspect or various possible designs of the first aspect, or causes the device to perform data interaction with the storage system through the communication interface.
  • the apparatus for mounting a file system in the second aspect is implemented.
  • a fourth aspect of the present application provides a computer readable storage medium having stored therein computer executed instructions, when the at least one processor of the device executes the computer to execute an instruction, the device performs the first aspect or the first aspect A variety of possible ways to design a method of mounting a file system.
  • a fifth aspect of the present application provides a computer program product comprising computer executed instructions stored in a computer readable storage medium.
  • At least one processor of the device can read the computer-executable instructions from the computer readable storage medium, and the at least one processor executes the computer-executable instructions to cause the device to implement the mounting file provided by the first aspect or the various possible designs of the first aspect Systematic approach.
  • FIG. 1 is a schematic diagram of a system architecture of a file system in an embodiment of the invention
  • FIG. 2 is a schematic diagram of a method for mounting a file system according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of an apparatus for mounting a file system according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a device in an embodiment of the present invention.
  • the embodiment of the invention provides a method, a device and a device for mounting a file system, which are used to improve the transmission rate between the client and the storage system, and improve the operational stability of the service required by the high bandwidth.
  • FIG. 1 is a schematic structural diagram of a system for mounting a file system according to an embodiment of the present invention.
  • the storage system is composed of a plurality of storage nodes, and the plurality of storage nodes are connected to the switch through the optical fiber, and the switch connects the client through the optical fiber, thereby interconnecting the plurality of storage nodes and the client.
  • the optical fiber can be a 10 Gigabit Ethernet (Gigabit Ethernet, English abbreviation: GE) fiber optic cable
  • the switch can be a 10 GE high-rate transmission switch.
  • the storage node stores the entire file or the data block of the target file that the client needs to obtain, and the plurality of storage nodes can mutually transmit data, and the client can access the storage node by accessing a storage node or The target file that is not stored on this storage node.
  • the storage node traverses the other storage nodes in the entire storage system by using the identifier of the target file, thereby acquiring data of the target file from other storage nodes.
  • a single client can also establish a communication connection with multiple storage nodes in the storage system by using the method for mounting files provided by the present invention, thereby implementing parallel access to multiple storage nodes to obtain target files, and improving the relationship between the client and the storage system. Data transfer rate.
  • the client may be a personal computer (English name: Personal Computer, English abbreviation: PC) or a computing device such as a server; specifically, the device runs a client program that can access the storage system to make The device becomes a client.
  • the storage system can be network attached storage (English full name: Network Attached Storage, English abbreviation: NAS), open system direct-attached storage (English full name: Direct Access Storage English abbreviation: DAS), storage area network (English full name: Storage Area Network, English abbreviation: SAN) and so on.
  • the hardware medium of the storage node can be isomorphic or heterogeneous.
  • NAS storage any type of the storage node
  • FIG. 2 is an embodiment of a method for mounting a file system according to an embodiment of the present invention, where the file system is used to manage files stored in a storage system, and the method is specifically:
  • the client acquires identifiers of multiple target storage nodes, where the storage system includes multiple storage nodes.
  • the client needs to establish multiple communication connections with multiple storage nodes in the storage system.
  • the client Before establishing a communication connection, the client needs to obtain the identifiers of the plurality of target storage nodes, and the identifier may be the IP address of the storage node.
  • the client obtains the IP address of the storage node, and the client initiates a floating IP query to the storage system by using the domain name system (English name: Domain Name System, English abbreviation: DNS).
  • DNS Domain Name System
  • the floating IP address can be multiple.
  • the physical IP is matched with a virtual IP, where each floating IP corresponds to one storage node.
  • the DNS server returns a list containing the floating IP addresses in the storage system.
  • the identifier of the target storage node may be a unique identifier of the storage node, and is not limited to an IP address, and may also be a serial number of the storage node or a node name of the storage node.
  • the DNS server belongs to the storage system and is responsible for domain name resolution of the storage system.
  • the client can obtain the identifiers of all the storage nodes in the storage system, and establish a communication connection with all the storage nodes.
  • the client may also obtain the identifier of a part of the storage nodes in the storage system, and establish a communication connection with some storage nodes.
  • the client can select according to the connection load condition of all storage nodes, preferentially select a storage node with less connection load, and establish a communication connection with the storage node.
  • the client may also select a storage node by using a location of the target file to be acquired. Specifically, the client first determines, from the storage node included in the storage system, a storage node that stores any data block in the target file as the target storage node.
  • a single storage node generally stores only data blocks of files, and rarely stores the entire file. Therefore, when the client needs to obtain a complete target file, the storage node to which the client is connected is required. Other data blocks of the target file are obtained from other storage nodes, so that the client obtains the entire target file from the connected storage node.
  • the client preferentially selects the storage node that establishes the connection to store the data block of the target file that the client needs to acquire, for example, the storage node includes the first node, the second node, and the third node, and the target file is The two partial data blocks are respectively stored in the first node and the second node.
  • the first node and the second node storing the target file data block are preferentially selected, so that the client can directly obtain the target file from the first node and the second node.
  • the target file reduces the time for the storage node connected to the client to obtain the target file data block from other storage nodes, thereby improving the data transmission efficiency between the client and the storage system.
  • the client establishes multiple communication connections with the multiple target storage nodes according to the identifiers of the multiple target storage nodes.
  • the client After the client obtains the identifiers of the plurality of target storage nodes, that is, the floating IP address of the target storage node, the client can establish a communication connection with the plurality of target storage nodes, and the specific way of establishing the communication connection is: the client According to the list of floating IP addresses, the storage node corresponding to any floating IP (such as the first one) requests a mount connection, for example, the request may be a remote call request sent by the file system, such as a MOUNT RPC call request; The storage node corresponding to a floating IP will return the entry handle of the file system root directory through the mount module (for example, the mountd module), for example, ROOT FILEHANDLE; the client traverses the floating IP address list, and respectively corresponds to multiple corresponding storage nodes.
  • the mount module for example, the mountd module
  • the network file system (English name: Network File System, English abbreviation: NFS) service module (such as NFS Server) establishes a communication connection.
  • the communication connection can be a socket connection, and after the client establishes all the Socket connections, the connection state of all the Socket connections can also be saved.
  • the Socket connection status of each storage node includes information such as the normal or abnormal state of the Socket connection state of the storage node and the load status of the Socket connection of the storage node.
  • the client acquires an entry handle of the plurality of storage nodes in real time through the obtained floating IP addresses of the plurality of storage nodes, thereby establishing a communication connection.
  • the client may further determine a connection index of the multiple target storage nodes from the pre-stored connection index database, and then according to the multiple The connection index establishes a communication connection with a plurality of storage nodes.
  • the client Since the entry handle and the floating IP address of each storage node are generally not changed, the client will establish a communication connection with the client after acquiring the entry handle of the storage node and the floating IP address of the storage node for the first time. And storing the connection index of the communication connection, that is, the entry handle and the floating IP address of the storage node, into the connection index library, so that the next time the client needs to establish a communication connection with the storage node, the client only needs to If the obtained floating IP address matches the IP address in the connection index database, the corresponding connection index can be found, thereby establishing a communication connection.
  • the client does not need to send a remote call request and return an entry handle to the storage node to complete a preparation operation for establishing a communication connection, thereby improving the efficiency of establishing a communication connection between the client and the storage node.
  • the connection index of the storage node is not saved in the connection index library, so that the client needs to establish a communication connection through the above-mentioned real-time.
  • the storage node establishes a communication connection, and after establishing the communication connection, saves the connection index of the storage node in the connection index library.
  • the client mounts the file system by using the multiple communication connections.
  • the client After the client establishes a communication connection between multiple target storage nodes, it can instantiate the local mount point, that is, create a file system local mount point.
  • the mount point is actually the entry directory of the disk file system in the Linux operating system, similar to the C:, D:, E: and other drive letters used to access different partitions in the Windows operating system.
  • the client After the client instantiates the mount point, it can access the connected storage node to obtain the target file.
  • the client obtains the identifiers of the plurality of target storage nodes
  • the storage system includes the plurality of storage nodes
  • the client establishes the plurality of target storages according to the identifiers of the plurality of target storage nodes.
  • a plurality of communication connections of the node the client using the plurality of communication connections to mount the file system.
  • multiple communication connections established with multiple storage nodes are used, so that the client can access multiple storage nodes in parallel, thereby improving the transmission between the client and the storage system.
  • the rate also increases the operational stability of the high bandwidth requirements of the service.
  • the method for mounting a file system further includes:
  • the client After the client mounts the file system, the client determines at least one communication connection from the plurality of communication connections;
  • the client accesses the corresponding target storage node based on the at least one communication connection to acquire at least one object file.
  • the client can access multiple storage nodes according to the communication connections to obtain the target files in the storage node.
  • the client may select one or more communication connections from the plurality of communication connections to access one or more storage nodes to obtain one or more object files.
  • the manner in which the client obtains the target file may be transmitted by using the file as the granularity, or may be transmitted by using the data block of the target file stored in the storage node as a granularity. lose.
  • the client may select a communication connection to access a storage node.
  • the storage node If the storage node only stores a part of the data block of the target file or a data block that does not store the target file, the storage node is based on the identifier of the target file, such as a file.
  • the name searches for other storage nodes to obtain all the data blocks of the target file, thereby transmitting all the data blocks of the entire target file to the client.
  • the client may access multiple storage nodes simultaneously by using multiple communication connections, and obtain different data blocks of the target file from different storage nodes, and the client Re-integrate different data blocks to generate a complete object file.
  • the client may further acquire the multiple target files by accessing the corresponding target storage nodes in parallel according to the multiple communication connections.
  • the client and the storage system transmit at a file-level granularity, when the client acquires multiple target files at the same time, the client can access multiple target storage nodes in parallel through multiple communication connections to obtain multiple target files. For example, the client can access three storage nodes in parallel to obtain three target files.
  • the client obtains the target file, which may include a business operation such as a read operation or a write operation by the client on the target file.
  • the client determines the at least one communication connection from the multiple communication connections, and specifically:
  • the client determines at least one communication connection having a small communication load from the plurality of communication connections.
  • the connection state of the storage node is stored, and the connection state includes the load condition of the storage node.
  • the client can preferentially select the current communication load to teach the storage node of the small communication connection to perform data transmission.
  • the client may determine, from the plurality of target storage nodes, a first storage node having a small number of communication connections, and then determine, from the plurality of communication connections, a communication connection with the first storage node as a communication.
  • a small communication connection may be determined, from the plurality of target storage nodes, a first storage node having a small number of communication connections, and then determine, from the plurality of communication connections, a communication connection with the first storage node as a communication.
  • the two or more communication connections selected by the client are communication connections with small communication load.
  • the client may first obtain size information of each of the at least two target files.
  • the client calculates the time consumed by each file transfer according to the transmission rate of the storage system
  • the client calculates, according to the time consumed by each file transfer, the average number of files to be transmitted by each storage node according to the file granularity
  • the client acquires the at least two object files by accessing a plurality of storage nodes in parallel by the number of files.
  • the client needs to obtain four target files, and the client selects two communication connections for the transmission of the four files.
  • the client first sends a read request for the four files to the storage system, and the storage system obtains the size information of each of the four files according to the file identifier of the four files, wherein if the first file is 10G, The second file is 1G, the third file is 3G, and the fourth file is 2G.
  • the transmission is performed through two connections. If the transmission rate of each communication connection is 1G/ s, then it takes 10s to transfer the first file, 1s to transfer the second file, 3s to transfer the third file, and 2s to transfer the fourth file.
  • the first file can be transmitted through one communication line, and the other three files can be transmitted through another communication link.
  • the size of the fourth file is 8G
  • the total size of the four files is 22G, which is transmitted through two communication lines, and each of the communication lines can transmit two files on average.
  • the first file and the second file are transmitted on one communication line, and the second file and the third file are transmitted on another communication line. In this way, the data size load of data transmission between the client and the storage system is more balanced, and the transmission efficiency between the client and the storage system is improved.
  • the unified authentication module in the NAS storage can also be transplanted to the client, and the client and the NAS storage respectively solidify a pair of identical accounts. And the password is encrypted and saved. Then the client acts as the authentication client.
  • the NAS storage serves as the authentication server.
  • the unified authentication mechanism is used to complete the two-way authentication. After the two-way authentication is passed, the file system mount and file access can be performed.
  • an embodiment of an apparatus for mounting a file system in an embodiment of the present invention includes:
  • An obtaining unit 201 configured to acquire identifiers of multiple target storage nodes, where the storage system includes multiple storage nodes;
  • the connecting unit 202 is configured to establish, according to the identifiers of the multiple target storage nodes, multiple communication connections with the multiple target storage nodes;
  • the mounting unit 203 is configured to mount the file system by using the multiple communication connections.
  • the device includes:
  • a determining unit 204 configured to determine, from the storage node included in the storage system, a storage node that stores any data block in the target file as the target storage, before the obtaining unit acquires the identifiers of the plurality of target storage nodes node.
  • the connecting unit 202 is specifically configured to:
  • connection index library Determining, according to the identifier of the target storage node, a connection index of the plurality of target storage nodes from the connection index library
  • the determining unit 204 is further configured to: after the mounting unit 203 mounts the file system, determine at least one communication connection from the multiple communication connections;
  • the obtaining unit 201 is further configured to acquire at least one target file based on the at least one communication connection accessing the corresponding target storage node.
  • the determining unit 204 is specifically configured to:
  • At least one communication connection having a small communication load is determined from the plurality of communication connections.
  • the determining unit 204 is specifically configured to:
  • a communication connection with the first storage node is determined as a communication connection with a small communication load.
  • the obtaining unit 201 is further configured to:
  • the target storage nodes corresponding to the plurality of communication connections are accessed in parallel to obtain a plurality of target files.
  • the embodiment of the present invention further provides a storage system, which is a storage system described in the foregoing embodiment, and is configured to cooperate with a client to implement the foregoing method for mounting a file system.
  • the embodiment of the present invention further provides a storage node, which belongs to the storage system described in the foregoing embodiment, and the storage node is a storage node described in the foregoing embodiment, and the storage node is configured to cooperate with a client to implement the foregoing.
  • the method of mounting a file system is a storage node, which belongs to the storage system described in the foregoing embodiment, and the storage node is a storage node described in the foregoing embodiment, and the storage node is configured to cooperate with a client to implement the foregoing.
  • an embodiment of the present invention further provides an apparatus 300.
  • the device 300 includes at least one processor 301, a memory 302, and a communication interface 303.
  • the interfaces 303 are all connected by a bus 304;
  • the memory 302 is configured to store a computer execution instruction
  • the at least one processor 301 is configured to execute a computer-executed instruction stored by the memory 302, so that the device 300 performs data interaction with the storage system through the communication interface 303 to execute the mounted file system provided by the foregoing method embodiment.
  • the at least one processor 301 may include different types of processors 301, or include the same type of processor 301; the processor 301 may be any one of the following: a central processing unit (CPU), an ARM processor. , Field Programmable Gate Array (FPGA), dedicated processor and other devices with computational processing capabilities. In an optional implementation manner, the at least one processor 301 may also be integrated into a many-core processor.
  • processors 301 may include different types of processors 301, or include the same type of processor 301; the processor 301 may be any one of the following: a central processing unit (CPU), an ARM processor. , Field Programmable Gate Array (FPGA), dedicated processor and other devices with computational processing capabilities.
  • the at least one processor 301 may also be integrated into a many-core processor.
  • the memory 302 may be any one or any combination of the following: a random access memory (RAM), a read only memory (ROM), a non-volatile memory (non-volatile memory). , referred to as NVM), Solid State Drives (SSD), mechanical hard disks, disks, disk arrays and other storage media.
  • RAM random access memory
  • ROM read only memory
  • NVM non-volatile memory
  • SSD Solid State Drives
  • Communication interface 303 is used by device 300 for data interaction with other devices, such as storage nodes in a storage system.
  • the communication interface 303 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, or the like having a network access function.
  • the bus 304 can include an address bus, a data bus, a control bus, etc., for convenience of representation, Figure 4 shows the bus with a thick line.
  • the bus 304 may be any one or any combination of the following: an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, and an extended industry standard structure ( Extended Industry Standard Architecture (EISA) bus and other devices for wired data transmission.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated order The element can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Abstract

A method for mounting a file system, comprising: a client acquiring indicators of multiple target storage nodes (101); the client establishing multiple communication links to the multiple target storage nodes according to the indicators of the multiple target storage modes (102); the client using the multiple communication links to mount a file system (103). In this manner, when a single client is establishing mount points, the client uses multiple communication connections established to multiple storage nodes, enabling the client to access multiple storage nodes in parallel, thereby increasing transmission speed between the client and a storage system, and also increasing reliability of operation of services requiring high bandwidth.

Description

一种挂载文件系统的方法、装置及设备Method, device and device for mounting file system 技术领域Technical field
本发明涉及通信领域,尤其涉及一种挂载文件系统的方法、装置及设备。The present invention relates to the field of communications, and in particular, to a method, device, and device for mounting a file system.
背景技术Background technique
网络存储基于标准网络协议实现数据传输,为网络中的Windows/Linux/Mac OS等各种不同操作系统的客户端提供文件共享和数据备份。多个客户端通过连接同一存储系统(即网络存储)从而实现不同客户端的不同业务需求。比如基本的读取存储系统中的文件,或者往存储系统中写入文件等。由于存储系统对存储容量的要求较大,所以存储系统一般由多个存储节点组成。Network storage implements data transmission based on standard network protocols, and provides file sharing and data backup for clients of various operating systems such as Windows/Linux/Mac OS in the network. Multiple clients connect to the same storage system (ie, network storage) to achieve different business needs of different clients. For example, basic reading of files in the storage system, or writing files to the storage system. Since the storage system has a large storage capacity requirement, the storage system generally consists of multiple storage nodes.
客户端在访问存储系统中的多个文件时,单个客户端就需要很高的稳定带宽(通常>2GB/s)。但是,随着人们的需求越来越高,越来越多的业务需要更高更稳定的带宽才能够实现,比如视频制作、高性能计算机群(英文全称:High Performance Computing,英文缩写:HPC)等的业务。而在现有技术中,基于标准NFS协议,单个客户端只能与存储系统的单个存储节点建立连接。当存储系统中的单个存储节点稳点带宽不能满足单台工作站需求时,则会影响业务正常运行,比如视频业务,可能会出现卡顿情况。因此,如现有的连接机制中,单个客户端在运行高带宽要求的业务时,由于单节点带宽不稳定,提供的带宽不足,使得客户端与存储系统之间的传输速率较低,从而导致这类业务的正常运行。When a client accesses multiple files in the storage system, a single client requires a very high stable bandwidth (usually >2GB/s). However, as people's needs are higher and higher, more and more businesses need higher and more stable bandwidth, such as video production, high-performance computer group (English name: High Performance Computing, English abbreviation: HPC) Waiting for business. In the prior art, based on the standard NFS protocol, a single client can only establish a connection with a single storage node of the storage system. When the stability of a single storage node in a storage system cannot meet the requirements of a single workstation, it will affect the normal operation of the service, such as video services, and may be stuck. Therefore, in the existing connection mechanism, when a single client runs a service with high bandwidth requirements, the bandwidth of the single node is unstable, and the bandwidth provided is insufficient, so that the transmission rate between the client and the storage system is low, resulting in a low transmission rate. The normal operation of this type of business.
发明内容Summary of the invention
本申请提供了一种挂载文件系统的方法、装置及设备,用于提高客户端与存储系统之间的传输速率,提高了高带宽要求业务的运行稳定性。The present application provides a method, device, and device for mounting a file system, which are used to improve the transmission rate between a client and a storage system, and improve the operational stability of a service requiring high bandwidth.
本申请第一方面提供一种挂载文件系统的方法,该文件系统用于管理存储系统中存储的文件,该方法包括:The first aspect of the present application provides a method for mounting a file system, where the file system is used to manage files stored in a storage system, and the method includes:
该存储系统包括多个存储节点,客户端从存储系统中选择多个存储节点作为目标存储节点,该目标存储节点为客户端需要建立通信连接的存储节点,客户端获取多个目标存储节点的标识,该标识可以为存储节点的唯一识别标识,比如网络互连协议(英文全称:Internet Protocol,英文缩写:简称IP)地址,或者存储节点的序号,或者存储节点的节点名称等,客户端根据多个目标存储节点的标识,与多个目标存储节点建立多个通信连接,客户端再使用该多个通信连接挂载文件系统,创建挂载点。The storage system includes a plurality of storage nodes, and the client selects a plurality of storage nodes from the storage system as the target storage node, where the target storage node is a storage node that the client needs to establish a communication connection, and the client obtains the identifiers of the plurality of target storage nodes. The identifier may be a unique identifier of the storage node, such as a network interconnection protocol (English full name: Internet Protocol, abbreviation: IP) address, or a serial number of a storage node, or a node name of a storage node, etc., according to the client. The identifier of the target storage node establishes a plurality of communication connections with the plurality of target storage nodes, and the client uses the plurality of communication connections to mount the file system to create a mount point.
客户端获取多个目标存储节点的标识,客户端根据多个目标存储节点的标识,建立与多个目标存储节点的多个通信连接,客户端使用多个通信连接挂载文件系统。这样,单个客户端所建立的挂载点中,使用了与多个存储节点所建立的多条通信连接,使得客户端能够并行访问多个存储节点,从而提高客户端与存储系统之间的传输速率,同时也提高了高带宽要求业务的运行稳定性。The client obtains the identifiers of the plurality of target storage nodes, and the client establishes multiple communication connections with the plurality of target storage nodes according to the identifiers of the plurality of target storage nodes, and the client uses multiple communication connections to mount the file system. In this way, among the mount points established by a single client, multiple communication connections established with multiple storage nodes are used, so that the client can access multiple storage nodes in parallel, thereby improving the transmission between the client and the storage system. The rate also increases the operational stability of the high bandwidth requirements of the service.
在一种可能的实现方式中,在客户端获取多个目标存储节点的标识之前,该方法还可以包括:In a possible implementation manner, before the client obtains the identifiers of the multiple target storage nodes, the method may further include:
客户端从存储系统包括的存储节点中,确定存储有目标文件中任一数据块的存储节点作为所述目标存储节点。 The client determines, from the storage nodes included in the storage system, a storage node storing any of the data blocks in the target file as the target storage node.
在存储系统中,单个存储节点一般只存储有文件的数据块,在该实现方式中,客户端优先选择建立连接的存储节点存储有该客户端需要获取的目标文件的数据块,比如存储节点包括第一节点、第二节点以及第三节点,目标文件被分为两部分数据块分别存储于第一节点与第二节点中。而客户端在与存储节点建立通信连接之前,优先选择存储有目标文件数据块的第一节点与第二节点,从而使得客户端在获取目标文件时能够直接从第一节点与第二节点中获取目标文件,减少了客户端所连接的存储节点从其它存储节点获取目标文件数据块的时间,从而提高了客户端与存储系统之间的数据传输效率。In a storage system, a single storage node generally stores only data blocks of a file. In this implementation manner, the client preferentially selects a storage node that establishes a connection and stores a data block of the target file that the client needs to obtain, for example, the storage node includes The first node, the second node, and the third node, the object file is divided into two partial data blocks and stored in the first node and the second node, respectively. Before the client establishes a communication connection with the storage node, the first node and the second node storing the target file data block are preferentially selected, so that the client can directly obtain the target file from the first node and the second node. The target file reduces the time for the storage node connected to the client to obtain the target file data block from other storage nodes, thereby improving the data transmission efficiency between the client and the storage system.
在另一种可能的实现方式中,客户端根据多个目标存储节点的标识,建立与多个目标存储节点的多个通信连接具体可以为:In another possible implementation manner, the client establishes multiple communication connections with multiple target storage nodes according to the identifiers of the multiple target storage nodes, which may be:
客户端根据目标存储节点的标识,从连接索引库中确定多个目标存储节点的连接索引,客户端根据多个目标存储节点的连接索引,建立与多个目标存储节点的多个通信连接。The client determines a connection index of the plurality of target storage nodes from the connection index library according to the identifier of the target storage node, and the client establishes multiple communication connections with the plurality of target storage nodes according to the connection index of the plurality of target storage nodes.
在该实现方式中,客户端保存有一个连接索引库,该连接索引库中保存有每个存储节点与该客户端首次建立通信连接时所使用的连接索引。以便下次客户端再需要与该存储节点建立通信连接时,只需将客户端所获取到的目标存储节点的标识与连接索引库的存储节点的标识进行匹配,则能找到所对应的连接索引,从而建立通信连接。无需客户端再与存储节点进行发送远程调用请求和返回入口句柄等一系列建立通信连接的准备操作,提高了客户端与存储节点建立通信连接的效率。In this implementation, the client saves a connection index library, which stores a connection index used by each storage node to establish a communication connection with the client for the first time. So that the next time the client needs to establish a communication connection with the storage node, it is only necessary to match the identifier of the target storage node acquired by the client with the identifier of the storage node of the connection index library, and the corresponding connection index can be found. To establish a communication connection. The client does not need to send a remote call request and return an entry handle to the storage node to complete a preparation operation for establishing a communication connection, thereby improving the efficiency of establishing a communication connection between the client and the storage node.
在另一种可能的实现方式中,该方法还可以包括:In another possible implementation manner, the method may further include:
在客户端挂载所述文件系统后,客户端从多个通信连接中确定至少一个通信连接,客户端基于至少一个通信连接访问所对应的目标存储节点来获取至少一个目标文件。After the client mounts the file system, the client determines at least one communication connection from the plurality of communication connections, and the client accesses the corresponding target storage node based on the at least one communication connection to acquire the at least one target file.
若客户端与存储系统之间以文件为粒度进行传输,那么客户端在同时获取多个目标文件时,则可以通过多个通信连接并行访问多个目标存储节点来获取多个目标文件。比如客户端可以并行访问三个存储节点从而获取三个目标文件,这样,提高了客户端的业务运行速率。If the client and the storage system transmit at a file-level granularity, when the client acquires multiple target files at the same time, the client can access multiple target storage nodes in parallel through multiple communication connections to obtain multiple target files. For example, the client can access three storage nodes in parallel to obtain three target files, thus improving the service running speed of the client.
本申请第二方面提供一种挂载文件系统的装置,该文件系统用于管理存储系统中存储的文件,该装置可以通过软件实现,也可以通过软硬结合的方式实现,在一种实现方式中,该装置包括:The second aspect of the present application provides an apparatus for mounting a file system, where the file system is used to manage files stored in a storage system, and the device may be implemented by software or by a combination of software and hardware, in an implementation manner. The device includes:
获取单元,用于获取多个目标存储节点的标识,存储系统包括多个存储节点,该标识可以为存储节点的唯一识别标识,比如IP地址,或者存储节点的序号,或者存储节点的节点名称等;The obtaining unit is configured to obtain identifiers of the plurality of target storage nodes, where the storage system includes multiple storage nodes, where the identifier may be a unique identifier of the storage node, such as an IP address, or a serial number of the storage node, or a node name of the storage node. ;
连接单元,用于根据多个目标存储节点的标识,建立与多个目标存储节点的多个通信连接;a connecting unit, configured to establish multiple communication connections with multiple target storage nodes according to identifiers of multiple target storage nodes;
挂载单元,用于使用多个通信连接挂载文件系统。Mount unit for mounting a file system using multiple communication connections.
在另一种实现方式中,该装置包括:收发器、存储器以及处理器,收发器、存储器以及处理器通过总线连接,存储器内存储有计算机指令,处理器通过执行存储器内的计算机指令实现如上述第一方面所提供的挂载文件系统的方法。In another implementation, the apparatus includes: a transceiver, a memory, and a processor, the transceiver, the memory, and the processor are connected by a bus, the memory stores the computer instructions, and the processor implements the above by executing computer instructions in the memory. The method of mounting a file system provided by the first aspect.
第三方面提供一种设备,该设备括至少一个处理器、存储器和通信接口。所述至少一个处理器、所述存储器和所述通信接口均通过总线连接;所述存储器存储计算机执行指令;所述至少一个处理器执行所述存储器存储的计算机执行指令,使得所述设备通过 所述通信接口与存储系统进行数据交互来执行上述第一方面或者第一方面的各种可能设计提供的挂载文件系统的方法,或者使得所述设备通过所述通信接口与存储系统进行数据交互来实现第二方面中的挂载文件系统的装置。A third aspect provides an apparatus comprising at least one processor, a memory, and a communication interface. The at least one processor, the memory, and the communication interface are each connected by a bus; the memory storage computer executes instructions; the at least one processor executes computer stored instructions of the memory storage such that the device passes The communication interface performs data interaction with the storage system to perform the method of mounting the file system provided by the foregoing first aspect or various possible designs of the first aspect, or causes the device to perform data interaction with the storage system through the communication interface. The apparatus for mounting a file system in the second aspect is implemented.
本申请第四方面提供一种计算机可读存储介质,计算机可读存储介质中存储有计算机执行指令,当设备的至少一个处理器执行该计算机执行指令时,设备执行上述第一方面或者第一方面的各种可能设计提供的挂载文件系统的方法。A fourth aspect of the present application provides a computer readable storage medium having stored therein computer executed instructions, when the at least one processor of the device executes the computer to execute an instruction, the device performs the first aspect or the first aspect A variety of possible ways to design a method of mounting a file system.
本申请第五方面提供一种计算机程序产品,该计算机程序产品包括计算机执行指令,该计算机执行指令存储在计算机可读存储介质中。设备的至少一个处理器可以从计算机可读存储介质读取该计算机执行指令,至少一个处理器执行该计算机执行指令使得设备实施上述第一方面或者第一方面的各种可能设计提供的挂载文件系统的方法。A fifth aspect of the present application provides a computer program product comprising computer executed instructions stored in a computer readable storage medium. At least one processor of the device can read the computer-executable instructions from the computer readable storage medium, and the at least one processor executes the computer-executable instructions to cause the device to implement the mounting file provided by the first aspect or the various possible designs of the first aspect Systematic approach.
附图说明DRAWINGS
为了更清楚地说明本发明实施例的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍。In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the description of the embodiments will be briefly described below.
图1为发明实施例中挂载文件系统的一个系统架构示意图;1 is a schematic diagram of a system architecture of a file system in an embodiment of the invention;
图2为本发明实施例中挂载文件系统的方法的一个示意图;2 is a schematic diagram of a method for mounting a file system according to an embodiment of the present invention;
图3为本发明实施例中挂载文件系统的装置的一个示意图;3 is a schematic diagram of an apparatus for mounting a file system according to an embodiment of the present invention;
图4为本发明实施例中设备的一个示意图。4 is a schematic diagram of a device in an embodiment of the present invention.
具体实施方式detailed description
下面将结合附图,对本发明实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present invention will be clearly and completely described in the following with reference to the accompanying drawings.
本发明实施例提供了一种挂载文件系统的方法、装置及设备,用于提高客户端与存储系统之间的传输速率,提高了高带宽要求业务的运行稳定性。The embodiment of the invention provides a method, a device and a device for mounting a file system, which are used to improve the transmission rate between the client and the storage system, and improve the operational stability of the service required by the high bandwidth.
参照图1所示,图1为本发明实施例中挂载文件系统的一个系统架构示意图。在该系统架构中,包括客户端,存储系统。其中,存储系统由多个存储节点组成,多个存储节点通过光纤连接交换机,该交换机通过光纤连接该客户端,从而实现多个存储节点与客户端的互联。其中,光纤可以为10千兆以太网(英文全称:Gigabit Ethernet,英文缩写:GE)光纤线,交换机可以为10GE的高速率传输交换机。存储节点中存储有客户端需要获取的目标文件的整个文件或者文件的数据块,多个存储节点之间能够进行数据互传,客户端可以通过访问某一个存储节点来获取存储于该存储节点或者不存储于该存储节点上的目标文件。当客户端需要获取的目标文件不存储于该存储节点时,该存储节点通过该目标文件的标识遍历整个存储系统中的其它存储节点,从而从其它存储节点中获取该目标文件的数据。单个客户端还可以通过本发明提供的挂载文件的方法与该存储系统中的多个存储节点建立通信连接,从而实现并行访问多个存储节点来获取目标文件,提高客户端与存储系统之间的数据传输速率。Referring to FIG. 1, FIG. 1 is a schematic structural diagram of a system for mounting a file system according to an embodiment of the present invention. In the system architecture, including the client, the storage system. The storage system is composed of a plurality of storage nodes, and the plurality of storage nodes are connected to the switch through the optical fiber, and the switch connects the client through the optical fiber, thereby interconnecting the plurality of storage nodes and the client. The optical fiber can be a 10 Gigabit Ethernet (Gigabit Ethernet, English abbreviation: GE) fiber optic cable, and the switch can be a 10 GE high-rate transmission switch. The storage node stores the entire file or the data block of the target file that the client needs to obtain, and the plurality of storage nodes can mutually transmit data, and the client can access the storage node by accessing a storage node or The target file that is not stored on this storage node. When the target file that the client needs to obtain is not stored in the storage node, the storage node traverses the other storage nodes in the entire storage system by using the identifier of the target file, thereby acquiring data of the target file from other storage nodes. A single client can also establish a communication connection with multiple storage nodes in the storage system by using the method for mounting files provided by the present invention, thereby implementing parallel access to multiple storage nodes to obtain target files, and improving the relationship between the client and the storage system. Data transfer rate.
需要说明的是,该客户端可以为个人计算机(英文全称:Personal Computer,英文缩写:PC)或者服务器等具有计算能力的设备;具体地,该设备上运行可以访问存储系统的客户端程序来使得该设备成为客户端。存储系统可以为网络附属存储(英文全称:Network Attached Storage,英文缩写:NAS)、开放系统的直连式存储(英文全称:Direct Access Storage英文缩写:DAS),存储区域网络(英文全称:Storage Area Network,英文缩写:SAN)等。存储节点的硬件介质可同构或异构。为了便于阐述,本发明实施例以NAS存储为例进行说明,该举例并非对本发明方案的限定。 It should be noted that the client may be a personal computer (English name: Personal Computer, English abbreviation: PC) or a computing device such as a server; specifically, the device runs a client program that can access the storage system to make The device becomes a client. The storage system can be network attached storage (English full name: Network Attached Storage, English abbreviation: NAS), open system direct-attached storage (English full name: Direct Access Storage English abbreviation: DAS), storage area network (English full name: Storage Area Network, English abbreviation: SAN) and so on. The hardware medium of the storage node can be isomorphic or heterogeneous. For ease of explanation, the embodiment of the present invention is described by taking NAS storage as an example, and the example is not limited to the solution of the present invention.
参照图2所示,图2为本发明实施例中挂载文件系统的方法的一个实施例,其中,所述文件系统用于管理存储系统中存储的文件,该方法具体为:Referring to FIG. 2, FIG. 2 is an embodiment of a method for mounting a file system according to an embodiment of the present invention, where the file system is used to manage files stored in a storage system, and the method is specifically:
101、客户端获取多个目标存储节点的标识,所述存储系统包括多个存储节点。101. The client acquires identifiers of multiple target storage nodes, where the storage system includes multiple storage nodes.
为了提高客户端与存储系统之间的数据传输速率,在本发明实施例中,客户端需要与存储系统中的多个存储节点建立多个通信连接。而在建立通信连接之前,客户端需要获取多个目标存储节点的标识,该标识可以为存储节点的IP地址。具体的,客户端获取存储节点的IP地址可以为:客户端先向域名系统(英文全称:Domain Name System,英文缩写:DNS)服务器发起存储系统发起浮动IP查询,该浮动IP为可以跟多个物理IP进行匹配的虚拟IP,其中,每一个浮动IP对应一个存储节点。DNS服务器返回包含存储系统中浮动IP地址的列表。需要说明的是,目标存储节点的标识可以为存储节点的唯一识别标识,并不仅限于IP地址,还可以使存储节点的序号,或者存储节点的节点名称等。DNS服务器属于该存储系统,负责该存储系统的域名解析。In order to improve the data transmission rate between the client and the storage system, in the embodiment of the present invention, the client needs to establish multiple communication connections with multiple storage nodes in the storage system. Before establishing a communication connection, the client needs to obtain the identifiers of the plurality of target storage nodes, and the identifier may be the IP address of the storage node. Specifically, the client obtains the IP address of the storage node, and the client initiates a floating IP query to the storage system by using the domain name system (English name: Domain Name System, English abbreviation: DNS). The floating IP address can be multiple. The physical IP is matched with a virtual IP, where each floating IP corresponds to one storage node. The DNS server returns a list containing the floating IP addresses in the storage system. It should be noted that the identifier of the target storage node may be a unique identifier of the storage node, and is not limited to an IP address, and may also be a serial number of the storage node or a node name of the storage node. The DNS server belongs to the storage system and is responsible for domain name resolution of the storage system.
可选的,客户端可以获取存储系统中的所有存储节点的标识,与所有存储节点均建立通信连接。或者,客户端也可以获取存储系统中的部分存储节点的标识,与部分存储节点建立通信连接。在与部分存储节点建立通信连接的方式中,客户端可以根据所有存储节点的连接负载情况进行选择,优先选择连接负载较少的存储节点,并与之建立通信连接。Optionally, the client can obtain the identifiers of all the storage nodes in the storage system, and establish a communication connection with all the storage nodes. Alternatively, the client may also obtain the identifier of a part of the storage nodes in the storage system, and establish a communication connection with some storage nodes. In a manner of establishing a communication connection with a part of the storage nodes, the client can select according to the connection load condition of all storage nodes, preferentially select a storage node with less connection load, and establish a communication connection with the storage node.
在另一种可能的实现方式中,客户端还可以通过待获取的目标文件所在的位置来选择存储节点。具体为,客户端先从存储系统包括的存储节点中,确定存储有目标文件中任一数据块的存储节点作为目标存储节点。在存储系统中,单个存储节点一般只存储有文件的数据块,极少存储有整个文件,因此,当客户端需要获取某个完整的目标文件时,则需要该客户端所连接的存储节点从其它存储节点中获取该目标文件的其它数据块,从而使得客户端从所连接的存储节点获取该整个目标文件。因此,在该实现方式中,客户端优先选择建立连接的存储节点存储有该客户端需要获取的目标文件的数据块,比如存储节点包括第一节点、第二节点以及第三节点,目标文件被分为两部分数据块分别存储于第一节点与第二节点中。而客户端在与存储节点建立通信连接之前,优先选择存储有目标文件数据块的第一节点与第二节点,从而使得客户端在获取目标文件时能够直接从第一节点与第二节点中获取目标文件,减少了客户端所连接的存储节点从其它存储节点获取目标文件数据块的时间,从而提高了客户端与存储系统之间的数据传输效率。In another possible implementation manner, the client may also select a storage node by using a location of the target file to be acquired. Specifically, the client first determines, from the storage node included in the storage system, a storage node that stores any data block in the target file as the target storage node. In a storage system, a single storage node generally stores only data blocks of files, and rarely stores the entire file. Therefore, when the client needs to obtain a complete target file, the storage node to which the client is connected is required. Other data blocks of the target file are obtained from other storage nodes, so that the client obtains the entire target file from the connected storage node. Therefore, in this implementation manner, the client preferentially selects the storage node that establishes the connection to store the data block of the target file that the client needs to acquire, for example, the storage node includes the first node, the second node, and the third node, and the target file is The two partial data blocks are respectively stored in the first node and the second node. Before the client establishes a communication connection with the storage node, the first node and the second node storing the target file data block are preferentially selected, so that the client can directly obtain the target file from the first node and the second node. The target file reduces the time for the storage node connected to the client to obtain the target file data block from other storage nodes, thereby improving the data transmission efficiency between the client and the storage system.
102、客户端根据所述多个目标存储节点的标识,建立与所述多个目标存储节点的多个通信连接。102. The client establishes multiple communication connections with the multiple target storage nodes according to the identifiers of the multiple target storage nodes.
客户端在获取了多个目标存储节点的标识后,即上述的目标存储节点的浮动IP地址后,客户端则能够与多个目标存储节点建立通信连接,具体建立通信连接的方式为:客户端根据浮动IP地址的列表向任意一个浮动IP(比如第一个)所对应的存储节点请求挂载连接,例如请求的方式可以是发送文件系统挂载的远程调用请求,例如MOUNT RPC调用请求;第一个浮动IP所对应的存储节点则会通过挂载模块(例如mountd模块)返回文件系统根目录的入口句柄,例如ROOT FILEHANDLE;客户端遍历浮动IP地址列表,分别与多个对应的存储节点的网络文件系统(英文全称:Network File System,英文缩写:NFS)服务模块(例如NFS Server)建立通信连接。该通信连接可以为套接字(Socket)连接,客户端在建立的所有Socket连接后,还可以将所有Socket连接的连接状态进行存 储,每个存储节点的Socket连接状态包括该存储节点的Socket连接状态为正常或异常、以及该存储节点的Socket连接的负载情况等信息。在客户端与多个目标存储节点建立通信连接,即Socket连接后,则可以进行实例化,使用该多个通信连接挂载文件系统。After the client obtains the identifiers of the plurality of target storage nodes, that is, the floating IP address of the target storage node, the client can establish a communication connection with the plurality of target storage nodes, and the specific way of establishing the communication connection is: the client According to the list of floating IP addresses, the storage node corresponding to any floating IP (such as the first one) requests a mount connection, for example, the request may be a remote call request sent by the file system, such as a MOUNT RPC call request; The storage node corresponding to a floating IP will return the entry handle of the file system root directory through the mount module (for example, the mountd module), for example, ROOT FILEHANDLE; the client traverses the floating IP address list, and respectively corresponds to multiple corresponding storage nodes. The network file system (English name: Network File System, English abbreviation: NFS) service module (such as NFS Server) establishes a communication connection. The communication connection can be a socket connection, and after the client establishes all the Socket connections, the connection state of all the Socket connections can also be saved. The Socket connection status of each storage node includes information such as the normal or abnormal state of the Socket connection state of the storage node and the load status of the Socket connection of the storage node. After the client establishes a communication connection with multiple target storage nodes, that is, after the Socket connection, it can be instantiated, and the file system is mounted by using the multiple communication connections.
在上述建立通信连接的方式中,客户端为通过所获取到的多个存储节点的浮动IP地址,实时获取该多个存储节点的入口句柄,从而建立通信连接。而在另一种实现方式中,客户端在确定了多个目标存储节点的浮动IP地址后,还可以从预先存储的连接索引库中确定该多个目标存储节点的连接索引,再根据该多个连接索引与多个存储节点建立通信连接。In the above manner of establishing a communication connection, the client acquires an entry handle of the plurality of storage nodes in real time through the obtained floating IP addresses of the plurality of storage nodes, thereby establishing a communication connection. In another implementation manner, after determining the floating IP address of the multiple target storage nodes, the client may further determine a connection index of the multiple target storage nodes from the pre-stored connection index database, and then according to the multiple The connection index establishes a communication connection with a plurality of storage nodes.
由于每个存储节点的入口句柄以及浮动IP地址一般是不会更改的,因此客户端在首次获取到存储节点的入口句柄以及该存储节点的浮动IP地址后,则将与该客户端建立通信连接,并将该通信连接的连接索引,即入口句柄以及该存储节点的浮动IP地址保存到连接索引库中,以便下次客户端再需要与该存储节点建立通信连接时,只需将客户端所获取到的浮动IP地址与连接索引库中的IP地址进行匹配,则能找到所对应的连接索引,从而建立通信连接。无需客户端再与存储节点进行发送远程调用请求和返回入口句柄等一系列建立通信连接的准备操作,提高了客户端与存储节点建立通信连接的效率。需要说明的是,若客户端当前所需要连接的存储节点为第一次连接,那么在连接索引库中则没有保存该存储节点的连接索引,从而客户端需要通过上述实时建立通信连接的步骤与该存储节点建立通信连接,在建立通信连接后,将该存储节点的连接索引保存于该连接索引库中。Since the entry handle and the floating IP address of each storage node are generally not changed, the client will establish a communication connection with the client after acquiring the entry handle of the storage node and the floating IP address of the storage node for the first time. And storing the connection index of the communication connection, that is, the entry handle and the floating IP address of the storage node, into the connection index library, so that the next time the client needs to establish a communication connection with the storage node, the client only needs to If the obtained floating IP address matches the IP address in the connection index database, the corresponding connection index can be found, thereby establishing a communication connection. The client does not need to send a remote call request and return an entry handle to the storage node to complete a preparation operation for establishing a communication connection, thereby improving the efficiency of establishing a communication connection between the client and the storage node. It should be noted that if the storage node that the client currently needs to connect is the first connection, the connection index of the storage node is not saved in the connection index library, so that the client needs to establish a communication connection through the above-mentioned real-time. The storage node establishes a communication connection, and after establishing the communication connection, saves the connection index of the storage node in the connection index library.
103、客户端使用所述多个通信连接挂载文件系统。103. The client mounts the file system by using the multiple communication connections.
客户端在于多个目标存储节点建立通信连接后,则能够进行实例化本地挂载点,即创建好文件系统本地挂载点。挂载点实际上就是Linux操作系统中的磁盘文件系统的入口目录,类似于Windows操作系统中的用来访问不同分区的C:、D:、E:等盘符。客户端在实例化挂载点后,则可以访问所连接的存储节点,从而获取目标文件。After the client establishes a communication connection between multiple target storage nodes, it can instantiate the local mount point, that is, create a file system local mount point. The mount point is actually the entry directory of the disk file system in the Linux operating system, similar to the C:, D:, E: and other drive letters used to access different partitions in the Windows operating system. After the client instantiates the mount point, it can access the connected storage node to obtain the target file.
本发明实施例中,客户端获取多个目标存储节点的标识,所述存储系统包括所述多个存储节点,客户端根据所述多个目标存储节点的标识,建立与所述多个目标存储节点的多个通信连接,客户端使用所述多个通信连接挂载文件系统。这样,单个客户端所建立的挂载点中,使用了与多个存储节点所建立的多条通信连接,使得客户端能够并行访问多个存储节点,从而提高客户端与存储系统之间的传输速率,同时也提高了高带宽要求业务的运行稳定性。In the embodiment of the present invention, the client obtains the identifiers of the plurality of target storage nodes, the storage system includes the plurality of storage nodes, and the client establishes the plurality of target storages according to the identifiers of the plurality of target storage nodes. A plurality of communication connections of the node, the client using the plurality of communication connections to mount the file system. In this way, among the mount points established by a single client, multiple communication connections established with multiple storage nodes are used, so that the client can access multiple storage nodes in parallel, thereby improving the transmission between the client and the storage system. The rate also increases the operational stability of the high bandwidth requirements of the service.
结合图2实施例,在一个可选实施例中,该挂载文件系统的方法还包括:With reference to the embodiment of FIG. 2, in an optional embodiment, the method for mounting a file system further includes:
在客户端挂载所述文件系统后,客户端从所述多个通信连接中确定至少一个通信连接;After the client mounts the file system, the client determines at least one communication connection from the plurality of communication connections;
客户端基于所述至少一个通信连接访问所对应的目标存储节点来获取至少一个目标文件。The client accesses the corresponding target storage node based on the at least one communication connection to acquire at least one object file.
当客户端与多个存储节点建立多个通信连接,并通过这些多个通信连接挂载文件系统之后,客户端则能够根据这些通信连接访问多个存储节点,从而获取存储节点中的目标文件。可选的,客户端可以从多个通信连接中选择一条或多条通信连接,从而访问一个或多个存储节点,进而获取一个或多个目标文件。客户端获取目标文件的方式可以通过按文件为粒度进行传输,也可以是以目标文件存储于存储节点的数据块为粒度进行传 输。比如,客户端可以选择一条通信连接访问一个存储节点,若该存储节点只存储有目标文件的一部分数据块或未存储有目标文件的数据块,则该存储节点则根据目标文件的标识,比如文件名从搜索其它存储节点,从而获取该目标文件的所有数据块,从而将该整个目标文件的所有数据块发送给客户端。或者,多个存储节点中分别存储有目标文件的不同的数据块,那么客户端可以使用多个通信连接同时访问多个存储节点,从不同的存储节点获取到目标文件的不同数据块,客户端再重新整合不同的数据块,从而生成完整的目标文件。After the client establishes multiple communication connections with multiple storage nodes and mounts the file system through the multiple communication connections, the client can access multiple storage nodes according to the communication connections to obtain the target files in the storage node. Optionally, the client may select one or more communication connections from the plurality of communication connections to access one or more storage nodes to obtain one or more object files. The manner in which the client obtains the target file may be transmitted by using the file as the granularity, or may be transmitted by using the data block of the target file stored in the storage node as a granularity. lose. For example, the client may select a communication connection to access a storage node. If the storage node only stores a part of the data block of the target file or a data block that does not store the target file, the storage node is based on the identifier of the target file, such as a file. The name searches for other storage nodes to obtain all the data blocks of the target file, thereby transmitting all the data blocks of the entire target file to the client. Alternatively, if a plurality of storage nodes respectively store different data blocks of the target file, the client may access multiple storage nodes simultaneously by using multiple communication connections, and obtain different data blocks of the target file from different storage nodes, and the client Re-integrate different data blocks to generate a complete object file.
可选的,客户端还可以基于所述多个通信连接并行访问所对应的目标存储节点来获取多个目标文件。Optionally, the client may further acquire the multiple target files by accessing the corresponding target storage nodes in parallel according to the multiple communication connections.
若客户端与存储系统之间以文件为粒度进行传输,那么客户端在同时获取多个目标文件时,则可以通过多个通信连接并行访问多个目标存储节点来获取多个目标文件。比如客户端可以并行访问三个存储节点从而获取三个目标文件。客户端获取目标文件可以包括客户端对目标文件进行读操作或者写操作等业务操作。If the client and the storage system transmit at a file-level granularity, when the client acquires multiple target files at the same time, the client can access multiple target storage nodes in parallel through multiple communication connections to obtain multiple target files. For example, the client can access three storage nodes in parallel to obtain three target files. The client obtains the target file, which may include a business operation such as a read operation or a write operation by the client on the target file.
可选的,上述客户端从所述多个通信连接中确定所述至少一个通信连接,具体可以为:Optionally, the client determines the at least one communication connection from the multiple communication connections, and specifically:
客户端从所述多个通信连接中确定通信负载小的至少一个通信连接。The client determines at least one communication connection having a small communication load from the plurality of communication connections.
上述实施例中说明了客户端在存储节点建立通信连接后,则存储了存储节点的连接状态,该连接状态中包含了存储节点的负载情况。基于负载均衡原则,客户端在选择存储节点进行数据传输时,可以优先选择当前通信负载教小的通信连接的存储节点进行数据传输。具体可以为:客户端从所述多个目标存储节点中,确定通信连接数少的第一存储节点,再从所述多个通信连接中,确定与所述第一存储节点的通信连接作为通信负载小的通信连接。In the foregoing embodiment, after the client establishes a communication connection at the storage node, the connection state of the storage node is stored, and the connection state includes the load condition of the storage node. Based on the load balancing principle, when the client selects a storage node for data transmission, the client can preferentially select the current communication load to teach the storage node of the small communication connection to perform data transmission. Specifically, the client may determine, from the plurality of target storage nodes, a first storage node having a small number of communication connections, and then determine, from the plurality of communication connections, a communication connection with the first storage node as a communication. A small communication connection.
需要说明的是,当客户端需要使用两个以上的通信连接访问两个以上的存储节点时,那么客户端所选择的两个以上的通信连接均为通信负载小的通信连接。It should be noted that when the client needs to access more than two storage nodes by using more than two communication connections, then the two or more communication connections selected by the client are communication connections with small communication load.
可选的,当目标文件为至少两个时,客户端还可以先获取所述至少两个目标文件中每个文件的大小信息;Optionally, when the target file is at least two, the client may first obtain size information of each of the at least two target files.
客户端根据与存储系统的传输速率计算出所述每个文件传输所消耗的时间;The client calculates the time consumed by each file transfer according to the transmission rate of the storage system;
客户端根据所述每个文件传输所消耗的时间以文件为粒度计算出平均所述每个存储节点需传输的文件数目;The client calculates, according to the time consumed by each file transfer, the average number of files to be transmitted by each storage node according to the file granularity;
所述客户端通过所述文件数目并行访问多个存储节点来获取所述至少两个目标文件。The client acquires the at least two object files by accessing a plurality of storage nodes in parallel by the number of files.
比如客户端需要获取的目标文件为4个,客户端选择两个通信连接进行该4个文件的传输。客户端首先向存储系统发送该4个文件的读请求,存储系统则会根据该4个文件的文件标识获取到该4个文件的每个文件的大小信息,其中若第一个文件为10G,第二个文件为1G,第三个文件为3G,第四个文件为2G,那么按照以文件为粒度传输的方式,通过两个连接进行传输,若每条通信连接的传输速率均为1G/s,那么传输第一个文件需要花费10s,传输第二个文件需要花费1s,传输第三个文件需要花费3s,传输第四个文件需要花费2s。那么按照均衡的原则,可以通过一个通信线路来传输第一个文件,而另一条通信链路来传输另外三个文件。又或者,第四个文件的大小为8G,那么4个文件的总大小为22G,通过两条通信线路来传输,平均每条通信线路则可以传输两个文件, 分别是第一个文件与第二个文件在一条通信线路上进行传输,第二个文件与第三个文件在另一条通信线路上进行传输。这样,使得客户端与存储系统之间进行数据传输的数据大小负载更为均衡,提高了客户端与存储系统之间的传输效率。For example, the client needs to obtain four target files, and the client selects two communication connections for the transmission of the four files. The client first sends a read request for the four files to the storage system, and the storage system obtains the size information of each of the four files according to the file identifier of the four files, wherein if the first file is 10G, The second file is 1G, the third file is 3G, and the fourth file is 2G. Then, according to the file-size transmission method, the transmission is performed through two connections. If the transmission rate of each communication connection is 1G/ s, then it takes 10s to transfer the first file, 1s to transfer the second file, 3s to transfer the third file, and 2s to transfer the fourth file. Then, according to the principle of equalization, the first file can be transmitted through one communication line, and the other three files can be transmitted through another communication link. Or, the size of the fourth file is 8G, then the total size of the four files is 22G, which is transmitted through two communication lines, and each of the communication lines can transmit two files on average. The first file and the second file are transmitted on one communication line, and the second file and the third file are transmitted on another communication line. In this way, the data size load of data transmission between the client and the storage system is more balanced, and the transmission efficiency between the client and the storage system is improved.
可选的,为了提高XNFS多连接的安全性,以存储系统为NAS存储为例,还可以将NAS存储中的统一认证模块移植到客户端中,客户端和NAS存储分别固化一对相同的账户及密码,加密保存,然后客户端作为认证客户端,NAS存储作为认证服务端,使用上述统一认证机制完成双向认证,在双向认证通过后才能进行文件系统挂载和文件访问。Optionally, in order to improve the security of the XNFS multi-connection, taking the storage system as the NAS storage as an example, the unified authentication module in the NAS storage can also be transplanted to the client, and the client and the NAS storage respectively solidify a pair of identical accounts. And the password is encrypted and saved. Then the client acts as the authentication client. The NAS storage serves as the authentication server. The unified authentication mechanism is used to complete the two-way authentication. After the two-way authentication is passed, the file system mount and file access can be performed.
参照图3所示,本发明实施例中挂载文件系统的装置的一个实施例包括:Referring to FIG. 3, an embodiment of an apparatus for mounting a file system in an embodiment of the present invention includes:
获取单元201,用于获取多个目标存储节点的标识,所述存储系统包括多个存储节点;An obtaining unit 201, configured to acquire identifiers of multiple target storage nodes, where the storage system includes multiple storage nodes;
连接单元202,用于根据所述多个目标存储节点的标识,建立与所述多个目标存储节点的多个通信连接;The connecting unit 202 is configured to establish, according to the identifiers of the multiple target storage nodes, multiple communication connections with the multiple target storage nodes;
挂载单元203,用于使用所述多个通信连接挂载文件系统。The mounting unit 203 is configured to mount the file system by using the multiple communication connections.
可选的,所述装置包括:Optionally, the device includes:
确定单元204,用于在所述获取单元获取多个目标存储节点的标识之前,从所述存储系统包括的存储节点中,确定存储有目标文件中任一数据块的存储节点作为所述目标存储节点。a determining unit 204, configured to determine, from the storage node included in the storage system, a storage node that stores any data block in the target file as the target storage, before the obtaining unit acquires the identifiers of the plurality of target storage nodes node.
可选的,所述连接单元202具体用于:Optionally, the connecting unit 202 is specifically configured to:
根据所述目标存储节点的标识,从连接索引库中确定所述多个目标存储节点的连接索引;Determining, according to the identifier of the target storage node, a connection index of the plurality of target storage nodes from the connection index library;
根据所述多个目标存储节点的连接索引,建立与所述多个目标存储节点的多个通信连接。Establishing a plurality of communication connections with the plurality of target storage nodes according to a connection index of the plurality of target storage nodes.
可选的,所述确定单元204还用于,在所述挂载单元203挂载所述文件系统后,从所述多个通信连接中确定至少一个通信连接;Optionally, the determining unit 204 is further configured to: after the mounting unit 203 mounts the file system, determine at least one communication connection from the multiple communication connections;
所述获取单元201还用于,基于所述至少一个通信连接访问所对应的目标存储节点来获取至少一个目标文件。The obtaining unit 201 is further configured to acquire at least one target file based on the at least one communication connection accessing the corresponding target storage node.
可选的,所述确定单元204具体用于:Optionally, the determining unit 204 is specifically configured to:
从所述多个通信连接中确定通信负载小的至少一个通信连接。At least one communication connection having a small communication load is determined from the plurality of communication connections.
可选的,所述确定单元204具体用于:Optionally, the determining unit 204 is specifically configured to:
从所述多个目标存储节点中,确定通信连接数少的第一存储节点;Determining, from the plurality of target storage nodes, a first storage node having a small number of communication connections;
从所述多个通信连接中,确定与所述第一存储节点的通信连接作为通信负载小的通信连接。From the plurality of communication connections, a communication connection with the first storage node is determined as a communication connection with a small communication load.
可选的,所述获取单元201还用于:Optionally, the obtaining unit 201 is further configured to:
在挂载所述文件系统后,基于所述多个通信连接并行访问所对应的目标存储节点来获取多个目标文件。After the file system is mounted, the target storage nodes corresponding to the plurality of communication connections are accessed in parallel to obtain a plurality of target files.
关于图3实施例中各个单元的具体描述可以参照图2实施例以及图2的可选实施例中所提供的挂载文件系统的方法的详细描述,此处不做赘述。For a detailed description of the various units in the embodiment of FIG. 3, reference may be made to the detailed description of the method for mounting the file system provided in the embodiment of FIG. 2 and the optional embodiment of FIG. 2, and details are not described herein.
本发明实施例还提供一种存储系统,所述存储系统为上述实施例中描述的存储系统,用于与客户端配合来实现上述的挂载文件系统的方法。The embodiment of the present invention further provides a storage system, which is a storage system described in the foregoing embodiment, and is configured to cooperate with a client to implement the foregoing method for mounting a file system.
本发明实施例还提供一种存储节点,该存储节点属于上述实施例中描述的存储系统,该存储节点为上述实施例中描述的存储节点,该存储节点用于与客户端配合来实现上述 的挂载文件系统的方法。The embodiment of the present invention further provides a storage node, which belongs to the storage system described in the foregoing embodiment, and the storage node is a storage node described in the foregoing embodiment, and the storage node is configured to cooperate with a client to implement the foregoing. The method of mounting a file system.
参见图4所示,本发明实施例还提供了一种设备300,设备300包括至少一个处理器301、存储器302和通信接口303;所述至少一个处理器301、所述存储器302和所述通信接口303均通过总线304连接;Referring to FIG. 4, an embodiment of the present invention further provides an apparatus 300. The device 300 includes at least one processor 301, a memory 302, and a communication interface 303. The at least one processor 301, the memory 302, and the communication The interfaces 303 are all connected by a bus 304;
所述存储器302,用于存储计算机执行指令;The memory 302 is configured to store a computer execution instruction;
所述至少一个处理器301,用于执行所述存储器302存储的计算机执行指令,使得所述设备300通过所述通信接口303与存储系统进行数据交互来执行上述方法实施例提供的挂载文件系统的方法,或者使得所述设备300通过所述通信接口303与存储系统进行数据交互来实现挂载文件系统的装置的部分或者全部功能。The at least one processor 301 is configured to execute a computer-executed instruction stored by the memory 302, so that the device 300 performs data interaction with the storage system through the communication interface 303 to execute the mounted file system provided by the foregoing method embodiment. The method, or causing the device 300 to perform data interaction with the storage system through the communication interface 303 to implement some or all of the functions of the device that mounts the file system.
至少一个处理器301,可以包括不同类型的处理器301,或者包括相同类型的处理器301;处理器301可以是以下的任一种:中央处理器(Central Processing Unit,简称CPU)、ARM处理器、现场可编程门阵列(Field Programmable Gate Array,简称FPGA)、专用处理器等具有计算处理能力的器件。一种可选实施方式,所述至少一个处理器301还可以集成为众核处理器。The at least one processor 301 may include different types of processors 301, or include the same type of processor 301; the processor 301 may be any one of the following: a central processing unit (CPU), an ARM processor. , Field Programmable Gate Array (FPGA), dedicated processor and other devices with computational processing capabilities. In an optional implementation manner, the at least one processor 301 may also be integrated into a many-core processor.
存储器302可以是以下的任一种或任一种组合:随机存取存储器(Random Access Memory,简称RAM)、只读存储器(read only memory,简称ROM)、非易失性存储器(non-volatile memory,简称NVM)、固态硬盘(Solid State Drives,简称SSD)、机械硬盘、磁盘、磁盘整列等存储介质。The memory 302 may be any one or any combination of the following: a random access memory (RAM), a read only memory (ROM), a non-volatile memory (non-volatile memory). , referred to as NVM), Solid State Drives (SSD), mechanical hard disks, disks, disk arrays and other storage media.
通信接口303用于设备300与其他设备(例如存储系统中的存储节点)进行数据交互。通信接口303可以是以下的任一种或任一种组合:网络接口(例如以太网接口)、无线网卡等具有网络接入功能的器件。 Communication interface 303 is used by device 300 for data interaction with other devices, such as storage nodes in a storage system. The communication interface 303 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, or the like having a network access function.
该总线304可以包括地址总线、数据总线、控制总线等,为便于表示,图4用一条粗线表示该总线。总线304可以是以下的任一种或任一种组合:工业标准体系结构(Industry Standard Architecture,简称ISA)总线、外设组件互连标准(Peripheral Component Interconnect,简称PCI)总线、扩展工业标准结构(Extended Industry Standard Architecture,简称EISA)总线等有线数据传输的器件。The bus 304 can include an address bus, a data bus, a control bus, etc., for convenience of representation, Figure 4 shows the bus with a thick line. The bus 304 may be any one or any combination of the following: an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, and an extended industry standard structure ( Extended Industry Standard Architecture (EISA) bus and other devices for wired data transmission.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统,装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
在本申请所提供的几个实施例中,应该理解到,所揭露的系统,装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外,在本发明各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单 元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated order The element can be implemented in the form of hardware or in the form of a software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .
以上所述,以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。 The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the embodiments are modified, or the equivalents of the technical features are replaced by the equivalents of the technical solutions of the embodiments of the present invention.

Claims (15)

  1. 一种挂载文件系统的方法,其特征在于,所述文件系统用于管理存储系统中存储的文件;所述方法包括:A method for mounting a file system, wherein the file system is configured to manage files stored in a storage system; the method includes:
    获取多个目标存储节点的标识,所述存储系统包括多个存储节点;Obtaining identifiers of a plurality of target storage nodes, where the storage system includes a plurality of storage nodes;
    根据所述多个目标存储节点的标识,建立与所述多个目标存储节点的多个通信连接;Establishing a plurality of communication connections with the plurality of target storage nodes according to the identifiers of the plurality of target storage nodes;
    使用所述多个通信连接挂载文件系统。Mount the file system using the plurality of communication connections.
  2. 根据权利要求1所述的方法,其特征在于,在所述获取多个目标存储节点的标识之前,所述方法包括:The method according to claim 1, wherein before the obtaining the identifiers of the plurality of target storage nodes, the method comprises:
    从所述存储系统包括的存储节点中,确定存储有目标文件中任一数据块的存储节点作为所述目标存储节点。From a storage node included in the storage system, a storage node storing any one of the target files is determined as the target storage node.
  3. 根据权利要求1或2所述的方法,其特征在于,所述根据所述多个目标存储节点的标识,建立与所述多个目标存储节点的多个通信连接包括:The method according to claim 1 or 2, wherein the establishing a plurality of communication connections with the plurality of target storage nodes according to the identifiers of the plurality of target storage nodes comprises:
    根据所述目标存储节点的标识,从连接索引库中确定所述多个目标存储节点的连接索引;Determining, according to the identifier of the target storage node, a connection index of the plurality of target storage nodes from the connection index library;
    根据所述多个目标存储节点的连接索引,建立与所述多个目标存储节点的多个通信连接。Establishing a plurality of communication connections with the plurality of target storage nodes according to a connection index of the plurality of target storage nodes.
  4. 根据权利要求1至3任一项所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 1 to 3, wherein the method further comprises:
    在挂载所述文件系统后,从所述多个通信连接中确定至少一个通信连接;After the file system is mounted, determining at least one communication connection from the plurality of communication connections;
    基于所述至少一个通信连接访问所对应的目标存储节点来获取至少一个目标文件。Acquiring at least one object file based on the at least one communication connection accessing the corresponding target storage node.
  5. 根据权利要求4所述的方法,其特征在于,所述从所述多个通信连接中确定所述至少一个通信连接,具体为:The method according to claim 4, wherein the determining the at least one communication connection from the plurality of communication connections is specifically:
    从所述多个通信连接中确定通信负载小的至少一个通信连接。At least one communication connection having a small communication load is determined from the plurality of communication connections.
  6. 根据权利要求5所述的方法,其特征在于,所述从所述多个通信连接中确定通信负载小的至少一个通信连接,具体为:The method according to claim 5, wherein the determining at least one communication connection having a small communication load from the plurality of communication connections is specifically:
    从所述多个目标存储节点中,确定通信连接数少的第一存储节点;Determining, from the plurality of target storage nodes, a first storage node having a small number of communication connections;
    从所述多个通信连接中,确定与所述第一存储节点的通信连接作为通信负载小的通信连接。From the plurality of communication connections, a communication connection with the first storage node is determined as a communication connection with a small communication load.
  7. 根据权利要求1至3任一项所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 1 to 3, wherein the method further comprises:
    在挂载所述文件系统后,基于所述多个通信连接并行访问所对应的目标存储节点来获取多个目标文件。After the file system is mounted, the target storage nodes corresponding to the plurality of communication connections are accessed in parallel to obtain a plurality of target files.
  8. 一种挂载文件系统的装置,其特征在于,所述文件系统用于管理存储系统中存储的文件;所述装置包括:An apparatus for mounting a file system, wherein the file system is configured to manage files stored in a storage system; the apparatus includes:
    获取单元,用于获取多个目标存储节点的标识,所述存储系统包括多个存储节点;An obtaining unit, configured to acquire identifiers of multiple target storage nodes, where the storage system includes multiple storage nodes;
    连接单元,用于根据所述多个目标存储节点的标识,建立与所述多个目标存储节点的多个通信连接;a connecting unit, configured to establish, according to the identifiers of the multiple target storage nodes, multiple communication connections with the multiple target storage nodes;
    挂载单元,用于使用所述多个通信连接挂载文件系统。a mounting unit for mounting a file system using the plurality of communication connections.
  9. 根据权利要求8所述的装置,其特征在于,所述装置包括:The device of claim 8 wherein said device comprises:
    确定单元,用于在所述获取单元获取多个目标存储节点的标识之前,从所述存储系统包括的存储节点中,确定存储有目标文件中任一数据块的存储节点作为所述目标存储节点。 a determining unit, configured to determine, from the storage node included in the storage system, a storage node storing any data block in the target file as the target storage node, before the obtaining unit acquires the identifiers of the plurality of target storage nodes .
  10. 根据权利要求8或9所述的装置,其特征在于,所述连接单元具体用于:The device according to claim 8 or 9, wherein the connecting unit is specifically configured to:
    根据所述目标存储节点的标识,从连接索引库中确定所述多个目标存储节点的连接索引;Determining, according to the identifier of the target storage node, a connection index of the plurality of target storage nodes from the connection index library;
    根据所述多个目标存储节点的连接索引,建立与所述多个目标存储节点的多个通信连接。Establishing a plurality of communication connections with the plurality of target storage nodes according to a connection index of the plurality of target storage nodes.
  11. 根据权利要求8至10任一项所述的装置,其特征在于,A device according to any one of claims 8 to 10, characterized in that
    所述确定单元还用于,在所述挂载单元挂载所述文件系统后,从所述多个通信连接中确定至少一个通信连接;The determining unit is further configured to: after the mounting unit mounts the file system, determine at least one communication connection from the plurality of communication connections;
    所述获取单元还用于,基于所述至少一个通信连接访问所对应的目标存储节点来获取至少一个目标文件。The obtaining unit is further configured to acquire at least one object file based on the at least one communication connection accessing the corresponding target storage node.
  12. 根据权利要求11所述的装置,其特征在于,所述确定单元具体用于:The device according to claim 11, wherein the determining unit is specifically configured to:
    从所述多个通信连接中确定通信负载小的至少一个通信连接。At least one communication connection having a small communication load is determined from the plurality of communication connections.
  13. 根据权利要求12所述的装置,其特征在于,所述确定单元具体用于:The device according to claim 12, wherein the determining unit is specifically configured to:
    从所述多个目标存储节点中,确定通信连接数少的第一存储节点;Determining, from the plurality of target storage nodes, a first storage node having a small number of communication connections;
    从所述多个通信连接中,确定与所述第一存储节点的通信连接作为通信负载小的通信连接。From the plurality of communication connections, a communication connection with the first storage node is determined as a communication connection with a small communication load.
  14. 根据权利要求8至10任一项所述的装置,其特征在于,所述获取单元还用于:The device according to any one of claims 8 to 10, wherein the obtaining unit is further configured to:
    在挂载所述文件系统后,基于所述多个通信连接并行访问所对应的目标存储节点来获取多个目标文件。After the file system is mounted, the target storage nodes corresponding to the plurality of communication connections are accessed in parallel to obtain a plurality of target files.
  15. 一种设备,其特征在于,包括至少一个处理器、存储器和通信接口;所述至少一个处理器、所述存储器和所述通信接口均通过总线连接;An apparatus, comprising: at least one processor, a memory, and a communication interface; the at least one processor, the memory, and the communication interface are each connected by a bus;
    所述存储器,用于存储计算机执行指令;The memory is configured to store a computer execution instruction;
    所述至少一个处理器,用于执行所述存储器存储的计算机执行指令,使得所述设备通过所述通信接口与存储系统进行数据交互来执行权利要求1至7任一项所述的挂载文件系统的方法。 The at least one processor, configured to execute the computer-executed instructions stored in the memory, such that the device performs data interaction with the storage system through the communication interface to execute the mount file according to any one of claims 1 to 7. Systematic approach.
PCT/CN2017/097681 2016-09-12 2017-08-16 File system mounting method, apparatus and equipment WO2018045860A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610819478.6A CN106484322A (en) 2016-09-12 2016-09-12 A kind of method of carry file system, device and equipment
CN201610819478.6 2016-09-12

Publications (1)

Publication Number Publication Date
WO2018045860A1 true WO2018045860A1 (en) 2018-03-15

Family

ID=58273487

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/097681 WO2018045860A1 (en) 2016-09-12 2017-08-16 File system mounting method, apparatus and equipment

Country Status (2)

Country Link
CN (1) CN106484322A (en)
WO (1) WO2018045860A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110688073A (en) * 2019-10-10 2020-01-14 深圳市网心科技有限公司 Data storage method, electronic device, system and medium
CN112291292A (en) * 2019-07-26 2021-01-29 浙江宇视科技有限公司 Data storage method, device, equipment and medium

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106484322A (en) * 2016-09-12 2017-03-08 华为技术有限公司 A kind of method of carry file system, device and equipment
CN109460182B (en) * 2017-09-06 2020-10-30 杭州海康威视系统技术有限公司 Data storage and reading method and device
CN110045929B (en) * 2019-04-19 2020-06-19 苏州浪潮智能科技有限公司 Mounting point management method and related device
CN111078658B (en) * 2019-10-31 2022-08-12 苏州浪潮智能科技有限公司 Method and device for mounting cluster file system
CN111859414A (en) * 2020-06-18 2020-10-30 厦门亿联网络技术股份有限公司 Mounting method and device of file system and storage medium
CN113835613B (en) * 2020-06-24 2024-03-29 浙江宇视科技有限公司 File reading method and device, electronic equipment and storage medium
CN111988409B (en) * 2020-08-20 2023-06-13 深圳市臂云科技有限公司 Method and system for realizing cloud mobile phone mounted external storage starting
CN112492011A (en) * 2020-11-19 2021-03-12 苏州浪潮智能科技有限公司 Distributed storage system fault switching method, system, terminal and storage medium
CN113342281A (en) * 2021-06-28 2021-09-03 深圳市瑞驰信息技术有限公司 Method for realizing mounting in android system based on rbd storage
CN114124651B (en) * 2022-01-26 2022-05-10 苏州浪潮智能科技有限公司 Control method and device for cloud platform management node and cloud platform server cluster

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101567003A (en) * 2009-05-27 2009-10-28 清华大学 Method for managing and allocating resource in parallel file system
CN101699825A (en) * 2009-10-28 2010-04-28 成都市华为赛门铁克科技有限公司 Network file system access method, device and system
US20140222879A1 (en) * 2013-02-01 2014-08-07 Apple Inc. Method and system for unmounting an inaccessible network file system
CN104050248A (en) * 2014-06-05 2014-09-17 北京捷成世纪科技股份有限公司 File storage system and storage method
CN105677703A (en) * 2015-12-25 2016-06-15 曙光云计算技术有限公司 NAS file system, and access method and apparatus thereof
CN106484322A (en) * 2016-09-12 2017-03-08 华为技术有限公司 A kind of method of carry file system, device and equipment

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7509645B2 (en) * 2002-10-17 2009-03-24 Intel Corporation Methods and apparatus for load balancing storage nodes in a distributed network attached storage system
CN101510223B (en) * 2009-04-03 2012-04-25 成都市华为赛门铁克科技有限公司 Data processing method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101567003A (en) * 2009-05-27 2009-10-28 清华大学 Method for managing and allocating resource in parallel file system
CN101699825A (en) * 2009-10-28 2010-04-28 成都市华为赛门铁克科技有限公司 Network file system access method, device and system
US20140222879A1 (en) * 2013-02-01 2014-08-07 Apple Inc. Method and system for unmounting an inaccessible network file system
CN104050248A (en) * 2014-06-05 2014-09-17 北京捷成世纪科技股份有限公司 File storage system and storage method
CN105677703A (en) * 2015-12-25 2016-06-15 曙光云计算技术有限公司 NAS file system, and access method and apparatus thereof
CN106484322A (en) * 2016-09-12 2017-03-08 华为技术有限公司 A kind of method of carry file system, device and equipment

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112291292A (en) * 2019-07-26 2021-01-29 浙江宇视科技有限公司 Data storage method, device, equipment and medium
CN112291292B (en) * 2019-07-26 2023-04-07 浙江宇视科技有限公司 Data storage method, device, equipment and medium
CN110688073A (en) * 2019-10-10 2020-01-14 深圳市网心科技有限公司 Data storage method, electronic device, system and medium
CN110688073B (en) * 2019-10-10 2023-07-14 深圳市网心科技有限公司 Data storage method, electronic device, system and medium

Also Published As

Publication number Publication date
CN106484322A (en) 2017-03-08

Similar Documents

Publication Publication Date Title
WO2018045860A1 (en) File system mounting method, apparatus and equipment
US9507524B1 (en) In-band management using an intelligent adapter and methods thereof
US9720598B2 (en) Storage array having multiple controllers
US10216666B2 (en) Caching methods and systems using a network interface card
JP5026283B2 (en) Collaborative shared storage architecture
US20210075665A1 (en) Implementing switchover operations between computing nodes
US11921597B2 (en) Cross-platform replication
US20070088917A1 (en) System and method for creating and maintaining a logical serial attached SCSI communication channel among a plurality of storage systems
US9256377B2 (en) Consistent distributed storage communication protocol semantics in a clustered storage system
US9983813B2 (en) Maintenance of a fabric priority during synchronous copy operations
US10778771B2 (en) Methods and systems for managing networked storage system resources
US20160077752A1 (en) Fibre Channel Storage Array Methods for Handling Cache-Consistency Among Controllers of an Array and Consistency Among Arrays of a Pool
US20140281140A1 (en) Network storage system using flash storage
JP2007535763A (en) Online initial mirror synchronization and mirror synchronization verification in storage area networks
WO2017162175A1 (en) Data transmission method and device
US11379405B2 (en) Internet small computer interface systems extension for remote direct memory access (RDMA) for distributed hyper-converged storage systems
US20040162921A1 (en) SCSI enclosure services
US9172586B1 (en) Method and system for writing tag and data
US10708379B1 (en) Dynamic proxy for databases
US8554867B1 (en) Efficient data access in clustered storage system
WO2016101856A1 (en) Data access method and apparatus
US20200244599A1 (en) Fiber channel fabric login/logout system
US9477414B1 (en) Methods and systems for improved caching with data recovery
US9460017B1 (en) Methods and systems for efficient cache mirroring
US10798159B2 (en) Methods for managing workload throughput in a storage system and devices thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17848033

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17848033

Country of ref document: EP

Kind code of ref document: A1