WO2017088664A1 - Procédé et appareil de traitement de données destinés à un système de fichiers en grappes - Google Patents

Procédé et appareil de traitement de données destinés à un système de fichiers en grappes Download PDF

Info

Publication number
WO2017088664A1
WO2017088664A1 PCT/CN2016/105219 CN2016105219W WO2017088664A1 WO 2017088664 A1 WO2017088664 A1 WO 2017088664A1 CN 2016105219 W CN2016105219 W CN 2016105219W WO 2017088664 A1 WO2017088664 A1 WO 2017088664A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage node
request
read
data
file system
Prior art date
Application number
PCT/CN2016/105219
Other languages
English (en)
Chinese (zh)
Inventor
张勤
李璐
Original Assignee
深圳市中博科创信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市中博科创信息技术有限公司 filed Critical 深圳市中博科创信息技术有限公司
Publication of WO2017088664A1 publication Critical patent/WO2017088664A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus for a cluster file system.
  • cluster file systems have become a new trend in the development of computer technology.
  • the cluster file system provides a large-capacity file system shared storage application, such as CIFS, by concatenating a plurality of single independent hosts into a systematic whole and relying on the storage area network. Multiple concurrent user operations and big data transfers within the storage area network are implemented.
  • cluster nodes also increases the likelihood of node downtime or service failure. For example, when the client reads and writes a large file to the cluster, if the service node that accepts the read/write request fails, even if the service can switch to other nodes in time, since there is no data cache in the memory of other nodes, then the client The machine's read and write operations will still be interrupted.
  • the main object of the present invention is to provide a data processing method and apparatus for a cluster file system, which aims to solve the technical problem of interrupting the read and write operations of the client due to the failure of the cluster service node.
  • the present invention provides a data processing apparatus for a cluster file system, and the data processing apparatus of the cluster file system includes:
  • a storage module configured to: when the storage node receives the file read or write request, and the storage node where the storage module is located is the primary storage node, storing the request information in the read or write request into the cache Area;
  • a synchronization module configured to synchronize the request information to a slave storage node in the cluster file system
  • a data processing module configured to read data in the storage area according to the read request and the request information in the buffer area, or to the storage area according to the write request and the request information in the buffer area Write data in .
  • the present invention further provides a data processing method for a cluster file system, where the data processing method of the cluster file system includes:
  • the storage node determines whether data corresponding to the read or write request requires verification
  • the storage node acquires the verification information input by the user;
  • the storage node When the verification information matches the pre-stored verification information, the storage node reads data in its storage area according to the read request and the request information in the buffer area, or according to the write request and the The request information in the buffer area writes data to its storage area;
  • the storage node stores the request information in the read or write request into its buffer area
  • the storage node synchronizes the request information to a slave storage node in the cluster file system
  • the service request is not a read or write request, then responding to the service request.
  • the present invention further provides a data processing method for a cluster file system, where the data processing method of the cluster file system includes:
  • the storage node When the storage node receives a file read or write request, and the storage node is a primary storage node, the storage node reads the storage area according to the read request and the request information in the cache area. Data, or writing data to its storage area according to the write request and the request information in the buffer area;
  • the storage node stores the request information in the read or write request into its buffer area
  • the storage node synchronizes the request information to a slave storage node in the cluster file system.
  • the data processing method and apparatus of the cluster file system provided by the present invention, when the storage node receives a file read or write request, and the storage node is a primary storage node, the storage node according to the read request and the The request information in the buffer area reads data in its storage area, or writes data to its storage area according to the write request and the request information in the buffer area, and the storage node reads the data. Or the request information in the write request is stored in its cache area, and the storage node synchronizes the request information to the slave storage node in the cluster file system. After the primary storage node fails, the slave storage node may be in the cache area according to the cache node.
  • the service request information takes over the client's read or write process without interruption, ensuring the stability of the cluster file system service.
  • FIG. 1 is a schematic diagram of functional modules of a first embodiment of a data processing apparatus of a cluster file system according to the present invention
  • FIG. 2 is a schematic diagram of functional modules of a data processing apparatus of a cluster file system according to the present invention for processing other services such as reading and writing;
  • FIG. 3 is a schematic diagram of functional modules of a second embodiment of a data processing apparatus of a cluster file system according to the present invention.
  • FIG. 4 is a schematic diagram of a refinement function module of the detection module of FIG. 3;
  • FIG. 5 is a schematic diagram of functional modules of a third embodiment of a data processing apparatus of a cluster file system according to the present invention.
  • FIG. 6 is a schematic flowchart of a first embodiment of a data processing method of a cluster file system according to the present invention.
  • FIG. 7 is a schematic flowchart of a data processing method of a cluster file system according to the present invention for processing other services such as reading and writing data;
  • FIG. 8 is a schematic flowchart of a second embodiment of a data processing method of a cluster file system according to the present invention.
  • FIG. 9 is a schematic diagram of a refinement process of detecting a link connection state and an operation state of a primary storage node in FIG. 8;
  • FIG. 10 is a schematic flowchart diagram of a third embodiment of a data processing method of a cluster file system according to the present invention.
  • the present invention provides a data processing apparatus for a cluster file system.
  • FIG. 1 is a schematic diagram of functional modules of a first embodiment of a data processing apparatus of a cluster file system according to the present invention.
  • the data processing device of the cluster file system includes:
  • the data processing module 10 is configured to: when the storage node receives the file read or write request, read the data in the storage area according to the read request and the request information in the buffer area, or according to the writing The request and the request information in the buffer area write data to the storage area;
  • a storage module configured to store request information in the read or write request into its buffer area
  • the storage module 20 may acquire the configuration parameters on the control node at the initial startup, perform an initialization operation, and create the buffer area in each storage node. It can be understood that the configuration parameter may be set by the user or may be used during server manufacturing. Entered by the manufacturer, and the configuration parameters can be modified at any time and forwarded by the control node to all storage nodes.
  • the configuration parameter includes: a workgroup name, which is used to define a workgroup name of the cluster file system; a server name, which is used to define a name of each storage node of the cluster file system; and a maximum number of connections, which is used to define the most The number of clients that are allowed to access the cluster file system at the same time; the unified storage path is used to define the root directory of the cluster file system, that is, the same directory address or file can be accessed through any node address; the redundant network interface is used to define the cluster.
  • Each storage node of the file system is used as a redundant network interface, that is, a network card; a redundant network address is used to define a network address of each storage node of the cluster file system used as a redundant network interface (ie, an IP corresponding to the network card), Multiple addresses can be set; the redundant host address is used to define the public network address that the cluster file system provides access to the client, and only one can be set; the data cache size is used to define the storage file cache on each storage node of the cluster file system.
  • the memory size of the data For example, the user can set the storage area of the storage node to occupy 20 GB of storage space, and the storage space occupied by the storage area cannot exceed the maximum storage space of the storage node server.
  • the synchronization module 30 is configured to synchronize the request information to a slave storage node in the cluster file system.
  • the control node when the system initiates initialization, sends a message to the server that is set as the primary storage node, and controls the primary storage node to start the program, and the other storage nodes do not start the service program, and can only receive and A request information for storing a read or write request synchronized by the primary storage node, and only the primary storage node synchronizes the read or write request to all slaves in the cluster file system upon receiving a data read or write request
  • the storage node stores the requested information of the read or write request received from the storage node into its buffer area without any other operations.
  • the request information of the request After the primary storage node receives the data read or write request from the client, the request information of the request: the process number of the client data read or write, the I/O offset of the process, Synchronize to all slave storage nodes.
  • the storage node may determine the type of the service request when receiving the service request, and perform corresponding processing according to the type of the server request, that is, the data processing apparatus of the cluster file system further includes:
  • the detecting module 40 is configured to detect a type of the service request when receiving a service request, and determine whether the service request is a read or write request;
  • the response module 50 is configured to respond to the service request if the service request is not a read or write request;
  • the data processing module 10 is further configured to: if the service request is a read or write request, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to the storage area.
  • the detecting module 40 detects the service type invoked by the request, and when the request invokes the data reading or writing service, the synchronization module 30 sends the requested request information to the
  • the primary storage node network interface is further sent by the network interface of the primary storage node to the secondary storage node corresponding to the node address obtained from the control node, and the received synchronization request is stored in the buffer area by the storage module 10 of the storage node .
  • the primary storage node not only receives the data read or write request, but also may receive other types of service requests, such as configuration parameter modification requests, and the response module 50 responds directly to the service request.
  • the response module 50 responds directly to the service request.
  • the storage module 20 does not store the service request into its buffer area, and the synchronization module 20 does not synchronize the service request to all the slave storage nodes. .
  • the primary storage node when the primary storage node may not be able to provide data reading or writing services to the client due to system upgrade or maintenance, etc., the primary storage node selects a normal working state. From the storage node, the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.
  • the primary storage node after receiving the data operation request, the primary storage node synchronizes the requested information to all the slave storage nodes, and when the storage node needs to be switched, any one of them is in a normal working state.
  • the slave storage node can take over the client's data operation process, provide the client with uninterrupted data operation services, and improve the availability of the cluster file system.
  • the data processing apparatus of the cluster file system further includes:
  • the detecting module 40 is further configured to detect a link connection state and an operating state thereof;
  • the detecting module 40 includes:
  • the link detecting unit 41 is configured to periodically send the first detection data packet to the control node;
  • the read/write detecting unit 42 is configured to determine whether the reading and writing are normal when the response packet is received within a preset time interval;
  • the determining unit 43 is configured to determine that the operation is normal when the reading and the writing are normal, determine the operation failure when the reading and the writing are abnormal, and determine that the response data packet is not received within the preset time interval.
  • the link connection is faulty.
  • the primary storage node selects a secondary storage node in a normal working state.
  • the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.
  • the switching module 60 is configured to: when detecting a link connection failure or an operation failure, select a primary storage node from the storage node in a normal working state, and mark the selected address of the secondary storage node as a primary storage node address;
  • An update module 70 configured to send the marked primary node address to the control node and the selected secondary storage node, where the control node updates the saved primary node address by using the received primary node address update And the selected storage node switches the working state to the state of the primary storage node when receiving the address of the primary node.
  • the switching module 60 selects a slave storage node in a normal working state, marks the selected address of the slave node as the address of the new primary storage node, and the update module 70 sends the address.
  • the control node will forward the subsequent client request to the new primary storage node, the selected secondary storage node will switch to the working state of the primary storage node, and the new primary storage node can read its cache area.
  • the internal data reads or writes the process information, directly takes over the corresponding process, and does not interrupt the data operation process of the client.
  • the performance of the slave storage node may be substantially the same, or the performance difference may be large. Therefore, when the primary storage node performs the handover, if all the performances of the slave storage nodes are similar, the switching module 60 may randomly select one to be in normal operation.
  • the slave storage node of the state is used as the primary storage node. If the performance of all the slave storage nodes is different, the node with higher performance in the normal working state may be selected as the new primary storage node according to the performance level of the storage node.
  • the data processing device of the cluster file system in this embodiment periodically detects the link connection state of the primary storage node and the service program running state, discovers the fault of the primary storage node in time, and then switches the primary storage node to provide uninterrupted support for the client.
  • Data read and write services ensure the stability of cluster file system data processing services.
  • the present invention further provides a third embodiment of the data processing apparatus of the cluster file system based on the first or second embodiment.
  • the data processing apparatus of the cluster file system also includes:
  • a determining module 80 configured to determine, when a file read or write request is received, whether data corresponding to the read or write request needs to be verified;
  • the primary storage node when the primary storage node receives the file read or write request, it detects the access level of the file requested to be accessed. If the file is a shared file, it can be accessed by all users. There is no need to verify the identity rights of the user. If the file is a private file, that is, the file can only be accessed by a user with permission, the identity permission of the user needs to be verified to determine whether the file can be issued. Client access to the request.
  • the obtaining module 90 is configured to obtain verification information input by the user when the data corresponding to the read or write request needs to be verified;
  • the primary storage node When the file requested by the client is a private file, that is, when authentication is required, the primary storage node needs to obtain information input by the user and verify the authority of the user. At this point, the user may have logged in, and the primary storage node only needs to traverse the information forwarded by the control node, grab the data with the user name and password key or handle, and then obtain the information input by the user, and perform the user's authority. If the user has not logged in at this time, the primary storage node sends a message to the control node, notifying the control node to send a reminder to the client or controlling the client to pop up the login interface. After the user inputs the verification information, the control node inputs the user. The verification information is forwarded to the primary storage node for verification.
  • the data processing module 10 is further configured to: when the verification information matches the pre-stored verification information, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to its storage area.
  • the data processing apparatus of the cluster file system obtains the operation authority of the data after receiving the data operation request, and obtains the verification information input by the user if the data requires the authority verification, in the user When the input verification information matches the pre-stored verification information, the corresponding data operation is performed to improve the security of the cluster file system data.
  • the invention further provides a data processing method for a cluster file system.
  • the data processing method of the cluster file system includes:
  • Step S10 when the storage node receives the file read or write request, the storage node reads the data in the storage area according to the read request and the request information in the buffer area, or according to the write The incoming request and the request information in the buffer area write data to its storage area;
  • Step S20 the storage node stores the request information in the read or write request into its buffer area
  • the storage node may acquire the configuration parameters on the control node at the initial startup, perform an initialization operation, and create the cache area in the storage node. It can be understood that the configuration parameter may be set by the user, or may be set by the manufacturer during server manufacture. Enter.
  • configuration parameters are described in the first embodiment of the data processing of the cluster file system, and details are not described herein again.
  • the control node when the system initiates initialization, sends a message to the server that is set as the primary storage node, and controls the primary storage node to start the program, and the other storage nodes do not start the service program, and can only receive and A request information for storing a read or write request synchronized by the primary storage node, and only the primary storage node synchronizes the read or write request to all slaves in the cluster file system upon receiving a data read or write request
  • the storage node stores the requested information of the read or write request received from the storage node into its buffer area without any other operations.
  • the request information of the request After the primary storage node receives the data read or write request from the client, the request information of the request: the process number of the client data read or write, the I/O offset of the process, Synchronize to all slave storage nodes.
  • Step S30 the storage node synchronizes the request information to a slave storage node in the cluster file system.
  • the primary storage node After receiving the data read or write request, the primary storage node responds to the request and provides a data read or write service to the client according to the request information.
  • the primary storage node not only receives the data read or write request, but also may receive other types of service requests, such as configuration parameter modification requests, and directly respond to the request, as described above.
  • the corresponding parameter is requested to be modified accordingly, but the service request is not stored in its cache area, and the service request is not synchronized to all the slave storage nodes.
  • the storage node may determine the type of the service request when the service request is received, and perform corresponding processing according to the type of the server request, that is, the data processing method of the cluster file system further includes:
  • Step S40 When the storage node receives the service request, detecting a type of the service request.
  • Step S50 determining whether the service request is a read or write request
  • the service request is a read or write request, executing, by the storage node, reading data in a storage area according to the read request and request information in the buffer area, or according to the write request And a step of writing data in the storage area to the request information in the buffer area;
  • the data processing method of the cluster file system further includes:
  • Step S60 if the service request is not a read or write request, responding to the service request.
  • the primary storage node After receiving the service request, the primary storage node detects the service type invoked by the request, and sends the requested request information to the primary storage node network interface when the request invokes the data read or write service. Then, the network interface of the primary storage node sends the slave storage node corresponding to the node address acquired from the control node, and the synchronization request received from the storage node is stored in its buffer area.
  • the primary storage node when the primary storage node may not be able to provide data reading or writing services to the client due to system upgrade or maintenance, etc., the primary storage node selects a normal working state. From the storage node, the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.
  • the data processing method of the cluster file system in this embodiment after receiving the data operation request, the primary storage node synchronizes the requested information to all the slave storage nodes, and when the storage node needs to be switched, any one is in a normal working state.
  • the slave storage node can take over the client's data operation process, provide the client with uninterrupted data operation services, and improve the availability of the cluster file system.
  • the primary storage node when the primary storage node provides services to the client, a failure may occur, and the primary storage node needs to periodically detect the link connection status and the service program running status, so as to timely find the fault and perform corresponding processing, based on the first
  • An embodiment provides a second embodiment of the data processing method of the cluster file system of the present invention.
  • the data processing method of the cluster file system further includes:
  • Step S70 When the storage node is a primary storage node, the storage node detects a link connection state thereof and an operation state of the storage node.
  • the step of detecting, by the storage node, its link connection state and the running state of the storage node includes:
  • Step S71 the storage node periodically sends a first detection data packet to the control node
  • Step S72 Receive a second response data packet that is sent back by the control node based on the detection data packet:
  • Step S73 when receiving the response data packet within a preset time interval, determining whether the reading and writing of the storage node are normal;
  • Step S74 when the reading and writing of the storage node are normal, determining that the storage node is operating normally;
  • Step S75 determining that the storage node is faulty when the storage node reads and writes an abnormality
  • Step S76 When the response data packet is not received within the preset time interval, determine that the link connection of the storage node is faulty.
  • the primary storage node when the primary storage node detects that it is faulty and cannot continue to provide data reading or writing services to the client, the primary storage node selects a secondary storage node that is in a normal working state, and the selected slave will be selected. The storage node switches to the primary storage node and takes over the data read or write process of the client.
  • Step S80 when the acquiring storage node detects its link connection failure or the storage node running fault in real time, the storage node selects a primary storage node from the storage node in a normal working state, and selects the selected storage node.
  • the address from the storage node is marked as the primary storage node address;
  • Step S90 sending the marked primary node address to the control node and the selected secondary storage node, wherein the control node updates the saved primary node address by using the received primary node address, and selects When the storage node receives the address of the primary node, the working state is switched to the state of the primary storage node.
  • a slave storage node in a normal working state is selected, the selected address of the slave service node is marked as the address of the new primary storage node, and the address is sent to the control node, and the control node will The subsequent client request will be forwarded to the new primary storage node, and the selected secondary storage node will switch to the working state of the primary storage node, and the new primary storage node can read the data in the buffer to read or write.
  • the process information directly take over the corresponding process, will not interrupt the client's data operation process.
  • the performance of the slave storage node may be substantially the same, or the performance difference may be large. Therefore, when the primary storage node performs the handover, if the performance of all the slave storage nodes is almost the same, the slaves in the normal working state may be randomly selected. As the primary storage node, if the performance of all the secondary storage nodes is different, the node with higher performance in the normal working state can be selected as the new primary storage node according to the performance of the storage node.
  • the data processing method of the cluster file system in this embodiment periodically detects the link connection state of the primary storage node and the service program running state, discovers the fault of the primary storage node in time, and then switches the primary storage node to provide uninterrupted support for the client.
  • Data read and write services ensure the stability of cluster file system data processing services.
  • the present invention further provides a third embodiment of the data processing method of the cluster file system based on the first or second embodiment.
  • the The data processing method of the cluster file system further includes the steps of:
  • Step S100 the storage node determines whether data corresponding to the read or write request needs to be verified
  • the primary storage node when the primary storage node receives the file read or write request, it detects the access level of the file requested to be accessed. If the file is a shared file, it can be accessed by all users. There is no need to verify the identity rights of the user. If the file is a private file, that is, the file can only be accessed by a user with permission, the identity permission of the user needs to be verified to determine whether the file can be issued. Client access to the request.
  • Step S110 when the data corresponding to the read or write request needs to be verified, the storage node acquires the verification information input by the user;
  • the primary storage node When the file requested by the client is a private file, that is, when authentication is required, the primary storage node needs to obtain information input by the user and verify the authority of the user. At this point, the user may have logged in, and the primary storage node only needs to traverse the information forwarded by the control node, grab the data with the user name and password key or handle, and then obtain the authentication information input by the user, and the authority to the user. If the user has not logged in yet, the primary storage node sends a message to the control node, notifying the control node to send a permission verification reminder to the client or controlling the client to pop up the login interface. After the user inputs the verification information, the control node will The verification information input by the user is forwarded to the primary storage node for verification.
  • Step S120 determining whether the verification information matches the pre-stored verification information
  • step S10 When the verification information matches the pre-stored verification information, executing the storage node to read data in the storage area according to the read request and the request information in the buffer area, or according to the write request and The step of writing the request information in the buffer area to the data area thereof, that is, step S10.
  • the data processing method of the cluster file system acquires the operation authority of the data after receiving the data operation request, and if the data requires the authority verification, the verification information input by the user is obtained, and the user inputs the When the verification information matches the pre-stored verification information, the corresponding data operation is performed to improve the security of the cluster file system data.
  • the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better.
  • Implementation Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk,
  • the optical disc includes instructions for causing a terminal device (which may be a mobile phone, a computer, a cloud server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)

Abstract

L'invention concerne un appareil de traitement de données pour un système de fichiers en grappes. L'appareil de traitement de données pour un système de fichiers en grappes comprend : un module de traitement de données pour lire, lorsqu'un nœud de stockage reçoit une requête de lecture ou d'écriture de fichier, des données dans une zone de stockage en fonction de la requête de lecture et d'informations de requête dans une mémoire tampon ou écrire des données dans la zone de stockage en fonction de la requête d'écriture et d'informations de requête dans la mémoire tampon; un module de stockage, pour stocker les informations de requête dans la requête de lecture ou d'écriture dans la mémoire tampon; et un module de synchronisation, pour synchroniser les informations de requête avec un nœud de stockage esclave dans un système de fichiers en grappes. L'invention concerne également un procédé de traitement de données pour un système de fichiers en grappes. Après une défaillance d'un nœud de stockage maître, l'appareil et le procédé permettent à un nœud de stockage esclave de prendre en charge un processus de lecture ou d'écriture d'un client, sans interruption, en fonction d'informations de requête de service dans la mémoire tampon, garantissant ainsi la stabilité de service d'un système de fichiers en grappes.
PCT/CN2016/105219 2015-11-26 2016-11-09 Procédé et appareil de traitement de données destinés à un système de fichiers en grappes WO2017088664A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510847053.1A CN105511805B (zh) 2015-11-26 2015-11-26 集群文件系统的数据处理方法和装置
CN201510847053.1 2015-11-26

Publications (1)

Publication Number Publication Date
WO2017088664A1 true WO2017088664A1 (fr) 2017-06-01

Family

ID=55719825

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105219 WO2017088664A1 (fr) 2015-11-26 2016-11-09 Procédé et appareil de traitement de données destinés à un système de fichiers en grappes

Country Status (2)

Country Link
CN (1) CN105511805B (fr)
WO (1) WO2017088664A1 (fr)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704201A (zh) * 2017-09-11 2018-02-16 厦门集微科技有限公司 数据存储处理方法及装置
CN109508317A (zh) * 2018-10-31 2019-03-22 武汉光谷联众大数据技术有限责任公司 一种大容量数据和服务管理系统
CN109587221A (zh) * 2018-11-09 2019-04-05 平安科技（深圳）有限公司 大数据集群管理方法、装置、存储介质和计算机设备
CN110674095A (zh) * 2019-09-27 2020-01-10 浪潮电子信息产业股份有限公司 一种ctdb集群扩展方法、装置、设备及可读存储介质
CN110716692A (zh) * 2018-07-13 2020-01-21 浙江宇视科技有限公司 读取性能提升方法、装置、存储节点及数据读取方法
CN111314129A (zh) * 2020-02-13 2020-06-19 上海凯岸信息科技有限公司 一种基于文件式存储服务高可用架构
CN112395165A (zh) * 2020-11-27 2021-02-23 中电科技（北京）有限公司 获取服务器的资源利用率的方法、设备和服务器
CN112468330A (zh) * 2020-11-13 2021-03-09 苏州浪潮智能科技有限公司 一种故障节点的设置方法、系统、设备以及介质
CN112671905A (zh) * 2020-12-23 2021-04-16 广州三七互娱科技有限公司 服务调度方法、装置及系统
CN112737962A (zh) * 2020-12-24 2021-04-30 平安科技（深圳）有限公司 存储服务请求的处理方法、装置、计算机设备及存储介质
CN112988905A (zh) * 2021-04-27 2021-06-18 北京沃丰时代数据科技有限公司 用于集群部署的节点内存同步方法及装置
CN113590040A (zh) * 2021-07-29 2021-11-02 郑州阿帕斯数云信息科技有限公司 数据处理方法、装置、设备和存储介质
CN114189547A (zh) * 2022-02-14 2022-03-15 北京安盟信息技术股份有限公司 一种集群下ssl隧道快速切换方法
WO2023109381A1 (fr) * 2021-12-16 2023-06-22 中移(苏州)软件技术有限公司 Procédé et appareil de traitement d'informations et support de stockage

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105511805B (zh) * 2015-11-26 2019-03-19 深圳市中博科创信息技术有限公司 集群文件系统的数据处理方法和装置
CN106254103B (zh) * 2016-07-28 2019-08-16 北京国电通网络技术有限公司 一种rtmp集群系统可动态配置方法及装置
CN106598762B (zh) * 2016-12-29 2020-04-17 上海理想信息产业（集团）有限公司 一种消息同步方法
CN108512753B (zh) 2017-02-28 2020-09-29 华为技术有限公司 一种集群文件系统中消息传输的方法及装置
CN107168649B (zh) * 2017-05-05 2019-12-17 南京城市职业学院 一种分布式存储系统中数据分布的方法及装置
CN107707620B (zh) * 2017-08-30 2020-09-11 华为技术有限公司 处理io请求的方法及装置
CN109543204B (zh) * 2017-09-22 2022-09-13 南京理工大学 人体静电作用下半导体器件电热一体化分析方法
CN108023772B (zh) * 2017-12-07 2021-02-26 海能达通信股份有限公司 一种异常节点修复方法、装置及相关设备
CN110099084B (zh) * 2018-01-31 2021-06-15 北京易真学思教育科技有限公司 一种保证存储服务可用性的方法、系统及计算机可读介质
CN108829720B (zh) * 2018-05-07 2022-01-14 麒麟合盛网络技术股份有限公司 数据处理方法及装置
CN109213507A (zh) * 2018-08-27 2019-01-15 郑州云海信息技术有限公司 一种升级方法及服务器
CN110474981A (zh) * 2019-08-13 2019-11-19 中科天御(苏州)科技有限公司 一种软件定义动态安全存储方法及装置
CN110868323B (zh) * 2019-11-15 2022-07-22 浪潮电子信息产业股份有限公司 一种带宽控制方法、装置、设备及介质
CN111371865B (zh) * 2020-02-26 2023-02-24 上海达梦数据库有限公司 一种客户端连接关系调整方法、系统及节点
CN113051349A (zh) * 2021-04-02 2021-06-29 广东美电贝尔科技集团股份有限公司 一种执勤系统数据同步方法
CN113259092A (zh) * 2021-04-04 2021-08-13 余绍祥 一种文档分布式加密系统
CN116644039B (zh) * 2023-05-25 2023-12-19 安徽继远软件有限公司 一种基于大数据的在线能力运营日志自动采集分析的方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055494A1 (en) * 2009-08-25 2011-03-03 Yahoo! Inc. Method for distributed direct object access storage
CN102035862A (zh) * 2009-09-30 2011-04-27 国际商业机器公司 Svc集群中配置节点的故障移交方法和系统
US20130073717A1 (en) * 2011-09-15 2013-03-21 International Business Machines Corporation Optimizing clustered network attached storage (nas) usage
CN105511805A (zh) * 2015-11-26 2016-04-20 深圳市中博科创信息技术有限公司 集群文件系统的数据处理方法和装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4643543B2 (ja) * 2006-11-10 2011-03-02 株式会社東芝 キャッシュ一貫性保証機能を有するストレージクラスタシステム
US8875146B2 (en) * 2011-08-01 2014-10-28 Honeywell International Inc. Systems and methods for bounding processing times on multiple processing units
CN103634269B (zh) * 2012-08-21 2017-04-19 中国银联股份有限公司 单点登录系统及方法
US9189510B2 (en) * 2013-02-26 2015-11-17 Facebook, Inc. System and method for implementing cache consistent regional clusters
CN103207841B (zh) * 2013-03-06 2016-01-20 青岛海信传媒网络技术有限公司 基于键值对缓存的数据读写方法及装置
CN103207894A (zh) * 2013-03-14 2013-07-17 深圳市知正科技有限公司 一种多路实时视频数据存储系统及其进行缓存控制的方法
CN104361030A (zh) * 2014-10-24 2015-02-18 西安未来国际信息股份有限公司 一种具有任务分发功能的分布式缓存架构及缓存方法
CN104580432A (zh) * 2014-12-23 2015-04-29 上海帝联信息科技股份有限公司 memcached系统及内存缓存数据提供、维护和集群维护方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055494A1 (en) * 2009-08-25 2011-03-03 Yahoo! Inc. Method for distributed direct object access storage
CN102035862A (zh) * 2009-09-30 2011-04-27 国际商业机器公司 Svc集群中配置节点的故障移交方法和系统
US20130073717A1 (en) * 2011-09-15 2013-03-21 International Business Machines Corporation Optimizing clustered network attached storage (nas) usage
CN105511805A (zh) * 2015-11-26 2016-04-20 深圳市中博科创信息技术有限公司 集群文件系统的数据处理方法和装置

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704201A (zh) * 2017-09-11 2018-02-16 厦门集微科技有限公司 数据存储处理方法及装置
CN107704201B (zh) * 2017-09-11 2020-07-31 厦门集微科技有限公司 数据存储处理方法及装置
CN110716692B (zh) * 2018-07-13 2022-11-25 浙江宇视科技有限公司 读取性能提升方法、装置、存储节点及数据读取方法
CN110716692A (zh) * 2018-07-13 2020-01-21 浙江宇视科技有限公司 读取性能提升方法、装置、存储节点及数据读取方法
CN109508317A (zh) * 2018-10-31 2019-03-22 武汉光谷联众大数据技术有限责任公司 一种大容量数据和服务管理系统
CN109508317B (zh) * 2018-10-31 2023-06-09 陕西合友网络科技有限公司 一种大容量数据和服务管理系统
CN109587221A (zh) * 2018-11-09 2019-04-05 平安科技（深圳）有限公司 大数据集群管理方法、装置、存储介质和计算机设备
CN110674095A (zh) * 2019-09-27 2020-01-10 浪潮电子信息产业股份有限公司 一种ctdb集群扩展方法、装置、设备及可读存储介质
CN110674095B (zh) * 2019-09-27 2022-06-10 浪潮电子信息产业股份有限公司 一种ctdb集群扩展方法、装置、设备及可读存储介质
CN111314129A (zh) * 2020-02-13 2020-06-19 上海凯岸信息科技有限公司 一种基于文件式存储服务高可用架构
CN112468330A (zh) * 2020-11-13 2021-03-09 苏州浪潮智能科技有限公司 一种故障节点的设置方法、系统、设备以及介质
CN112468330B (zh) * 2020-11-13 2022-12-06 苏州浪潮智能科技有限公司 一种故障节点的设置方法、系统、设备以及介质
CN112395165A (zh) * 2020-11-27 2021-02-23 中电科技（北京）有限公司 获取服务器的资源利用率的方法、设备和服务器
CN112671905A (zh) * 2020-12-23 2021-04-16 广州三七互娱科技有限公司 服务调度方法、装置及系统
CN112737962A (zh) * 2020-12-24 2021-04-30 平安科技（深圳）有限公司 存储服务请求的处理方法、装置、计算机设备及存储介质
CN112737962B (zh) * 2020-12-24 2023-06-02 平安科技（深圳）有限公司 存储服务请求的处理方法、装置、计算机设备及存储介质
CN112988905B (zh) * 2021-04-27 2021-08-10 北京沃丰时代数据科技有限公司 用于集群部署的节点内存同步方法及装置
CN112988905A (zh) * 2021-04-27 2021-06-18 北京沃丰时代数据科技有限公司 用于集群部署的节点内存同步方法及装置
CN113590040A (zh) * 2021-07-29 2021-11-02 郑州阿帕斯数云信息科技有限公司 数据处理方法、装置、设备和存储介质
CN113590040B (zh) * 2021-07-29 2024-03-19 郑州阿帕斯数云信息科技有限公司 数据处理方法、装置、设备和存储介质
WO2023109381A1 (fr) * 2021-12-16 2023-06-22 中移(苏州)软件技术有限公司 Procédé et appareil de traitement d'informations et support de stockage
CN114189547B (zh) * 2022-02-14 2022-05-03 北京安盟信息技术股份有限公司 一种集群下ssl隧道快速切换方法
CN114189547A (zh) * 2022-02-14 2022-03-15 北京安盟信息技术股份有限公司 一种集群下ssl隧道快速切换方法

Also Published As

Publication number Publication date
CN105511805A (zh) 2016-04-20
CN105511805B (zh) 2019-03-19

Similar Documents

Publication Publication Date Title
WO2017088664A1 (fr) Procédé et appareil de traitement de données destinés à un système de fichiers en grappes
WO2016068622A1 (fr) Dispositif de terminal et son procédé de commande
WO2018233370A1 (fr) Procédé, système, dispositif et support de stockage lisible par ordinateur de synchronisation d'une image
WO2013131444A1 (fr) Procédé, terminal, serveur et système de partage de contenu et support de stockage informatique
WO2013017004A1 (fr) Procédé, système, client et serveur de numérisation de fichier
WO2019128174A1 (fr) Procédé de lecture audio, téléviseur intelligent, et support d'informations lisible par ordinateur
WO2015172684A1 (fr) Procédé de connexion à un ap, terminal et serveur
WO2018068411A1 (fr) Procédé de débogage à distance de télévision intelligente et système de débogage à distance de télévision intelligente
WO2019205272A1 (fr) Procédé, dispositif et équipement de fourniture de service de machine virtuelle, et support d'informations lisible par ordinateur
WO2015120774A1 (fr) Procédé et appareil d'accès à un réseau appliqués à une application mobile
WO2015157942A1 (fr) Dispositif et procédé d'accès à un réseau sans fil
WO2018028121A1 (fr) Procédé et dispositif de gestion de l'espace de stockage d'une partition de données
WO2018076864A1 (fr) Procédé, appareil, support de stockage et dispositif électronique de synchronisation de données
WO2018014567A1 (fr) Procédé servant à améliorer les performances d'une machine virtuelle, et terminal, dispositif et support de stockage lisible par ordinateur
WO2016058258A1 (fr) Procédé et système de commande à distance de terminal
WO2018053963A1 (fr) Procédé et appareil de mise à niveau d'un système de téléviseur intelligent
WO2016000560A1 (fr) Procédé et appareil de transmission de fichier et appareil de réception de fichier
WO2018076875A1 (fr) Procédé de synchronisation de données de sauvegarde, appareil, support de stockage, dispositif électronique, et serveur
WO2021241849A1 (fr) Dispositif électronique pour la réalisation d'un service informatique périphérique et procédé de fonctionnement de dispositif électronique
WO2018120680A1 (fr) Système de sauvegarde de disque virtuel, procédé, appareil, hôte de service et support d'enregistrement
WO2018121026A1 (fr) Procédé et système de configuration de boîtier adaptateur
WO2018076870A1 (fr) Procédé et appareil de traitement de données, support de stockage, serveur, et système de traitement de données
WO2017024805A1 (fr) Procédé, appareil, et système de distribution de fichier
WO2018076873A1 (fr) Procédé de partage de données, appareil, support, dispositif électronique, et système
EP3777210A1 (fr) Procédé de lecture de contenu avec continuité et dispositif électronique correspondant

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16867888

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16867888

Country of ref document: EP

Kind code of ref document: A1