WO2006089479A1 - Méthode de gestion de données dans un système de stockage en réseau et système de stockage en réseau reposant sur la méthode - Google Patents

Méthode de gestion de données dans un système de stockage en réseau et système de stockage en réseau reposant sur la méthode Download PDF

Info

Publication number
WO2006089479A1
WO2006089479A1 PCT/CN2006/000256 CN2006000256W WO2006089479A1 WO 2006089479 A1 WO2006089479 A1 WO 2006089479A1 CN 2006000256 W CN2006000256 W CN 2006000256W WO 2006089479 A1 WO2006089479 A1 WO 2006089479A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
partition
manager
storage manager
mapping
Prior art date
Application number
PCT/CN2006/000256
Other languages
English (en)
Chinese (zh)
Inventor
Yaolong Zhu
Hui Xiong
Jie Yan
Original Assignee
Zhang, Jinkui
Zhou, Feng
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhang, Jinkui, Zhou, Feng filed Critical Zhang, Jinkui
Priority to US11/884,938 priority Critical patent/US8145873B2/en
Publication of WO2006089479A1 publication Critical patent/WO2006089479A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/14—Details of searching files based on file metadata
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0613—Improving I/O performance in relation to throughput
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management

Definitions

  • a network storage system is a data storage system based on a storage network and a storage server as a core, and provides a data storage service for an application server.
  • the system emphasizes scalability, high reliability, and dynamic processing capability of data services, and currently used technologies. It is a storage area network (SAN).
  • SAN storage area network
  • a storage area network is a separate data network consisting of several storage servers and storage devices and storage network devices. It is connected by a dedicated interconnection and is a high-speed, dedicated, centralized management and security information system architecture.
  • the storage server that provides external data services will cause the system's I/O performance to degrade when it is expanded, thus limiting the application of this type of server.
  • the problem of dynamic expansion of storage servers has become an important factor affecting the overall performance expansion of the system.
  • each metadata server uses its own corresponding storage space to store metadata, and the corresponding storage space is in the local storage device corresponding to the metadata server.
  • Storage space In general, metadata manager clusters (that is, storage server clusters) primarily use two metadata storage and management methods.
  • each metadata server in the cluster saves part of the global metadata table, that is, the global metadata table is merged by the child tables on all the metadata servers in the cluster.
  • each metadata server can have high metadata access performance and metadata storage efficiency, and no other synchronization mechanisms are required.
  • this design makes the metadata server cluster less prone to disaster tolerance and load balancing. Because the failure of a metadata server will result in the loss of some metadata tables. The unbalanced dynamic load of the metadata will result in a large amount of metadata being accessed from a certain server, which will reduce the performance of the metadata server cluster.
  • Each metadata server in the cluster locally saves a copy of the synchronized global metadata table, that is, each metadata server can access the global metadata table, including read access. And write access.
  • the metadata server uses a certain synchronization mechanism to ensure the real-time and integrity of the global metadata. In this way, the metadata server cluster can have good load balancing and disaster tolerance. But because usually metadata access is a very frequent small I/O request, this synchronization mechanism between metadata servers will damage the overall performance of the metadata server cluster, which in turn will degrade the performance of the entire storage system. And because of the overhead of this synchronization mechanism, the scalability of the metadata server cluster is also limited. Because adding a new metadata server means there are more global metadata tables that need to be synchronized.
  • An object of the present invention is to provide a data management method in a network storage system, which can effectively solve the problem of expansion of a storage server in an existing network storage system, and also solves the dynamic processing and recovery of a storage server failure.
  • the technical problems of load balancing and reconfiguration of the storage server cluster can maximize the dynamic processing capability of the system, thus greatly improving the reliability and availability of the data.
  • Another object of the present invention is to provide a network storage system constructed by the method.
  • the storage server in the system has superior dynamic expansion functions, and can dynamically process and recover the failure of the storage server, that is, has better disaster tolerance performance.
  • the load balancing and reconstruction of the storage server cluster can be realized under the condition of ensuring high storage efficiency of the system.
  • a technical solution for achieving the first object of the present invention is as follows: A data management method in a network storage system, the network storage system comprising an interconnection network, and a data storage service provided to an application server connected to the interconnection network.
  • a storage server cluster and a storage space corresponding to each storage server where the data stored and managed by each storage server includes original data that the application server needs to access and metadata stored in the corresponding storage space
  • a core manager is disposed in the storage server, and the storage space corresponding to each storage server is formed into a shared storage space by the core manager, and each storage server is separated from its corresponding metadata to become a storage manager.
  • the core manager in the storage manager manages the storage location of the metadata in the shared storage space and the correspondence between the metadata and the storage managers.
  • the present invention forms a shared storage space by concentrating the storage spaces corresponding to the storage servers, so that the storage spaces corresponding to the respective storage servers are no longer private spaces of the storage servers, but are managed by the core manager.
  • the shared storage space is arranged according to the needs of efficient data management, and the metadata in the storage space corresponding to each storage server is correspondingly Separate from the respective storage servers, and become assignable metadata managed by each storage server according to efficient management needs under the core manager allocation.
  • the storage server that manages only the metadata is defined as the storage manager, and the corresponding storage server cluster becomes the storage manager cluster, that is, each storage manager no longer has the corresponding storage space and corresponding metadata, but only the core management.
  • the metadata assigned to it is managed.
  • each storage manager does not have its own metadata, that is, separation from metadata, storage manager expansion, failure handling, dynamic load balancing and reconstruction of the storage manager cluster become independent of metadata.
  • the operation can effectively solve the scalability problem in the existing design scheme, and also solve the dynamic processing and recovery of the storage manager failure, the load balancing of the storage manager cluster, and the storage manager cluster reconstruction technology. problem.
  • a more optimized design is that the shared storage space is provided with a partition, and the storage manager has a one-to-one and/or one-to-many mapping relationship with the partition.
  • the management of the core manager is more convenient, that is, only the partitions are numbered, and the correspondence between the partitions and the storage managers is set, so that the data management can be ordered and efficient.
  • Each partition of the shared storage space can be mapped to any storage manager, but at the same time, only one storage manager can install the partition and gain exclusive access to it.
  • the core manager is composed of a distribution controller, a mapping controller, and a partition management controller disposed in the storage manager, and the distribution controller implements distribution of metadata of the file on a partition of the shared storage space,
  • the mapping controller implements mapping management of the partition of the shared storage space and each storage manager, and maintains the mapping relationship through a mapping table, and the partition management controller manages the partition of the shared storage space.
  • the method of the invention can efficiently realize the information access of the system, the expansion of the storage manager, and the failure processing.
  • the detailed process of dynamic load balancing and reconstruction of the storage manager cluster is described in detail in the specific implementation manner.
  • a second object of the present invention is the network storage system constructed by the foregoing data management method, including an interconnection network, a storage manager cluster and a storage element connected to the interconnection network to provide a data storage service for an application server.
  • a storage space of the data wherein the storage manager is provided with a core manager; the storage space is a shared storage space shared by all storage managers in the storage manager cluster, and the core manager pairs the metadata in the storage
  • the storage location of the shared storage space and the correspondence between the metadata and the storage managers are managed.
  • the network storage system constructed by the method of the present invention, because the storage manager does not have a corresponding storage space to store corresponding metadata, but is allocated by the core manager to use the shared storage space. Between, that is, each storage manager realizes the separation from the metadata and becomes a replaceable server. Therefore, the expansion of the storage manager, the failure processing, the dynamic load balancing and reconstruction of the storage manager cluster become metadata-independent. Operation, so the storage manager in the system has superior dynamic expansion function, and can dynamically process and recover the failure of the storage server, that is, has better disaster tolerance performance, and can realize storage under the condition of ensuring high storage efficiency of the system. Load balancing and reconfiguration of server clusters.
  • the network storage system constructed by the method of the present invention preferably sets a partition in the shared storage space, and the storage manager has a one-to-one and/or one-to-many mapping relationship with the partition;
  • the core manager is set by a distribution controller, a mapping controller, and a partition management controller in a storage manager, the distribution controller implementing distribution of files on a partition of a shared storage space, the mapping controller implementing partitioning and storage of a shared storage space
  • the mapping management of the manager maintains the mapping relationship through a mapping table that manages the partitions of the shared storage space.
  • each storage manager caches data on the partitions it manages, but the metadata of the data is concentrated on the shared storage space. Therefore, processes such as storage management server expansion, invalidation processing, and dynamic load balancing do not need to move actual data, but only need to adjust the mapping relationship between the storage manager and the partition.
  • the distribution of data storage also becomes partition-based rather than storage-based, that is, the storage location of the data is determined by the partition number it stores, regardless of the storage manager used for storage.
  • the network storage system of the present invention makes the storage manager cluster an architecture with excellent seamless scalability.
  • the process of storage management server expansion, failure processing, and dynamic load balancing is a data-independent operation process, to the greatest extent.
  • the dynamic processing capability of the storage manager cluster is utilized to greatly improve the reliability and availability of data.
  • FIG. 1 is a schematic diagram showing the logical structure of a network storage system constructed by the method of the present invention.
  • Figure 2 Schematic diagram of the functional separation of the functions of the network storage system constructed by the method of the present invention.
  • Figure 3 Flow chart of information access implemented by the method of the present invention.
  • Figure 4 Flow chart of the method of the present invention implementing a storage manager extension.
  • Figure 5 Flow chart of the method for implementing storage manager cluster reconfiguration according to the method of the present invention.
  • Figure 6 Flowchart of the process of the present invention for the failure of the storage manager.
  • Figure 8 Schematic diagram of the structure of the storage system constructed by the method of the present invention for the metadata server.
  • Figure 9 Schematic diagram of the structure of the storage system constructed by the method of the present invention for a file server.
  • FIG. 1 depicts a logical structure diagram of a preferred network storage system constructed by the method of the present invention, the network storage system including an interconnection network (not shown in FIG. 1, see FIG. 7, FIG. 8 in the storage network 103), and a storage manager cluster 2 that provides a data storage service for the application server 1 and a storage space for storing metadata, and a storage manager is provided in each storage manager of the storage manager cluster 2;
  • the storage space is a shared storage space 3 shared by all the storage managers in the storage manager cluster, and the core manager pairs the storage locations of the metadata in the shared storage space and the correspondence between the metadata and the storage managers.
  • the core manager in this embodiment is composed of a distribution controller 10, a mapping controller 20, and a partition management controller 30, and the shared storage space 3 is simultaneously divided into partitions 100.
  • the distribution controller 10 is used to implement the distribution of files on the partition 100 of the shared storage space 3, and can implement a static storage load balancing based on the partition 100.
  • the mapping controller 20 is used to implement mapping management between the partition 100 of the shared storage space 3 and the storage manager cluster 2, and maintains this mapping relationship through a mapping table.
  • the partition management controller 30 implements management functions for the partition 100 of the shared storage space 3, for example, installing or unloading the partition 100 of the shared storage space 3 from each storage manager.
  • the mapping table maintained by the mapping controller 20 regarding the partitions 100 in the shared storage space 3 and the respective storage managers can be shared by all servers.
  • Figure 2 depicts a functionally separated logical structure diagram of a preferred network storage system constructed by the method of the present invention.
  • the distribution controller 10 and the mapping controller 20 operate on the application server side, and the partition management controller 30 operates on the shared storage space side.
  • a variety of file-based application servers 1 provide corresponding services to the outside, and can generate I/O requests to the network storage system, which is the initiator of the storage process.
  • the application server cluster includes a video server, a web server, a database server, an email server, and a file server, as shown in FIG. 8 and FIG. 9.
  • the above servers and managers are distributed through the interconnected switching network. Network communication.
  • the application 50 in FIG. 1 is an application running on the application server 1.
  • the technical solution of the method of the present invention realizes the seamless expansion of the storage manager cluster is centralized storage and management, centralized means forming a common storage space, and the optimization scheme performs partition management on the shared storage space. That is, the core manager is set in the original storage server, and the storage space corresponding to the storage server is formed into a shared storage space by the core manager, and each storage server is separated from the metadata managed by the storage server to become a storage manager, and each storage is stored.
  • the core manager in the manager manages the storage location of the metadata in the shared storage space and the correspondence between the metadata and the storage manager, because each storage manager is no longer the corresponding storage space.
  • the storage metadata is separated from the metadata, so although the core manager is set in each storage manager, a synchronization mechanism is required, but since the synchronization is independent of the metadata, the overall performance of the storage manager cluster 2 is not affected, nor is it This can degrade the performance of the entire storage system.
  • Shared storage 3 Partitioning provides the most efficient and exclusive access to storage resources.
  • the technical solution implemented under this guiding idea makes various storage managers essentially become dynamically replaceable servers that contain metadata.
  • the expansion, failure processing, and dynamic load balancing process of the storage manager cluster 2 can be easily and efficiently solved by adjusting the mapping relationship between the partition 100 of the shared storage space 3 and each storage manager.
  • the above process of dynamic expansion is simplified into some simple partition installation and uninstallation work, and does not require any data movement, thereby achieving seamless dynamic expansion of the storage manager cluster 2.
  • a large number of partitions can ensure seamless dynamic scalability of the cluster.
  • the number of partitions limits the number of storage manager clusters. For example, 1024 partitions indicate that there can be up to 1024 storage managers in the system.
  • the system needs to generate extended shared storage space and generate more partitions, such as 4096. Then, after the storage manager cluster is refactored, a larger extension of the manager cluster can be achieved.
  • the technical effects of the above technical solutions are obvious.
  • Storage Manager 2 a replaceable server that does not contain metadata, because of the number According to the distribution of the storage is based on the partition rather than the storage manager, that is, the storage location of the data is determined by the partition number it stores, regardless of the storage manager used for storage. Therefore, the process of storage management server expansion, failure processing and dynamic load balancing become some data-independent operations, that is, the implementation of the above process does not need to move the actual data, but only needs to adjust the shared storage space between the partitions and the storage manager. Mapping relationship. Therefore, the technical solution of the present invention enables the network storage-based storage manager cluster architecture to have excellent seamless expansion performance, maximizes the dynamic processing capability of the storage manager cluster, and greatly improves data reliability and availability.
  • each partition in the shared storage space in the present invention can only be exclusively accessed by one storage manager, this makes it unnecessary for the storage manager to consider the shared access (for example, distributed file system) during the access process. Synchronization issue. This access process can achieve similar performance to the local file system, and maximize the use of the storage manager's cache, greatly improving data access performance.
  • Figure 3 depicts a flow diagram of system implementation information access in the method of the present invention.
  • the application 50 for various file-based application servers 1 can generate a request for file information.
  • the distribution controller 10 can use the corresponding algorithm to obtain the area code of the partition in which the file is stored based on the file request.
  • the file name can be obtained by hashing the area code of a partition.
  • different file names are allowed to calculate the same partition number. Because, a partition often has to save information about many files.
  • the mapping controller 20 After obtaining the partition number corresponding to the requested file, the mapping controller 20 searches the storage manager and the partition mapping table for the storage manager corresponding to the partition number. The information request is then transmitted over the network. For write type requests, the written information is also passed to the corresponding storage manager. Each storage manager completes the corresponding access through the data access controller 40. For the requested information of the read type request, the last access result will be transmitted to the application server 1 through the network.
  • Figure 4 depicts the process by which the method of the present invention implements the expansion of the storage manager 2.
  • the administrator can dynamically join a new storage manager.
  • adding a new storage manager means that the storage manager is connected to the system.
  • the system can automatically discover this new storage manager.
  • the mapping controller 20 adjusts the mapping table according to a certain algorithm, and some partitions are corresponding to the original storage manager cluster in the mapping table. Delete the record and add the mapping associated with the new storage manager.
  • the partition management controller 30 locks the access queues of the partitions that need to be transferred, and stops the processing of requests in the access queue.
  • Figure 5 depicts the flow of the method of the present invention to implement storage manager cluster reconfiguration.
  • the expansion of the storage manager cluster 2 is different from the general extension (the extension process shown in FIG. 4), but the storage manager cluster reconstruction is first performed.
  • the number of partitions has increased from 128 to 1024, which allows for clusters with up to 1024 storage managers.
  • the expansion process shown in Figure 4 the corresponding expansion.
  • data movement is required, but by the management method of the present invention, such reconfiguration operation can be made to have a fast response time.
  • the allocation controller saves the original allocation algorithm and generates a new allocation algorithm that maps the file name to the expanded number of partitions, while the mapping controller generates a new usage partition. Mapping table.
  • the partition management controller transfers or installs all partitions according to the new mapping table. The operation of the above process can be completed quickly.
  • the system returns the completion information of the cluster reconfiguration command, and the system enters the storage manager cluster reconfiguration state.
  • the storage manager When the system is reconstructing the state, all the data existing before the reconstruction needs to be transferred to different partitions according to the new allocation algorithm.
  • the storage manager receives the data access request from the application server, if the requested information exists, the normal operation as shown in FIG. 3 is performed. If the requested information does not exist, it may be because the information has not been transferred. Therefore, the storage manager can calculate the partition number originally stored in the file according to the saved old allocation algorithm, and the storage manager obtains the storage manager currently managing the partition according to the current mapping table, and the data is from the original Transfer the partition to the new partition. Finally, the application server's request can be responded to.
  • Figure 6 depicts the flow of the method of the present invention for the failure of the storage manager.
  • the system can find that the storage manager cannot respond to the information request.
  • the mapping manager's mapping table all access queues corresponding to the partition managed by the storage manager will be locked.
  • the mapping manager adjusts the mapping table, transfers the affected partitions to the records of other storage managers in the cluster, and deletes the records corresponding to the failed storage manager.
  • the partition management controller then transfers the partitions managed by the failed storage manager to the storage managers corresponding to those partitions in the new mapping table.
  • log-like techniques can be used to recover data from individual partitions.
  • the system can seamlessly implement storage manager failure handling without affecting data access.
  • FIG. 7 depicts the flow of the method of the present invention to implement dynamic load balancing of the storage manager.
  • Dynamic load balancing is used to dynamically redistribute the partitions managed by the storage manager at runtime to solve the problem of some storage managers being over-loaded and other managers being too lightly loaded.
  • the system can monitor the load of each storage manager by itself. When the load of the manager exceeds a preset alert load value, the mapping controller adjusts the mapping table to transfer a portion of the partition managed by the overloaded storage manager to some of the lighter storage managers.
  • the partition management controller locks the access queues of the partitions that need to be transferred, the partitions to be transferred are unloaded from the current storage manager that manages the partitions, and the partitions are installed on the newly allocated storage manager. . Finally, the access queue of the partition that needs to be transferred is unlocked.
  • Figure 8 depicts the network storage system architecture of the present invention in an application environment using a metadata server.
  • the storage systems accessed by the various application servers 1 are composed of a separate metadata server cluster 5 and a storage device cluster 7.
  • the various devices in the system are connected to one another via a storage network 103.
  • the metadata server 5 processes the application server 1 request for metadata, and the data is transmitted through the high speed data channel between the storage device cluster 7 and the application server cluster 1.
  • the metadata server 5 is equivalent to the storage manager of the present invention.
  • the shared storage space 3 is the metadata storage for the metadata server 5.
  • FIG. 9 depicts the network storage system architecture of the present invention in a file server cluster.
  • the application server 1 can access the storage device 7 in the storage network through the file server cluster 8. Since these file server clusters 8 are on the I/O path, the scalability requirements for the file server cluster are even more urgent.
  • the file server cluster 8 is equivalent to the storage manager cluster in the present invention.
  • the storage device cluster 7 can implement a shared storage space 3 through the storage network 103.
  • All data and metadata are stored on the shared storage space 3 through the file server cluster 8.
  • the file server cluster 8 and the shared storage space 3 are connected by a storage network, and the application server 1 and the file server cluster 8 are connected by a network 102, and the request level of the application server 1 is a file level request.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention porte sur une méthode de gestion de données dans un système de stockage en réseau. Un gestionnaire central est défini dans les serveurs de stockage du système de stockage en réseau. Ledit gestionnaire central fait que les espaces de stockage correspondant au serveur de stockage sont constitués en espace public de stockage et sépare chaque serveur de stockage et les méta-données correspondantes que ledit serveur de stockage contrôle de façon à ce qu’il devienne un gestionnaire de stockage. Ledit gestionnaire central des gestionnaires de stockage contrôle l’emplacement de stockage des méta-données dans l’espace public de stockage et la relation des méta-données et dudit gestionnaire de stockage. Il peut démarrer des processus tels l’expansion, la disposition d’invalidation, la compensation de charge dynamique et/ou la reconstruction du cluster et autres facilement et efficacement en ajustant la relation correspondante. La présente invention résout effectivement le problème de possibilité d’expansion d’un système conventionnel de stockage en réseau, étend largement la capacité de traitement dynamique du cluster de gestion de stockage et améliore la fiabilité et la capacité d’usage des données.
PCT/CN2006/000256 2005-02-23 2006-02-23 Méthode de gestion de données dans un système de stockage en réseau et système de stockage en réseau reposant sur la méthode WO2006089479A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/884,938 US8145873B2 (en) 2005-02-23 2006-02-23 Data management method for network storage system and the network storage system built thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CNB2005100113405A CN100337218C (zh) 2005-02-23 2005-02-23 网络存储系统中的数据管理方法及其构建的网络存储系统
CN200510011340.5 2005-02-23

Publications (1)

Publication Number Publication Date
WO2006089479A1 true WO2006089479A1 (fr) 2006-08-31

Family

ID=34875535

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2006/000256 WO2006089479A1 (fr) 2005-02-23 2006-02-23 Méthode de gestion de données dans un système de stockage en réseau et système de stockage en réseau reposant sur la méthode

Country Status (3)

Country Link
US (1) US8145873B2 (fr)
CN (1) CN100337218C (fr)
WO (1) WO2006089479A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113572837A (zh) * 2021-07-22 2021-10-29 南方电网数字电网研究院有限公司 基于分片技术的电网监控系统数据处理服务调度方法

Families Citing this family (101)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE487302T1 (de) * 2007-08-10 2010-11-15 Alcatel Lucent Zugangssystem und netzwerkarchitektur mit zugangssystem
CN101291347B (zh) * 2008-06-06 2010-12-22 中国科学院计算技术研究所 一种网络存储系统
EP2297921B1 (fr) * 2008-07-10 2021-02-24 Juniper Networks, Inc. Stockage en réseau
JP4670968B2 (ja) * 2009-01-22 2011-04-13 富士ゼロックス株式会社 情報管理プログラム及び情報管理システム
CN101997884B (zh) * 2009-08-18 2014-05-14 升东网络科技发展(上海)有限公司 分布式存储系统和方法
CN101827120A (zh) * 2010-02-25 2010-09-08 浪潮(北京)电子信息产业有限公司 一种集群存储方法及系统
US8458299B2 (en) * 2010-03-17 2013-06-04 Hitachi, Ltd. Metadata management method for NAS global namespace design
CN102438030A (zh) * 2010-09-29 2012-05-02 新奥特（北京）视频技术有限公司 一种适用于广电云架构平台的打包系统及方法
CN102594852B (zh) * 2011-01-04 2016-03-30 中国移动通信集团公司 数据访问方法、节点及系统
CN102833278A (zh) * 2011-06-14 2012-12-19 上海博泰悦臻电子设备制造有限公司 一种车载系统的需求处理方法
CN102904913B (zh) * 2011-07-27 2015-06-17 鸿富锦精密工业（深圳）有限公司 云端存储空间管理方法与系统及使用所述方法的网关设备
US8966172B2 (en) 2011-11-15 2015-02-24 Pavilion Data Systems, Inc. Processor agnostic data storage in a PCIE based shared storage enviroment
US9652182B2 (en) 2012-01-31 2017-05-16 Pavilion Data Systems, Inc. Shareable virtual non-volatile storage device for a server
CN102663007B (zh) * 2012-03-20 2013-12-18 广东电子工业研究院有限公司 一种支持敏捷开发和横向扩展的数据存储与查询方法
CN102647362B (zh) * 2012-04-27 2016-12-21 浪潮电子信息产业股份有限公司 一种多控制器存储系统负载均衡的方法
CN102750324A (zh) * 2012-05-28 2012-10-24 华为技术有限公司 文件存储系统、装置及文件存取方法
CN103546502B (zh) * 2012-07-11 2017-02-08 中国电信股份有限公司 元数据共享的方法和云存储服务器
CN102929791A (zh) * 2012-10-25 2013-02-13 浪潮电子信息产业股份有限公司 一种基于相同扩展模块的集群存储系统扩展方法
JP6036190B2 (ja) * 2012-11-07 2016-11-30 富士通株式会社 情報処理装置、情報処理システムの制御方法及び情報処理システムの制御プログラム
CN103034615B (zh) * 2012-12-07 2016-04-13 无锡美森微电子科技有限公司 一种适用于流应用多核处理器的存储管理方法
US20140181061A1 (en) * 2012-12-21 2014-06-26 Hong Jiang Data distribution in a cloud computing system
WO2014101218A1 (fr) 2012-12-31 2014-07-03 华为技术有限公司 Système de grappe intégrée de calcul et de stockage
CN103916441A (zh) * 2013-01-07 2014-07-09 爱国者电子科技有限公司 防计算机病毒的nas存储系统及查杀计算机病毒的方法
CN103458023B (zh) * 2013-08-30 2016-12-28 清华大学 分布式闪存存储系统
CN103618771B (zh) * 2013-11-20 2018-08-14 国家电网公司 自动管理的终端档案分区缓存系统及其工作方法
CN104239438B (zh) * 2014-08-29 2017-11-10 北京大学深圳研究生院 基于分离存储的文件信息存储方法和文件信息读写方法
CN104239101A (zh) * 2014-09-11 2014-12-24 四川长虹电器股份有限公司 基于安卓系统的设备的网络图片的缓存方法
US9830193B1 (en) 2014-09-30 2017-11-28 Amazon Technologies, Inc. Automatic management of low latency computational capacity
US9146764B1 (en) 2014-09-30 2015-09-29 Amazon Technologies, Inc. Processing event messages for user requests to execute program code
US9600312B2 (en) 2014-09-30 2017-03-21 Amazon Technologies, Inc. Threading as a service
US10048974B1 (en) * 2014-09-30 2018-08-14 Amazon Technologies, Inc. Message-based computation request scheduling
US9678773B1 (en) 2014-09-30 2017-06-13 Amazon Technologies, Inc. Low latency computational capacity provisioning
US9323556B2 (en) 2014-09-30 2016-04-26 Amazon Technologies, Inc. Programmatic event detection and message generation for requests to execute program code
CN105518659B (zh) * 2014-10-28 2019-07-26 华为技术有限公司 分布式数据库的数据分区分配方法及装置
US9565269B2 (en) 2014-11-04 2017-02-07 Pavilion Data Systems, Inc. Non-volatile memory express over ethernet
US9712619B2 (en) 2014-11-04 2017-07-18 Pavilion Data Systems, Inc. Virtual non-volatile memory express drive
US9537788B2 (en) 2014-12-05 2017-01-03 Amazon Technologies, Inc. Automatic determination of resource sizing
US10469580B2 (en) * 2014-12-12 2019-11-05 International Business Machines Corporation Clientless software defined grid
US10554749B2 (en) 2014-12-12 2020-02-04 International Business Machines Corporation Clientless software defined grid
CN105988712B (zh) * 2015-01-27 2019-01-25 珠海金山办公软件有限公司 一种网盘空间计算方法及装置
US9588790B1 (en) 2015-02-04 2017-03-07 Amazon Technologies, Inc. Stateful virtual compute system
US9733967B2 (en) 2015-02-04 2017-08-15 Amazon Technologies, Inc. Security protocols for low latency execution of program code
CN104679889A (zh) * 2015-03-17 2015-06-03 浪潮集团有限公司 一种面向大数据处理的数据存储方法和装置
US9930103B2 (en) 2015-04-08 2018-03-27 Amazon Technologies, Inc. Endpoint management system providing an application programming interface proxy service
US9785476B2 (en) 2015-04-08 2017-10-10 Amazon Technologies, Inc. Endpoint management system and virtual compute system
CN105516313A (zh) * 2015-12-10 2016-04-20 国网四川省电力公司信息通信公司 一种用于大数据的分布式存储系统
US9910713B2 (en) 2015-12-21 2018-03-06 Amazon Technologies, Inc. Code execution request routing
US10067801B1 (en) 2015-12-21 2018-09-04 Amazon Technologies, Inc. Acquisition and maintenance of compute capacity
US11132213B1 (en) 2016-03-30 2021-09-28 Amazon Technologies, Inc. Dependency-based process of pre-existing data sets at an on demand code execution environment
US10891145B2 (en) 2016-03-30 2021-01-12 Amazon Technologies, Inc. Processing pre-existing data sets at an on demand code execution environment
US10102040B2 (en) 2016-06-29 2018-10-16 Amazon Technologies, Inc Adjusting variable limit on concurrent code executions
CN107704462B (zh) * 2016-08-08 2021-07-06 阿里巴巴集团控股有限公司 资源的元数据维护方法、设备及存储装置
US10884787B1 (en) 2016-09-23 2021-01-05 Amazon Technologies, Inc. Execution guarantees in an on-demand network code execution system
US10437477B2 (en) * 2017-07-20 2019-10-08 Dell Products, Lp System and method to detect storage controller workloads and to dynamically split a backplane
CN108170371B (zh) * 2017-12-07 2019-05-24 北京空间技术研制试验中心 判读结论压缩存储方法
US10831898B1 (en) 2018-02-05 2020-11-10 Amazon Technologies, Inc. Detecting privilege escalations in code including cross-service calls
US10733085B1 (en) 2018-02-05 2020-08-04 Amazon Technologies, Inc. Detecting impedance mismatches due to cross-service calls
US10725752B1 (en) 2018-02-13 2020-07-28 Amazon Technologies, Inc. Dependency handling in an on-demand network code execution system
US10776091B1 (en) 2018-02-26 2020-09-15 Amazon Technologies, Inc. Logging endpoint in an on-demand code execution system
US10853115B2 (en) 2018-06-25 2020-12-01 Amazon Technologies, Inc. Execution of auxiliary functions in an on-demand network code execution system
US10649749B1 (en) 2018-06-26 2020-05-12 Amazon Technologies, Inc. Cross-environment application of tracing information for improved code execution
US11146569B1 (en) 2018-06-28 2021-10-12 Amazon Technologies, Inc. Escalation-resistant secure network services using request-scoped authentication information
US10949237B2 (en) 2018-06-29 2021-03-16 Amazon Technologies, Inc. Operating system customization in an on-demand network code execution system
US11099870B1 (en) 2018-07-25 2021-08-24 Amazon Technologies, Inc. Reducing execution times in an on-demand network code execution system using saved machine states
CN109344000B (zh) * 2018-08-16 2021-01-26 北京京东尚科信息技术有限公司 区块链网络服务平台、恢复工具及其故障处理方法、存储介质
CN108925310A (zh) * 2018-09-14 2018-12-04 苏州格瑞格登新材料科技有限公司 一种农业用的大棚种植全生命周期管理系统
US11243953B2 (en) 2018-09-27 2022-02-08 Amazon Technologies, Inc. Mapreduce implementation in an on-demand network code execution system and stream data processing system
US11099917B2 (en) 2018-09-27 2021-08-24 Amazon Technologies, Inc. Efficient state maintenance for execution environments in an on-demand code execution system
US11943093B1 (en) 2018-11-20 2024-03-26 Amazon Technologies, Inc. Network connection recovery after virtual machine transition in an on-demand network code execution system
US10884812B2 (en) 2018-12-13 2021-01-05 Amazon Technologies, Inc. Performance-based hardware emulation in an on-demand network code execution system
US11010188B1 (en) 2019-02-05 2021-05-18 Amazon Technologies, Inc. Simulated data object storage using on-demand computation of data objects
US11861386B1 (en) 2019-03-22 2024-01-02 Amazon Technologies, Inc. Application gateways in an on-demand network code execution system
US11119809B1 (en) 2019-06-20 2021-09-14 Amazon Technologies, Inc. Virtualization-based transaction handling in an on-demand network code execution system
US11190609B2 (en) 2019-06-28 2021-11-30 Amazon Technologies, Inc. Connection pooling for scalable network services
US11115404B2 (en) 2019-06-28 2021-09-07 Amazon Technologies, Inc. Facilitating service connections in serverless code executions
US11159528B2 (en) 2019-06-28 2021-10-26 Amazon Technologies, Inc. Authentication to network-services using hosted authentication information
US11360948B2 (en) 2019-09-27 2022-06-14 Amazon Technologies, Inc. Inserting owner-specified data processing pipelines into input/output path of object storage service
US11656892B1 (en) 2019-09-27 2023-05-23 Amazon Technologies, Inc. Sequential execution of user-submitted code and native functions
US11416628B2 (en) 2019-09-27 2022-08-16 Amazon Technologies, Inc. User-specific data manipulation system for object storage service based on user-submitted code
US11023416B2 (en) 2019-09-27 2021-06-01 Amazon Technologies, Inc. Data access control system for object storage service based on owner-defined code
US11550944B2 (en) 2019-09-27 2023-01-10 Amazon Technologies, Inc. Code execution environment customization system for object storage service
US10996961B2 (en) 2019-09-27 2021-05-04 Amazon Technologies, Inc. On-demand indexing of data in input path of object storage service
US11023311B2 (en) 2019-09-27 2021-06-01 Amazon Technologies, Inc. On-demand code execution in input path of data uploaded to storage service in multiple data portions
US11386230B2 (en) 2019-09-27 2022-07-12 Amazon Technologies, Inc. On-demand code obfuscation of data in input path of object storage service
US11394761B1 (en) 2019-09-27 2022-07-19 Amazon Technologies, Inc. Execution of user-submitted code on a stream of data
US11106477B2 (en) 2019-09-27 2021-08-31 Amazon Technologies, Inc. Execution of owner-specified code during input/output path to object storage service
US11250007B1 (en) 2019-09-27 2022-02-15 Amazon Technologies, Inc. On-demand execution of object combination code in output path of object storage service
US11263220B2 (en) 2019-09-27 2022-03-01 Amazon Technologies, Inc. On-demand execution of object transformation code in output path of object storage service
US11055112B2 (en) 2019-09-27 2021-07-06 Amazon Technologies, Inc. Inserting executions of owner-specified code into input/output path of object storage service
US10908927B1 (en) 2019-09-27 2021-02-02 Amazon Technologies, Inc. On-demand execution of object filter code in output path of object storage service
US10942795B1 (en) 2019-11-27 2021-03-09 Amazon Technologies, Inc. Serverless call distribution to utilize reserved capacity without inhibiting scaling
US11119826B2 (en) 2019-11-27 2021-09-14 Amazon Technologies, Inc. Serverless call distribution to implement spillover while avoiding cold starts
US11714682B1 (en) 2020-03-03 2023-08-01 Amazon Technologies, Inc. Reclaiming computing resources in an on-demand code execution system
US11188391B1 (en) 2020-03-11 2021-11-30 Amazon Technologies, Inc. Allocating resources to on-demand code executions under scarcity conditions
US11775640B1 (en) 2020-03-30 2023-10-03 Amazon Technologies, Inc. Resource utilization-based malicious task detection in an on-demand code execution system
US11550713B1 (en) 2020-11-25 2023-01-10 Amazon Technologies, Inc. Garbage collection in distributed systems using life cycled storage roots
US11593270B1 (en) 2020-11-25 2023-02-28 Amazon Technologies, Inc. Fast distributed caching using erasure coded object parts
US11388210B1 (en) 2021-06-30 2022-07-12 Amazon Technologies, Inc. Streaming analytics using a serverless compute system
US11968280B1 (en) 2021-11-24 2024-04-23 Amazon Technologies, Inc. Controlling ingestion of streaming data to serverless function executions
US12015603B2 (en) 2021-12-10 2024-06-18 Amazon Technologies, Inc. Multi-tenant mode for serverless code execution
CN118170762B (zh) * 2024-03-19 2024-11-01 中煤科工开采研究院有限公司 数据处理方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6640278B1 (en) * 1999-03-25 2003-10-28 Dell Products L.P. Method for configuration and management of storage resources in a storage network
US20030210672A1 (en) * 2002-05-08 2003-11-13 International Business Machines Corporation Bandwidth management in a wireless network
US6714968B1 (en) * 2000-02-09 2004-03-30 Mitch Prust Method and system for seamless access to a remote storage server utilizing multiple access interfaces executing on the remote server
CN1540524A (zh) * 2003-10-31 2004-10-27 清华大学 存储区域网络中分布式虚拟化存储的方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6343324B1 (en) * 1999-09-13 2002-01-29 International Business Machines Corporation Method and system for controlling access share storage devices in a network environment by configuring host-to-volume mapping data structures in the controller memory for granting and denying access to the devices
WO2002069151A1 (fr) * 2001-02-21 2002-09-06 Storageapps Inc. Systeme, procede et logiciel informatiques pour dispositif partage de compaction de memoire
JP3704573B2 (ja) * 2001-03-14 2005-10-12 東芝ソリューション株式会社 クラスタシステム
US7000085B2 (en) * 2001-12-28 2006-02-14 Hewlett-Packard Development Company, L.P. System and method for securing drive access to data storage media based on medium identifiers
US6848034B2 (en) * 2002-04-04 2005-01-25 International Business Machines Corporation Dense server environment that shares an IDE drive
CN1322709C (zh) * 2003-01-28 2007-06-20 华为技术有限公司 一种实现网络管理系统中配置数据组织与集中管理的方法
US20040205120A1 (en) * 2003-03-24 2004-10-14 Shaul Dar Network service optimization
JP4631301B2 (ja) * 2004-03-31 2011-02-16 株式会社日立製作所 記憶装置のキャッシュ管理方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6640278B1 (en) * 1999-03-25 2003-10-28 Dell Products L.P. Method for configuration and management of storage resources in a storage network
US6714968B1 (en) * 2000-02-09 2004-03-30 Mitch Prust Method and system for seamless access to a remote storage server utilizing multiple access interfaces executing on the remote server
US20030210672A1 (en) * 2002-05-08 2003-11-13 International Business Machines Corporation Bandwidth management in a wireless network
CN1540524A (zh) * 2003-10-31 2004-10-27 清华大学 存储区域网络中分布式虚拟化存储的方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113572837A (zh) * 2021-07-22 2021-10-29 南方电网数字电网研究院有限公司 基于分片技术的电网监控系统数据处理服务调度方法

Also Published As

Publication number Publication date
US8145873B2 (en) 2012-03-27
US20090177860A1 (en) 2009-07-09
CN1652090A (zh) 2005-08-10
CN100337218C (zh) 2007-09-12

Similar Documents

Publication Publication Date Title
WO2006089479A1 (fr) Méthode de gestion de données dans un système de stockage en réseau et système de stockage en réseau reposant sur la méthode
US11593319B2 (en) Virtualized data storage system architecture
CN106874383B (zh) 一种分布式文件系统元数据的解耦合分布方法
KR101597384B1 (ko) 분할되고 확장가능하며 사용가능한 구조적 저장소에서의 파티션 관리
CN108833503B (zh) 一种基于ZooKeeper的Redis集群方法
JP6346937B2 (ja) データベースエンジンを備えたデータベースシステム及び別個の分散型ストレージサービス
US7653699B1 (en) System and method for partitioning a file system for enhanced availability and scalability
US8156164B2 (en) Concurrent directory update in a cluster file system
US8219639B2 (en) Storage area network file system
US9424140B1 (en) Providing data volume recovery access in a distributed data store to multiple recovery agents
US20130232215A1 (en) Virtualized data storage system architecture using prefetching agent
US20050144178A1 (en) Distributing files across multiple, permissibly heterogeneous, storage devices
US20080046538A1 (en) Automatic load spreading in a clustered network storage system
CN1723434A (zh) 用于可扩展网络附接存储系统的装置和方法
US12045207B2 (en) Distributed file system that provides scalability and resiliency
US10885023B1 (en) Asynchronous processing for synchronous requests in a database
US20230393787A1 (en) Distributed File System with Disaggregated Data Management and Storage Management Layers
US20220391361A1 (en) Distributed File System with Reduced Write and Read Latencies
US20230205638A1 (en) Active-active storage system and data processing method thereof
KR20120063946A (ko) 대용량 통합 메모리를 위한 메모리 장치 및 이의 메타데이터 관리 방법
WO2017180143A1 (fr) Gestion de verrouillage distribué permettant une extensibilité
CN116700608A (zh) 一种缓存方法、装置、设备及可读存储介质
JP2005234919A (ja) クラスタメモリファイルシステム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 11884938

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 11884938

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 06705677

Country of ref document: EP

Kind code of ref document: A1

WWW Wipo information: withdrawn in national office

Ref document number: 6705677

Country of ref document: EP