CN111488321A - Management system for storage volume - Google Patents
Management system for storage volume Download PDFInfo
- Publication number
- CN111488321A CN111488321A CN202010145511.8A CN202010145511A CN111488321A CN 111488321 A CN111488321 A CN 111488321A CN 202010145511 A CN202010145511 A CN 202010145511A CN 111488321 A CN111488321 A CN 111488321A
- Authority
- CN
- China
- Prior art keywords
- file
- metadata
- task
- node
- subunit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 34
- 238000012544 monitoring process Methods 0.000 claims abstract description 27
- 238000007726 management method Methods 0.000 claims description 30
- 238000000034 method Methods 0.000 claims description 22
- 238000012986 modification Methods 0.000 claims description 15
- 230000004048 modification Effects 0.000 claims description 15
- 238000013500 data storage Methods 0.000 claims description 13
- 238000011084 recovery Methods 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 8
- 230000035876 healing Effects 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 claims description 4
- 238000004220 aggregation Methods 0.000 claims description 4
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a management system for a storage volume, which comprises a plurality of nodes, a database and an interface layer, wherein the plurality of nodes are connected with the database; each node comprises a plurality of disks, at least 1 file system and a node file management and control system; the node file management and control system is used for monitoring and collecting metadata of files in the file system on the node; the database is used for storing the metadata of the files in each file system; and the interface layer is used for managing the metadata of the database based on the service required by the user. The invention solves the problems of slow file directory structure acquisition and complex file management of the distributed file system.
Description
Technical Field
The invention relates to the field of storage management, in particular to a management system for a storage volume.
Background
The development of information technology brings about explosive growth of data, PB-scale unstructured data is more and more common, and how to effectively manage the data and further develop the value of the data becomes a problem which must be regarded by IT managers. Meanwhile, the 4V characteristic of the big data also puts higher requirements on the large capacity, high performance, easy expansion, easy use and the like of the storage system. Traditional SAN and NAS storage architectures have difficulty meeting the intensive I/O concurrent access management needs for massive amounts of data.
As shown in fig. 1, in the era of big data, internet and cloud computing, a Distributed File System (DFS) or a Network File System (NFS) is a File System that allows a File to be shared on multiple hosts through a Network, so that multiple users on multiple machines can share files and storage spaces. In such file systems, the client does not directly access the underlying data storage blocks, but communicates with the server via a network using a particular communication protocol. By means of the design of the communication protocol, the client and the server can limit the access to the file system according to the access control list or authorization. It is a combination of various storage servers interfitted by ethernet or Infiniband technologies and remote direct memory access, RDMA, resulting in a large parallel file system network. Distributed systems operate in remote data centers and have no complete control over the infrastructure, making management and monitoring more difficult than a standalone deployment. An application must be in a state of public running information, an administrator can use the information to manage and monitor the system, and the existing distributed file system has the problems of slow file directory structure acquisition, complex file management, insufficient intelligence and the like.
Disclosure of Invention
In order to solve the above-mentioned deficiencies in the prior art, the present invention provides a management system for storage volumes.
The invention provides a management system for a storage volume, which comprises:
a plurality of nodes, a database and an interface layer;
each node comprises a plurality of disks, at least 1 file system and a node file management and control system; the node file management and control system is used for monitoring and collecting metadata of files in the file system on the node;
the database is used for storing the metadata of the files in each file system;
and the interface layer is used for managing the metadata of the database based on the service required by the user.
Preferably, the node file management and control system includes: the monitoring module and the acquisition module;
the monitoring module is used for monitoring whether the file system in each node is operated or not by butting a monitoring file system component in an operating system kernel on the node, inserting the operation into an events event list in an event mode, generating an event notification of the file system based on the events event list, and generating an event warning task of the file system based on the event notification, and the monitoring module is also used for sending the event warning task to the acquisition module;
the acquisition module is used for acquiring metadata of a corresponding file system based on the event warning task and storing an acquisition result into a database;
the file system operated operation comprises the operations of creating, deleting, reading, writing and moving files in the file system; the events include creation, deletion, reading, writing, and/or movement of files.
Preferably, the acquisition module comprises:
message middleware, a task execution unit and a task execution result unit;
the message middleware is used for providing a task scheduling queue for storing the event alarm task;
the task execution unit is used for acquiring metadata of the file system based on the event alarm task in the task scheduling queue and storing an acquisition result into a database;
and the task execution result unit is used for storing the execution result of the task execution unit.
Preferably, the task execution unit includes:
the system comprises a task scheduling subunit, a plurality of metadata acquisition subunits and a data storage subunit;
the task scheduling subunit is used for acquiring the event alarm task from the message middleware and scheduling the metadata acquisition subunit to execute the event alarm task;
the metadata acquisition subunit is used for acquiring metadata of a file system based on the tasks distributed by the task scheduling subunit and sending the metadata of the file system to the data storage subunit;
the data storage subunit is used for storing the acquisition result of the metadata acquisition subunit into a database;
the metadata of the file system comprises the name, the type, the path, the size, the volume, the node, the file system, the creation time and the modification time of the file.
Preferably, the task scheduling subunit is further configured to determine the scheduled metadata acquisition subunit by determining the busy level of each metadata acquisition subunit and the number of times of executing the task before scheduling the metadata acquisition subunit.
Preferably, the interface layer includes: the device comprises a statistic unit, a retrieval unit and a data curing unit;
the statistical unit is used for counting the file types, the file quantities and the file distribution in the database;
the retrieval unit is used for retrieving in the database according to the name, type, path, size, volume, node, file system, creation time and/or modification time of the file;
the data recovery unit is used for determining the original and the copy of the damaged file according to the file name, the type, the path and the volume where the damaged file is located when the damaged file on the node needs to be recovered, selecting a recovery basis according to the creation time, the modification time and the size of the damaged file, selecting a strategy according to the creation time old and new of the damaged file, the modification time old and new and the size of the damaged file, and finally recovering the damaged file through an rsync tool.
Preferably, the statistical unit includes:
the file type counting subunit is used for calling an aggregate method to cluster in the database based on the type field and counting the number of all types of files in the metadata;
the file quantity counting subunit is used for searching all file metadata in the database and counting the quantity of all files by using a count algorithm;
and the file distribution counting subunit is used for calling an aggregation method to cluster in the database according to the node field where the file is located, the volume field where the file is located and the file field where the file is located, and counting the number of all files under the node, the number of files in the volume, the number of files in the file system and the position of the positive copy in the distributed file system.
Preferably, the interface layer further includes:
and the scoring unit is used for scoring according to the cluster file inconsistency proportion and the file split-brain proportion to obtain a scoring result after the node is down or the network is abnormal.
Preferably, the interface layer further includes:
and the data display unit is used for displaying the statistical result, the retrieval result and the healing result.
Compared with the prior art, the invention has the beneficial effects that:
the management system provided by the invention comprises a plurality of nodes, a database and an interface layer; each node comprises a plurality of disks, at least 1 file system and a node file management and control system; the node file management and control system is used for monitoring and collecting metadata of files in the file system on the node; the database is used for storing the metadata of the files in each file system; the interface layer is used for managing metadata of a database based on service required by a user, the disks of a plurality of nodes forming the distributed file system are virtualized to obtain a plurality of file systems, and the metadata of files in the file systems are managed and controlled, so that the problem that the distributed file systems are slow in acquiring file directory structures is solved, and the problem of file management in the file systems is simplified.
Drawings
FIG. 1 is a diagram illustrating a typical distributed file management system;
FIG. 2 is a diagram of a system for managing storage volumes according to the present invention.
Detailed Description
For a better understanding of the present invention, reference is made to the following description taken in conjunction with the accompanying drawings and examples.
With the development of distributed storage, the standardization process of the storage industry is also promoted continuously, and the intelligent volume management technology preferentially adopts an industry standard interface for storage access. On the platform level, the heterogeneous storage resources are abstracted, and traditional storage device-level operations are packaged into storage resource-oriented operations, so that the operations of the heterogeneous storage infrastructure are simplified, the whole storage life cycle process such as creation, change and recovery can be automatically executed, and each member part in the whole process can have some states, such as own responsible module, own load condition, the grasp of some data and the like. These data, which are related to other processes, become very important in file monitoring, data statistics, data recovery, etc. As shown in fig. 2, according to the technical architecture of the above-mentioned intelligent volume management technology, the present invention provides a management system for a storage volume, including: a plurality of nodes, a database and an interface layer;
each node comprises a plurality of disks, at least 1 file system and a node file management and control system; the node file management and control system is used for monitoring and collecting metadata of files in the file system on the node;
the database is used for storing the metadata of the files in each file system;
and the interface layer is used for managing the metadata of the database based on the service required by the user.
The management system provided by the invention is based on virtualizing a plurality of disks of each node into different file systems, collects and monitors all the file systems by the existing collection technology, stores metadata of the adding, deleting and modifying events of the file systems by mogodb, and finally comprehensively analyzes the metadata by using different modules of a python algorithm, thereby realizing the intelligent management functions of file metadata monitoring, file metadata quick management, massive file quick retrieval, file type statistics, file quantity statistics, file distribution statistics, cluster health scoring, file recovery after damage and the like of a distributed file system volume, and solving the problems of slow file directory structure and complex file management of the distributed file system. Wherein the MongoDB is a database based on distributed file storage.
The virtualization of the disk comprises the steps of establishing the disk into RAID, then establishing a VolumGroup, wherein the process is also called a resource pooling process of the disk, creating L VM in the VolumGroup, then creating a file system in L VM, wherein a storage volume is established on the created file system, then a user can store files on the storage volume, and a monitoring module can monitor and acquire metadata of the files as long as the files operate in the storage volume.
In the distributed file system, each node has a plurality of file systems, and each node has a monitoring module and an acquisition module, the monitoring module monitors the change of the file system in each node by interfacing with an operating system kernel on the node to monitor a file system component, in this embodiment, taking the L inux operating system as an example, when operations such as creation, deletion, reading, writing, moving and the like of a file occur, the kernel inserts an event into an events event list, then wakes up a program waiting for the acquisition module, and sends a related event alarm task to the acquisition module.
And the acquisition module is used for acquiring metadata of the file system based on the event warning task sent by the monitoring module and storing an acquisition result into a database.
And integrating the metadata of a plurality of disks in the nodes into a resource pool, and virtualizing the metadata into a plurality of file systems by the resource pool according to classification rules.
The acquisition module in the present invention uses a producer-consumer model;
the collection module includes: the system comprises message middleware, a task execution unit and a task execution result unit.
Message middleware: the message middle person is a task scheduling queue, is an independent service and is a producer consumer mode, the monitoring module is used as a producer to put an event warning task of the file system into the task scheduling queue, and the consumer takes out the task from the task scheduling queue to execute;
a task execution unit: the event warning task is taken out from a task scheduling queue, metadata of a file system is collected based on the event warning task, and a collection result is stored in a database;
task execution result unit: since the execution of the task is separated from the main program, if the main program wants to obtain the result of the task execution, it must be stored through the middleware. The event alarm task execution module is used for storing the result of the event alarm task after execution so as to process the failed task.
The task execution unit in the invention comprises: the system comprises a task scheduling subunit, a plurality of metadata acquisition subunits and a data storage subunit; the method mainly realizes task scheduling, metadata acquisition and data storage.
The task scheduling subunit is used for acquiring an event alarm task from the message middleware, selecting the metadata acquisition subunit by judging the busy degree and the task execution times of each metadata acquisition subunit, and scheduling the event alarm to the selected metadata acquisition subunit;
the metadata acquisition subunit is used for acquiring metadata of a file system based on the tasks distributed by the task scheduling subunit and sending the metadata of the file system to the data storage subunit;
the data storage subunit is configured to store the metadata of the file system corresponding to the event alert task in a database;
the metadata of the file system includes: the name, type, path, size, volume, node, file system, creation time, and modification time of the file.
The specific process of the monitoring module and the acquisition module working cooperatively in this embodiment includes:
firstly, a monitoring module is used as a producer to monitor the event notification of the file system in real time and put the event notification into a message middleware.
And secondly, the task execution unit is used as a consumer, and after the task scheduling subunit receives the task, the task scheduling subunit schedules the metadata acquisition subunit to execute the task in the queue, acquires metadata of the file system according to the task, and transmits the acquired metadata to the data storage subunit.
And thirdly, the task passes through the data storage subunit, and finally the data is stored in a distributed database.
And fourthly, after the task execution unit finishes executing, placing the task execution result in a task execution result unit, finishing the task execution if the task execution is successful, and specifically analyzing the file if the task execution is failed.
The interface layer of the invention comprises: and the statistical unit is used for carrying out clustering algorithm based on the contents of the names, types, paths, sizes, volumes, nodes, file systems, creation time, modification time and the like of the files stored in the database, and counting the file types, the file quantities and the file distribution.
Further, the statistical unit specifically includes:
a file type statistics subunit: the method is used for calling an aggregation method for clustering in the database according to the type field, and the number of all types of files in the metadata can be counted.
A file number counting subunit: the method is used for searching all file metadata in the database, and the count algorithm is carried out, so that all the file quantity can be crowded.
A file distribution statistics subunit: the method is used for calling an aggregation method for clustering in a database according to the fields of the nodes where the files are located, the fields of the volumes where the files are located and the fields of the file systems where the files are located, so that the number of all the files under the nodes, the number of the files in the volumes, the number of the files in the file systems, the positions of the positive copies in the distributed file systems and the like can be counted.
The interface layer of the invention also comprises: and the retrieval unit is used for retrieving the massive files stored in the database according to fields such as name, type, path, size, volume, node, file system, creation time, modification time and the like, so that the retrieval in the file system is replaced, and the retrieval efficiency is greatly improved. After the file is searched, the node, the volume and the path of the file can be quickly known according to the metadata of the file, and then the file can be deleted and downloaded in batches.
The interface layer of the invention also comprises: the scoring unit is used for scoring according to the cluster file inconsistency proportion and the file split-brain proportion, counting the number of inconsistent file systems and the number of split-brain files through a distributed file system tool, and scoring according to the number proportion, the file new and old and important levels. After scoring, the user can know the health status of the cluster according to the score and decide whether to initiate healing.
The interface layer of the invention also comprises: the recovery unit is used for storing the name, type, path, size, volume, node, file system, creation time, modification time and other contents of the file in the database, acquiring the original and copy positions of the damaged file in the file system according to the file name, type, path and volume when the file damage of the node needs to be cured, and selecting a recovery basis according to the creation time, modification time and size of the damaged file; and then selecting a strategy according to the creation time old and new of the damaged file, the modification time old and new and the size, and finally restoring the damaged file through an rsync tool, namely copying the selected correct file from the correct file system to the damaged file system through the rsync tool, thereby realizing the quick restoration of the service. Because a distributed file system is built on top of each file system, a unified file is written in multiple file systems if it is a replica volume.
The middle interface layer also provides a data display unit for obtaining and displaying a request result from the database based on the request of the client, and displaying a scoring result, a healing result, a retrieval result and the like.
The intelligent volume management method provided by the invention comprehensively uses the bottom technical algorithms such as the acquisition technology, the file system synchronization technology, the metadata storage technology, python and the like to realize the storage of the volume, and can realize the file metadata monitoring, the file metadata quick management, the mass file quick retrieval, the file type statistics, the file quantity statistics, the file distribution statistics, the cluster health scoring, the file damage healing and the like through the real-time monitoring of the file system, thereby having the following functional advantages:
(1) high performance, easy deployment and easy use;
(2) simple, flexible and reliable;
(3) the method is suitable for environments such as cloud storage and enterprise-level distributed storage;
(4) the data management performance of the distributed file system is improved;
(5) the running state of the file system is monitored in real time, and the operation and maintenance capacity is improved;
(6) the data are synchronized and cured quickly, and the data security is improved;
(7) the availability of the service data is improved;
(8) the complexity of storage planning and management is reduced.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The present invention is not limited to the above embodiments, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the scope of the claims of the present invention which are filed as the application.
Claims (9)
1. A system for managing storage volumes, comprising:
a plurality of nodes, a database and an interface layer;
each node comprises a plurality of disks, at least 1 file system and a node file management and control system; the node file management and control system is used for monitoring and collecting metadata of files in the file system on the node;
the database is used for storing the metadata of the files in each file system;
and the interface layer is used for managing the metadata of the database based on the service required by the user.
2. The system of claim 1, wherein the node file administration system comprises: the monitoring module and the acquisition module;
the monitoring module is used for monitoring whether the file system in each node is operated or not by butting a monitoring file system component in an operating system kernel on the node, inserting the operation into an events event list in an event mode, generating an event notification of the file system based on the events event list, and generating an event warning task of the file system based on the event notification, and the monitoring module is also used for sending the event warning task to the acquisition module;
the acquisition module is used for acquiring metadata of a corresponding file system based on the event warning task and storing an acquisition result into a database;
the file system operated operation comprises the operations of creating, deleting, reading, writing and moving files in the file system; the events include creation, deletion, reading, writing, and/or movement of files.
3. The system of claim 2, wherein the acquisition module comprises:
message middleware, a task execution unit and a task execution result unit;
the message middleware is used for providing a task scheduling queue for storing the event alarm task;
the task execution unit is used for acquiring metadata of the file system based on the event alarm task in the task scheduling queue and storing an acquisition result into a database;
and the task execution result unit is used for storing the execution result of the task execution unit.
4. The system of claim 3, wherein the task execution unit comprises:
the system comprises a task scheduling subunit, a plurality of metadata acquisition subunits and a data storage subunit;
the task scheduling subunit is used for acquiring the event alarm task from the message middleware and scheduling the metadata acquisition subunit to execute the event alarm task;
the metadata acquisition subunit is used for acquiring metadata of a file system based on the tasks distributed by the task scheduling subunit and sending the metadata of the file system to the data storage subunit;
the data storage subunit is used for storing the acquisition result of the metadata acquisition subunit into a database;
the metadata of the file system comprises the name, the type, the path, the size, the volume, the node, the file system, the creation time and the modification time of the file.
5. The system of claim 3, wherein the task scheduling subunit is further configured to determine the scheduled metadata collection subunit by determining how busy each metadata collection subunit is and how many times to execute the task before scheduling the metadata collection subunit.
6. The system of claim 4, wherein the interface layer comprises: the device comprises a statistic unit, a retrieval unit and a data curing unit;
the statistical unit is used for counting the file types, the file quantities and the file distribution in the database;
the retrieval unit is used for retrieving in the database according to the name, type, path, size, volume, node, file system, creation time and/or modification time of the file;
the data recovery unit is used for determining the original and the copy of the damaged file according to the file name, the type, the path and the volume where the damaged file is located when the damaged file on the node needs to be recovered, selecting a recovery basis according to the creation time, the modification time and the size of the damaged file, selecting a strategy according to the creation time old and new of the damaged file, the modification time old and new and the size of the damaged file, and finally recovering the damaged file through an rsync tool.
7. The system of claim 6, wherein the statistics unit comprises:
the file type counting subunit is used for calling an aggregate method to cluster in the database based on the type field and counting the number of all types of files in the metadata;
the file quantity counting subunit is used for searching all file metadata in the database and counting the quantity of all files by using a count algorithm;
and the file distribution counting subunit is used for calling an aggregation method to cluster in the database according to the node field where the file is located, the volume field where the file is located and the file field where the file is located, and counting the number of all files under the node, the number of files in the volume, the number of files in the file system and the position of the positive copy in the distributed file system.
8. The system of claim 6, wherein the interface layer further comprises:
and the scoring unit is used for scoring according to the cluster file inconsistency proportion and the file split-brain proportion to obtain a scoring result after the node is down or the network is abnormal.
9. The system of claim 6, wherein the interface layer further comprises:
and the data display unit is used for displaying the statistical result, the retrieval result and the healing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010145511.8A CN111488321A (en) | 2020-03-05 | 2020-03-05 | Management system for storage volume |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010145511.8A CN111488321A (en) | 2020-03-05 | 2020-03-05 | Management system for storage volume |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111488321A true CN111488321A (en) | 2020-08-04 |
Family
ID=71794369
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010145511.8A Pending CN111488321A (en) | 2020-03-05 | 2020-03-05 | Management system for storage volume |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111488321A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380163A (en) * | 2020-10-20 | 2021-02-19 | 广州西山居世游网络科技有限公司 | S3 file system space occupation monitoring method and system |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102004769A (en) * | 2010-11-12 | 2011-04-06 | 成都市华为赛门铁克科技有限公司 | File management method, equipment and memory system |
| CN102307221A (en) * | 2011-03-25 | 2012-01-04 | 国云科技股份有限公司 | Cloud storage system and implementation method thereof |
| CN102624911A (en) * | 2012-03-14 | 2012-08-01 | 中山大学 | Cluster-based visible media storage system |
| CN104268061A (en) * | 2014-09-12 | 2015-01-07 | 国云科技股份有限公司 | Storage state monitoring mechanism for virtual machine |
| CN105718304A (en) * | 2016-01-25 | 2016-06-29 | 汉柏科技有限公司 | Virtual machine management method and system |
| CN107547653A (en) * | 2017-09-11 | 2018-01-05 | 华北水利水电大学 | A kind of distributed file storage system |
| CN109327539A (en) * | 2018-11-15 | 2019-02-12 | 上海天玑数据技术有限公司 | A kind of distributed block storage system and its data routing method |
| CN109447048A (en) * | 2018-12-25 | 2019-03-08 | 苏州闪驰数控系统集成有限公司 | A kind of artificial intelligence early warning system |
| CN109558215A (en) * | 2018-12-10 | 2019-04-02 | 深圳市木浪云数据有限公司 | Backup method, restoration methods, device and the backup server cluster of virtual machine |
| CN109756366A (en) * | 2018-12-24 | 2019-05-14 | 上海欣方智能系统有限公司 | System is realized in intelligent network SCP cloud service based on CAAS |
| CN110213352A (en) * | 2019-05-17 | 2019-09-06 | 北京航空航天大学 | The unified Decentralized Autonomous storage resource polymerization of name space |
| CN110415831A (en) * | 2019-07-18 | 2019-11-05 | 天宜(天津)信息科技有限公司 | A kind of medical treatment big data cloud service analysis platform |
-
2020
- 2020-03-05 CN CN202010145511.8A patent/CN111488321A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102004769A (en) * | 2010-11-12 | 2011-04-06 | 成都市华为赛门铁克科技有限公司 | File management method, equipment and memory system |
| CN102307221A (en) * | 2011-03-25 | 2012-01-04 | 国云科技股份有限公司 | Cloud storage system and implementation method thereof |
| CN102624911A (en) * | 2012-03-14 | 2012-08-01 | 中山大学 | Cluster-based visible media storage system |
| CN104268061A (en) * | 2014-09-12 | 2015-01-07 | 国云科技股份有限公司 | Storage state monitoring mechanism for virtual machine |
| CN105718304A (en) * | 2016-01-25 | 2016-06-29 | 汉柏科技有限公司 | Virtual machine management method and system |
| CN107547653A (en) * | 2017-09-11 | 2018-01-05 | 华北水利水电大学 | A kind of distributed file storage system |
| CN109327539A (en) * | 2018-11-15 | 2019-02-12 | 上海天玑数据技术有限公司 | A kind of distributed block storage system and its data routing method |
| CN109558215A (en) * | 2018-12-10 | 2019-04-02 | 深圳市木浪云数据有限公司 | Backup method, restoration methods, device and the backup server cluster of virtual machine |
| CN109756366A (en) * | 2018-12-24 | 2019-05-14 | 上海欣方智能系统有限公司 | System is realized in intelligent network SCP cloud service based on CAAS |
| CN109447048A (en) * | 2018-12-25 | 2019-03-08 | 苏州闪驰数控系统集成有限公司 | A kind of artificial intelligence early warning system |
| CN110213352A (en) * | 2019-05-17 | 2019-09-06 | 北京航空航天大学 | The unified Decentralized Autonomous storage resource polymerization of name space |
| CN110415831A (en) * | 2019-07-18 | 2019-11-05 | 天宜(天津)信息科技有限公司 | A kind of medical treatment big data cloud service analysis platform |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380163A (en) * | 2020-10-20 | 2021-02-19 | 广州西山居世游网络科技有限公司 | S3 file system space occupation monitoring method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10795905B2 (en) | Data stream ingestion and persistence techniques | |
| US10691716B2 (en) | Dynamic partitioning techniques for data streams | |
| US10467105B2 (en) | Chained replication techniques for large-scale data streams | |
| US11349917B2 (en) | Replication handling among distinct networks | |
| CA2930026C (en) | Data stream ingestion and persistence techniques | |
| US20220083245A1 (en) | Declarative provisioning of storage | |
| US11360689B1 (en) | Cloning a tracking copy of replica data | |
| US9471585B1 (en) | Decentralized de-duplication techniques for largescale data streams | |
| CA2929776C (en) | Client-configurable security options for data streams | |
| EP3069274B1 (en) | Managed service for acquisition, storage and consumption of large-scale data streams | |
| US11995336B2 (en) | Bucket views | |
| US20210019063A1 (en) | Utilizing data views to optimize secure data access in a storage system | |
| WO2021050875A1 (en) | Configurable data replication | |
| CN111133409A (en) | Ensuring reproducibility in artificial intelligence infrastructure | |
| US11442652B1 (en) | Replication handling during storage system transportation | |
| WO2019068002A1 (en) | Rule-based autonomous database cloud service framework | |
| CN115668172A (en) | Managing host mapping of replication endpoints | |
| US20150227602A1 (en) | Virtual data backup | |
| US20150134626A1 (en) | Partition-based data stream processing framework | |
| CN114041112A (en) | Virtual storage system architecture | |
| Fu et al. | Real-time data infrastructure at uber | |
| US20210055885A1 (en) | Enhanced data access using composite data views | |
| US9984139B1 (en) | Publish session framework for datastore operation records | |
| WO2019226652A1 (en) | Auto-scaling a software application | |
| US20210165760A1 (en) | Managing Dependent Delete Operations among Data Stores |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |